Skip to content
This repository has been archived by the owner on Aug 17, 2023. It is now read-only.

[WIP] Add ConfigMap builder #21

Closed
wants to merge 2 commits into from

Conversation

wbuchwalter
Copy link
Contributor

@wbuchwalter wbuchwalter commented Nov 22, 2018

Re-implementation of #15 following the refactoring in #20.
cc @ashahab

In this current state, there is now way to specify different base images for worker and ps.
I will address that in a separate PR, as this need to be solved for all builders not just this one.

This PR also contains some unrelated changes such as renaiming native in kubernetes.


This change is Reviewable

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: wbuchwalter

If they are not already assigned, you can assign the PR to them by writing /assign @wbuchwalter in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@wbuchwalter wbuchwalter changed the title Add ConfigMap builder [WIP] Add ConfigMap builder Nov 22, 2018

def __init__(self, notebook_name=None):
if notebook_name is None:
self.notebook_name = notebook_helper.get_notebook_name()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work? If not, we should make notebook_name required.

Copy link
Contributor Author

@wbuchwalter wbuchwalter Nov 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works in standalone notebooks, not with JupyterHub.
I can improve the error handling in the helper to display an explicit error when user JH such as: "notebook_name is required when using JupyterHub"

client.V1ConfigMap(
api_version="v1",
data={"code.py": code},
metadata=client.V1ObjectMeta(name=self.job_id)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is job_id a parameter or member? Who sets it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

job_id is generated automatically by the Deployment and is used to ensure we get the correct logs.

k8s.create_config_map(namespace, config_map)
return self._generate_pod_spec(job_id, base_image)

def _generate_pod_spec(self, job_id, base_image):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it would be this PR but would be great to provide this pod-spec using a configmap(that way specifying images, environments, and other mounts is completely upto the user).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be part of the discussion here: #28 and be in it's own PR.

@jlewi
Copy link
Contributor

jlewi commented Jan 7, 2019

@wbuchwalter Any update on this?

@wbuchwalter
Copy link
Contributor Author

@jlewi Still not sure about this one. I would like to avoid having a ton of different builders if we can converge on one.
@ashahab, as far as I know your use case is the only one requiring this. I would like confirmation that the append builder definitly cannot suit your needs.
If not, then would there be any drawback to publish this builder as an external package? Having this on an external repo and as an additional pip package?

@ashahab
Copy link

ashahab commented Jan 7, 2019 via email

@jlewi
Copy link
Contributor

jlewi commented Mar 28, 2019

I think this is obsolete.
Lets reopen if still relevant.

@jlewi jlewi closed this Mar 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants