Batch Processor Component #1915
Conversation
…sors_gang_scheduler
|
Mon Jun 8 11:17:42 UTC 2020 impatient try |
|
Mon Jun 8 11:19:06 UTC 2020 impatient try |
|
The kubeflow pipelines notes is still badly formatted. |
|
@cliveseldon I just had a look, the notebook has been updated to only have 2nd level headers, did you have a look at the latest file? Maybe the notebook where the comments are is cached to the previous changes |
|
Mon Jun 8 15:17:36 UTC 2020 impatient try |
|
Mon Jun 8 15:19:08 UTC 2020 impatient try |
|
@cliveseldon @adriangonz @RafalSkolasinski seems all the comments have been addressed, is there any further changes required? Happy to jump on it today if there's anything outstanding |
|
Would be good to confirm that argo example can run in |
|
@RafalSkolasinski I think you had mentioned it had fixed the RBAC errors with that command provided above? It seems it's just a standard RBAC issue, not a KIND specific issue. Would you be able to have a look? |
|
RBAC errors went away but still had some troubles running it - will give it another shot to verify if there is a |
|
Tue Jun 9 12:50:37 UTC 2020 impatient try |
|
Ok I have been able to run it in KIND, and it seems that the main issue is that Argo by default tries to mount "/var/run/docker.sock" for all workflows, which I don't really understand why as it seems quite bad practice. Due to this I added a note outlining that the example won't work on KIND as that issue is one that is specific to Argo on KIND. |
|
Tue Jun 9 12:52:26 UTC 2020 impatient try |
| Use the notebook to [set-up Seldon Core with Ambassador or Istio Ingress](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html). | ||
|
|
||
| Note: This example won't work in KIND (Kubernetes in Docker) due to a known requirement from Argo which mounts the docker socket: https://github.com/argoproj/argo/issues/826 | ||
|
|
There was a problem hiding this comment.
I managed to get this running with fix in argoproj/argo-workflows#2376 (comment)
Can you add thos instructions for Kind.
There was a problem hiding this comment.
Nice, thanks - ok I've added the comment
|
@axsaucedo I added comments on getting runnimg in Kind and a small fix for log output. Apart from that now got notebook to run e2e. |
|
@cliveseldon awesome, ok I've added the changes suggested, should be good to go 🚀 |
|
Tue Jun 9 17:21:45 UTC 2020 impatient try |
|
Tue Jun 9 17:21:50 UTC 2020 impatient try |
|
Nice one on the argo example! I only think that clusterrole required is Also the SeldonCore notebook setup leaves environment with set namespace being Besides that it works pretty well! |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: axsaucedo The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This PR contains the updated implementation of #1714 with the ability to be used in workflow managers. In contrast with previous PR, this one does not implement the data download step within the batch processor and expects the data download steps to be carried out by a separate container.
PR shows example with Argo Workflows and Kubeflow Pipelines.