[WIP] Reworked bootstrapper #262

GiriSrini · 2021-11-22T18:04:09Z

Removed the files from the outdated bootstrapper implementation.
Folded quickstart/ files into bootstrapper/ & replaced all references to quickstart/.
Both Kubernetes & Docker Compose setups are initialized with bootstrapper/init_catalog.sh, which takes options for host, port, & disabling KFP functionality from the user.

TO DO:

Rewrite Kubernetes instructions in bootstrapper/README.md.

- Removed the files from the outdated bootstrapper implementation. - Folded `quickstart/` files into `bootstrapper/` & replaced all references to `quickstart/`. - Both Kubernetes & Docker Compose setups are initialized with `bootstrapper/init_catalog.sh`, which takes options for host, port, & disabling KFP functionality from the user. TO DO: - Rewrite Kubernetes instructions in `bootstrapper/README.md`.

mlx-bot · 2021-11-22T18:04:12Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: girigsrini
To complete the pull request process, please assign animeshsingh after the PR has been reviewed.
You can assign the PR to them by writing /assign @animeshsingh in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ckadner

Thanks Giri. Sorry for not responding sooner. We need to make a few more changes.

The Quickstart guide should stay in place. The only change under quickstart will be how the catalog gets initialized

ckadner · 2021-12-02T00:26:35Z

README.md


 For a full deployment, we use [Kubeflow Kfctl](https://github.com/kubeflow/kfctl) tooling. 

-* #### [MLX using Docker Compose (Asset Catalog Only)](./quickstart)
+* #### [MLX using Docker Compose (Asset Catalog Only)](./bootstrapper)


these 2 links should still point to ./quickstart since the Quickstart as a concept and easy-install remains in place

ckadner · 2021-12-02T00:27:25Z

api/README.md

@@ -100,7 +100,7 @@ Bring up the Quickstart without the `mlx-api` service, since we will run the MLX
 from our local source code, instead of using the pre-built Docker image `mlexchange/mlx-api:nightly-main`.

    # cd <mlx_root_dir>
-    cd quickstart
+    cd bootstrapper


this folder needs to stay quickstart since that is where the docker compose app needs to start

ckadner · 2021-12-02T00:29:18Z

bootstrapper/README.md

-   After 2-5 minutes, the assets in [configmap.yaml](configmap.yaml) should be populated.
+# Bootstrapper
+
+## Running MLX with Docker Compose


the bootstrapper/README.md should stay in place (with some modifications). it should not be replaced by the quickstart/README.md

ckadner · 2021-12-02T00:30:25Z

bootstrapper/README.md

@@ -1,12 +1,121 @@
-# How to use it
-1. Install MLX


the install step should point to one of 3 options, like on the main README where we have kubernetes and Docker compose deployment options

ckadner · 2021-12-02T00:32:32Z

bootstrapper/README.md

@@ -1,12 +1,121 @@
-# How to use it
-1. Install MLX
-2. Get the [github.ibm.com personal access token](https://github.ibm.com/settings/tokens/new) and give it access to read all public repos.


the remaining steps should describe how to run the bootstrapper script, i.e. set the host and port to point to the MLX server (either remote cluster or local Quickstart, but the local Quickstart catalog should be described in the quickstart/README.md)

ckadner · 2021-12-02T00:33:33Z

dashboard/origin-mlx/README.md

@@ -95,7 +95,7 @@ from our local source code, instead of using the pre-built Docker image `mlexcha

 ```Bash
 # cd <mlx_root_directory>
-cd quickstart
+cd bootstrapper


stays quickstart

ckadner · 2021-12-02T02:40:48Z

I think we should separate the changes for the bootstrapper and quickstart, starting by adding your new init_catalog.sh script under the bootstrapper folder.

The main idea for this PR should be that the bootstrapper folder would just keep the bootstrapper/catalog_upload.json and add a small Python or Bash script to call POST '/apis/v1alpha1/catalog' API endpoint with the catalog_upload.json file as payload.

We can then remove all of these files:

bootstrapper/bootstrap.yaml
bootstrapper/configmap.yaml
bootstrapper/Dockerfile
bootstrapper/start.py

We will keep:

bootstrapper/README.md
bootstrapper/catalog_upload.json

And add a script to call the /catalog API:

bootstrapper/init_catalog.sh

For the updates/simplifications to Quickstart we would open a follow up PR.

There are two recent changes that should make that rework easier:

Feature all assets from catalog upload #256 --> makes .../publish_approved and .../featured API calls obsolete
Added upload-from-url to catalog service #259 --> makes it obsolete for quickstart/ to have a copy of catalog_upload.json

So in the quickstart folder we would:

in the docker-compose.yaml:
- change the catalog service entrypoint/command to call (curl) the mlx-api service (localhost:8080) at the /catalog/upload_from_url API endpoint with the URL of https://raw.githubusercontent.com/machine-learning-exchange/mlx/main/bootstrapper/catalog_upload.json
- and remove the volumes for /catalog_upload.json and ./init_catalog.sh:/init_catalog.sh
remove the files quickstart/init_catalog.sh and quickstart/catalog_upload.json

ckadner · 2021-12-02T18:19:43Z

@girigsrini -- I took a look at the changes required for the Quickstart guide (files under quickstart) and they turned out to be quite comprehensive. So that should definitely be a separate PR.

For this PR, let's focus on changes under the bootstrapper folder as I described above

mlx-bot requested review from drewbutlerbb4 and Tomcli November 22, 2021 18:04

ckadner added the RCOS Potential work items for RCOS student interns label Dec 1, 2021

ckadner requested review from ckadner and Tomcli and removed request for Tomcli and drewbutlerbb4 December 2, 2021 00:22

ckadner linked an issue Dec 2, 2021 that may be closed by this pull request

Rework bootstrapper #23

Open

ckadner requested changes Dec 2, 2021

View reviewed changes

ckadner marked this pull request as draft December 2, 2021 02:26

mlx-bot added the do-not-merge/work-in-progress label Dec 2, 2021

ckadner assigned GiriSrini Dec 2, 2021

ckadner changed the title ~~Reworked bootstrapper~~ [WIP] Reworked bootstrapper Dec 2, 2021

ckadner mentioned this pull request Dec 2, 2021

Simplify Quickstart #281

Merged

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Reworked bootstrapper #262

[WIP] Reworked bootstrapper #262

GiriSrini commented Nov 22, 2021

mlx-bot commented Nov 22, 2021

ckadner left a comment

ckadner Dec 2, 2021

ckadner Dec 2, 2021

ckadner Dec 2, 2021

ckadner Dec 2, 2021

ckadner Dec 2, 2021

ckadner Dec 2, 2021

ckadner commented Dec 2, 2021 •

edited

Loading

ckadner commented Dec 2, 2021

[WIP] Reworked bootstrapper #262

[WIP] Reworked bootstrapper #262

Conversation

GiriSrini commented Nov 22, 2021

mlx-bot commented Nov 22, 2021

ckadner left a comment

Choose a reason for hiding this comment

ckadner Dec 2, 2021

Choose a reason for hiding this comment

ckadner Dec 2, 2021

Choose a reason for hiding this comment

ckadner Dec 2, 2021

Choose a reason for hiding this comment

ckadner Dec 2, 2021

Choose a reason for hiding this comment

ckadner Dec 2, 2021

Choose a reason for hiding this comment

ckadner Dec 2, 2021

Choose a reason for hiding this comment

ckadner commented Dec 2, 2021 • edited Loading

ckadner commented Dec 2, 2021

ckadner commented Dec 2, 2021 •

edited

Loading