[WIP] Reworked bootstrapper

README.md

@@ -31,11 +31,11 @@ Additionally it provides:
 ## 1. Deployment
 <img src="docs/images/mlx-architecture-4.png" height="40%" width="40%">
 
-For a simple up-and-running MLX with asset catalog only, we created a [Quickstart Guide](./quickstart) using [Docker Compose](https://docs.docker.com/compose/install/).
+For a simple up-and-running MLX with asset catalog only, we created a [Quickstart Guide](./bootstrapper) using [Docker Compose](https://docs.docker.com/compose/install/).
 
 For a full deployment, we use [Kubeflow Kfctl](https://github.com/kubeflow/kfctl) tooling. 
 
-* #### [MLX using Docker Compose (Asset Catalog Only)](./quickstart)
+* #### [MLX using Docker Compose (Asset Catalog Only)](./bootstrapper)
 
 * #### [MLX Deployment on Kubernetes or OpenShift](./docs/mlx-setup.md)
 

api/README.md

@@ -4,7 +4,7 @@ An extension to the Kubeflow Pipeline API for Components and Models
 
 ---
 
-# Quickstart    
+# Quickstart
 
 ## Deploy to Kubernetes
 
@@ -100,7 +100,7 @@ Bring up the Quickstart without the `mlx-api` service, since we will run the MLX
 from our local source code, instead of using the pre-built Docker image `mlexchange/mlx-api:nightly-main`.
 
     # cd <mlx_root_dir>
-    cd quickstart
+    cd bootstrapper
 
     docker compose --project-name  no_api   up   minio miniosetup mysql mlx-ui
 
@@ -177,5 +177,5 @@ After testing or debugging your code changes, bring down the Swagger Server
 to populate the MLX asset catalog
 
     # cd <mlx_root_directory>
-    cd quickstart
+    cd bootstrapper
     ./init_catalog.sh

api/examples/catalog_api.py

@@ -23,7 +23,7 @@
 
 api_base_path = 'apis/v1alpha1'
 
-catalog_upload_file = "./../../bootstrapper/catalog_upload.json"
+catalog_upload_file = "../../bootstrapper/catalog_upload.json"
 
 IBM_GHE_API_TOKEN = env.get("IBM_GHE_API_TOKEN")
 

bootstrapper/README.md

@@ -1,12 +1,121 @@
-# How to use it
-1. Install MLX
-2. Get the [github.ibm.com personal access token](https://github.ibm.com/settings/tokens/new) and give it access to read all public repos.
-3. Fillin the below environment variables in [bootstrap.yaml](bootstrap.yaml):
-   - **enterprise_github_token**: github.ibm.com personal access token from step 2.
-4. Deploy boostrapper:
-   ```shell
-   kubectl apply -f bootstrapper/bootstrap.yaml -n kubeflow
-   kubectl apply -f bootstrapper/configmap.yaml -n kubeflow 
-   ```
-
-   After 2-5 minutes, the assets in [configmap.yaml](configmap.yaml) should be populated.
+# Bootstrapper
+
+## Running MLX with Docker Compose
+
+The "quickstart" setup uses [Docker Compose](https://docs.docker.com/compose/)
+to bring up the MLX API server and the MLX Dashboard, together with the MySQL
+database and Minio S3 storage backend.
+In this configuration, the preloaded asset catalog of Components, Datasets, Models,
+Notebooks and Pipelines can be browsed, asset metadata and sample code can be
+downloaded and new assets can be registered. Sample pipeline code can be generated
+for each asset type, however their execution on a Kubeflow Pipelines (KFP) cluster
+is not enabled.
+In a Kubernetes cluster deployment of MLX, _Pipelines_ are registered using
+the KFP API and metadata storage is managed by KFP. In this Docker Compose setup
+the _Pipelines_ are stored in Minio and MySQL by the MLX API server.
+
+## Limitations
+
+The _Kubeflow Pipelines_ dashboard and _Inference Service_ capabilities are not
+available with this Docker Compose setup.
+
+## Prerequisites
+
+* Install [Docker Compose](https://docs.docker.com/compose/install/).
+* It may be necessary to increase the [Docker resources](https://docs.docker.com/docker-for-mac/#resources) from the
+default of 2 GB memory to 4 GB.
+* Approximately 10 GB of free storage
+
+Clone this repository and navigate to the `bootstrapper` folder:
+
+    git clone https://github.com/machine-learning-exchange/mlx.git
+    cd mlx/bootstrapper
+
+## Keep up to date
+
+If some time has passed since the `mlx` repository was cloned, 
+make sure to pull the latest sources:
+
+    git pull
+
+## Pull the Docker Images
+
+Our Docker images for the [mlx-api](https://hub.docker.com/r/mlexchange/mlx-api/tags?name=nightly)
+and [mlx-ui](https://hub.docker.com/r/mlexchange/mlx-ui/tags?name=nightly) 
+get rebuilt nightly. To get the latest version, run:
+
+    docker compose pull
+
+## Bring up the Docker Containers
+
+    docker compose up
+
+Wait for the containers to start up. When the MLX API and UI are ready, this
+message should show up in the terminal log:
+
+```Markdown
+dashboard_1   |
+dashboard_1   | ================================================
+dashboard_1   |  Open the MLX Dashboard at http://localhost:80/
+dashboard_1   | ================================================
+dashboard_1   |
+```
+
+Now open a web browser and type `localhost` in the address bar to open the MLX
+dashboard.
+
+The MLX API spec can be explored at `localhost:8080/apis/v1alpha1/ui/`
+
+**Note:** If the Docker compose stack is running on a remote host, and the
+MLX Web UI is running on `localhost`, export the environment
+variable `DOCKER_HOST_IP`, so that the MLX UI web app on `localhost` can connect
+to the MLX API on the Docker host.
+
+    export DOCKER_HOST_IP=127.0.0.1
+    docker compose up
+
+## Shut Down the Docker Containers
+
+Press `control` + `c` on the Terminal to stop and then remove the containers:
+
+    docker compose down -v
+
+## Remove the Data Created by Minio and MySQL
+
+    docker volume prune -f
+
+## Troubleshooting
+
+If you are working on a local clone of your fork, rather than a clone of the source
+repository, make sure to keep your code up to date:
+
+    git remote add upstream https://github.com/machine-learning-exchange/mlx.git
+    git fetch upstream
+    git checkout main
+    git rebase upstream/main
+    git push origin main --force
+
+Since we are actively developing MLX, there may have been changes to the data schema
+which could conflict with the data created by running the Quickstart in days prior.
+The symptoms of this could be empty dashboards with endlessly spinning wheels.
+To remove all previously created Docker Compose data run the following commands:
+
+    docker compose down -v --remove-orphans
+    docker compose rm -v -f
+    docker volume prune -f
+
+### Windows Subsystem for Linux (WSL) Issues
+
+#### Featured Assets Pages are Empty
+
+If there are no featured asset cards showing up in the MLX web UI and the Docker Compose log shows an error like this:
+
+    catalog_1     | /bin/sh: /init_catalog.sh: not found
+    catalog_1 exited with code 127
+
+Make sure you originally cloned/forked the source repo from inside the WSL sub-system, not Windows. This error happens
+because the MLX source files have Windows line endings (`\r\n` - CRLF) which `bash` cannot run.
+(https://askubuntu.com/questions/966488/how-do-i-fix-r-command-not-found-errors-running-bash-scripts-in-wsl#comment1553686_966488).
+This error in the `catalog_1` service prevents the loading of assets and objects into the MLX catalog.
+
+