wip: Vertex AI

Ch09/README.md

@@ -0,0 +1,36 @@
+This chapter uses Vertex AI to run custom training jobs.
+
+## Pre-requisite
+
+- Images for running custom training jobs are pushed to Artifact Registry
+- Training and Test datasets are saved on Google Cloud Storage
+
+## Architecture
+
+```markdown
+|-----------|                 |-------------------|
+|           | <--- 1.Pull --- | Artifact Registry |
+|           |                 |-------------------|
+|           |
+| Vertex AI |                 |-----|
+|           | <--- 2.Pull --- |     |
+|           |                 | GCS |
+|           | --- 3.Save ---> |     |
+|-----------|                 |-----|
+```
+
+1. Pull Image from `Artifact Registry`
+2. Pull dataset from `GCS`
+3. Save model and checkpoints to `GCS`
+
+## How to run a job
+
+```shell
+export PROJECT_ID=$(gcloud config get core/project)
+export REGION=$(gcloud config get compute/region)
+export REPOS_NAME=custom-training
+export IMAGE_TAG=YOUR_IMAGE_TAG
+export CONTAINER_IMAGE_URI=${REGION}-docker.pkg.dev/${PROJECT_ID}/${REPOS_NAME}/${IMAGE_TAG}
+
+./create_job.sh
+```

Ch09/config.yaml

@@ -0,0 +1,7 @@
+workerPoolSpecs:
+  machineSpec:
+    # Supported machine types
+    # https://cloud.google.com/vertex-ai/docs/training/configure-compute
+    machineType: n2-standard-4
+  replicaCount: 1
+

Ch09/create_job.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+
+TRAIN_DATA=/gcs/elite-caster-125113-jp/flights/train.csv
+EVAL_DATA=/gcs/elite-caster-125113-jp/flights/test.csv
+OUTPUT_DIR=/gcs/elite-caster-125113-jp/flights/output/
+
+gcloud ai custom-jobs create \
+  --region=asia-northeast1 \
+  --display-name=flights \
+  --config=config.yaml \
+  --args=--traindata="$TRAIN_DATA",--evaldata="$EVAL_DATA",--output="$OUTPUT_DIR" \
+  --worker-pool-spec=container-image-uri="$CONTAINER_IMAGE_URI"
+