example: add notebook for helloworld example (#3024)

star-whale · Nov 23, 2023 · 96f044c · 96f044c
1 parent 14407e8
commit 96f044c
Show file tree

Hide file tree

Showing 3 changed files with 3,184 additions and 343 deletions.
diff --git a/README.md b/README.md
@@ -213,6 +213,7 @@ We use [MNIST](https://paperswithcode.com/dataset/mnist) as the hello world exam
 
 ## Examples
 
+- 🔥 Helloworld: [Cloud](https://cloud.starwhale.cn/projects/15/evaluations), [Code](https://github.com/star-whale/starwhale/tree/main/example/helloworld).
 - 🚀 LLM:
   - 🐊 OpenSource LLMs Leaderboard: [Evaluation](https://cloud.starwhale.cn/projects/349/evaluations), [Code](https://github.com/star-whale/starwhale/tree/main/example/llm-leaderboard)
   - 🐢 Llama2: [Run llama2 chat in five minutes](https://starwhale.cn/docs/en/blog/run-llama2-chat-in-five-minutes/), [Code](https://github.com/star-whale/starwhale/tree/main/example/LLM/llama2)

diff --git a/example/notebooks/mnist-pytorch.ipynb b/example/notebooks/mnist-pytorch.ipynb
@@ -0,0 +1,345 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a href=\"https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/notebooks/mnist-pytorch.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Installing Starwhale\n",
+    "\n",
+    "Starwhale has three types of instances: Standalone, Server, and Cloud. Starting with the standalone mode is ideal for quickly understanding and mastering Starwhale. You install Starwhale Standalone by running:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "python3 -m pip install starwhale"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Getting Starwhale version and help info."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "swcli --version\n",
+    "swcli --help"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Downloading Examples\n",
+    "\n",
+    "Download Starwhale examples by cloning Starwhale via:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "[ -d \"starwhale\" ] || GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/star-whale/starwhale.git --depth 1\n",
+    "cd starwhale\n",
+    "git pull origin main"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To save time in the example downloading, we skip git-lfs and other commits info. We will use ML/DL HelloWorld code `MNIST` to start your Starwhale journey. The following steps are all performed in the `starwhale` directory."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Building Runtime\n",
+    "\n",
+    "Runtime example code are in the `example/runtime/pytorch` directory.\n",
+    "\n",
+    "- Build the Starwhale Runtime bundle:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "cd starwhale\n",
+    "swcli runtime build --yaml example/runtime/pytorch/runtime.yaml"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Check your local Starwhale Runtime:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "export COLUMNS=150\n",
+    "swcli runtime list\n",
+    "swcli runtime info pytorch"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Building Model\n",
+    "\n",
+    "Model example code are in the `example/mnist` directory.\n",
+    "\n",
+    "- Download pre-trained model file:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "cd starwhale/example/mnist\n",
+    "make download-model"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Build a Starwhale Model with prebuilt Starwhale Runtime:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "cd starwhale\n",
+    "swcli -vvv model build example/mnist --runtime pytorch"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Check your local Starwhale Model:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "export COLUMNS=150\n",
+    "swcli model list\n",
+    "swcli model info mnist"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Building Dataset\n",
+    "\n",
+    "Dataset example code are in the `example/mnist` directory.\n",
+    "\n",
+    "- Download the MNIST raw data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "cd starwhale/example/mnist\n",
+    "make download-data"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Build a Starwhale Dataset with prebuilt Starwhale Runtime:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "cd starwhale\n",
+    "swcli dataset build --yaml example/mnist/dataset.yaml"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Check your local Starwhale Dataset:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "export COLUMNS=150\n",
+    "swcli dataset list\n",
+    "swcli dataset info mnist"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Head some records from the mnist dataset:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "swcli dataset head mnist"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Running an Evaluation Job\n",
+    "\n",
+    "- Create an evaluation job:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "swcli -vvv model run --uri mnist --dataset mnist --runtime pytorch"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Check the evaluation result:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "\n",
+    "export COLUMNS=150\n",
+    "swcli job list\n",
+    "   swcli job info $(swcli job list | grep mnist | grep success | awk '{print $1}' | head -n 1)\n"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "👏 Now, you have completed the basic steps for Starwhale standalone."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "starwhale",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.9.13"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "44f309826dc86d485633ae8f49ea70651a134501b3cf4f8233b66a54380b4a5f"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}