Tips/Python Packages.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "616a1185-fc50-4a14-8670-37c35e29d9ea",
   "metadata": {},
   "source": [
    "# Python Packages - for Training Code\n",
    "\n",
    "At the simplest, all the training code may be in a single `filename.py` file that is a module. There are a couple of layers of depth that are commonly added to this:\n",
    "\n",
    "**Python Modules**\n",
    "\n",
    "Modules are files: `filename.py`\n",
    "\n",
    "**Python Project**\n",
    "\n",
    "Projects are collections of **Python Modules** in folders and possibly subfolders.  Here is an example project named `trainer`.\n",
    "```bash\n",
    "│   │   ├── trainer/\n",
    "│   │   │   ├── __init__.py\n",
    "│   │   │   ├── train.py\n",
    "│   │   │   ├── module_1.py\n",
    "│   │   │   ├── helpers/\n",
    "│   │   │   │   ├── __init__.py\n",
    "│   │   │   │   ├── module_a.py\n",
    "│   │   │   │   ├── module_a.py\n",
    "```\n",
    "Here the `train.py` might have `import module_1` and `import helpers.module_a as module_a`.  Note the `__init__.py` file in the folders - this is an empty file that lets Python know the folder can be imported as a module.\n",
    "\n",
    "**Python Packages**\n",
    "\n",
    "Packages are creating by adding necessary files to a **Python Project** to help create a distribution package.\n",
    "```bash\n",
    "├── training_package/\n",
    "│   ├── pyproject.toml\n",
    "│   ├── src/\n",
    "│   │   ├── trainer/\n",
    "│   │   │   ├── __init__.py\n",
    "│   │   │   ├── train.py\n",
    "│   │   │   ├── module_1.py\n",
    "│   │   │   ├── helpers/\n",
    "│   │   │   │   ├── __init__.py\n",
    "│   │   │   │   ├── module_a.py\n",
    "│   │   │   │   ├── module_a.py\n",
    "```\n",
    "\n",
    "Example `pyproject.toml` file that sets `setuptools` as the build system:\n",
    "```python\n",
    "[build-system]\n",
    "requires = [\"setuptools\"]\n",
    "build-backend = \"setuptools.build_meta\"\n",
    "\n",
    "[project]\n",
    "name = 'trainer'\n",
    "version = '0.1'\n",
    "dependencies = ['tensorflow_io', 'google-cloud-aiplatform>=1.17.0']\n",
    "description = 'Training Package'\n",
    "authors = [{{name = 'statmike'}}]\n",
    "```\n",
    "\n",
    "**Python Distribution Archive**\n",
    "\n",
    "Prepare **Python Packages** for distribution - called an archive, distribution, or distribution archive. There are two formats for these:\n",
    "- `file.tar.gz` is a source distributions\n",
    "    - created with `python setup.py sdist` or `python -m build` run in the package level folder\n",
    "    - tarballs, `file.tar`, a collection of files wrapped into a single file\n",
    "    - compressed, `file.tar.gz`, using [gzip](https://www.gzip.org/)\n",
    "    - contains metadata and source files to be installed by pip\n",
    "- `.whl` is a built distribution\n",
    "    - created with `python setup.py bdist_wheel` or `python -m build` from the `package` level folder\n",
    "    - wheels, `file.whl`, built into a compressed binary format that is portable\n",
    "\n",
    "Notes on distribution tools:\n",
    "- here we use the setuptools as the backend build tool specified in the `[build-system]` section of `pyproject.toml`\n",
    "    - `python -m build` uses `pyproject.toml` to automatically create both `.whl` built distribution and `.tar.gz` source distribution versions\n",
    "- another way you may see this done is using setuptools directly by creating a `setup.py` file instead of `pyproject.toml`.  It can then be used with setuptools:\n",
    "    - `python setup.py sdist` which automaticlaly creates `file.tar.gz` by default\n",
    "    - `python setup.py bdist_wheel` which creates `file.whl`\n",
    "    - this is the method mentioned on this Vertex AI documentation page for [creating a python training application for a pre-built container](https://cloud.google.com/vertex-ai/docs/training/create-python-pre-built-container)\n",
    "        - the method in this notebook also builds the source distribution in a compatible way for use with Vertex AI pre-built containers.  You can actually directly use `gzip` to create the source distribution for a folder of training files!\n",
    "- several advantages to using `build`\n",
    "    - automatically create source and built distribution in a `/dist` subfolder\n",
    "    - automatic discovery of modules for common directory structures - [link](https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#automatic-discovery)\n",
    "    - defaults to include package data files - [link](https://setuptools.pypa.io/en/latest/userguide/datafiles.html#include-package-data)\n",
    "    - can link to readme.md and license files\n",
    "\n",
    "**Installing Packages**\n",
    "\n",
    "When you `pip install ...` what is happening?  This causes pip to look for the package and install it.  The default location to look is [PyPI](https://pypi.org/).  This can be overridden:\n",
    "- local install `pip install path/to/file.tar.gz` or `pip install path/to/file.whl`\n",
    "- install from custom repository on Artifact Registry with `pip install --index-url https://{REGION}-python.pkg.dev/{PROJECT}/{REPOSITORY}/{PACKAGE}/ sampleproject`\n",
    "\n",
    "\n",
    "Resources:\n",
    "- [pip install](https://pip.pypa.io/en/stable/cli/pip_install/)\n",
    "- [Packaging Python Projects Tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/)\n",
    "- [setuptools](https://docs.python.org/3/distutils/sourcedist.html)\n",
    "- [setuptools quickstart](https://setuptools.pypa.io/en/latest/userguide/quickstart.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18937d6e-2c46-4590-9778-427316380644",
   "metadata": {},
   "source": [
    "---\n",
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be149cb7-9aab-4ca1-93c4-0e294d723a1a",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Package Installs (if needed)\n",
    "\n",
    "This notebook uses the Python Clients for\n",
    "- Google Service Usage\n",
    "    - to enable APIs (Artifact Registry)\n",
    "- Artifact Registry\n",
    "    - to create a repository for storing custom Python packages in a GCP Project\n",
    "\n",
    "The cells below check to see if the required Python libraries are installed.  If any are not it will print a message to do the install with the associated pip command to use.  These installs must be completed before continuing this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "id": "e517d857-e3fb-431a-a426-bcad424353d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import google.cloud.service_usage_v1\n",
    "except ImportError:\n",
    "    print('You need to pip install google-cloud-service-usage')\n",
    "    !pip install google-cloud-service-usage -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "id": "18845f24-9ddf-4b64-9eb6-40c77783136c",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import google.cloud.artifactregistry_v1\n",
    "except ImportError:\n",
    "    print('You need to pip install google-cloud-artifact-registry')\n",
    "    !pip install google-cloud-artifact-registry -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "id": "6115404d-af8c-4a30-b846-37bea8b411e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import build\n",
    "except ImportError:\n",
    "    print('You need to pip install build')\n",
    "    !pip install build -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "id": "940e3133-cfae-4b3a-80e2-5e9cd7dac1c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import twine\n",
    "except ImportError:\n",
    "    print('You need to pip install twine')\n",
    "    !pip install twine -q"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d46c79cf-e5d4-427d-9c5c-260bc8f36d14",
   "metadata": {},
   "source": [
    "### Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c3f1466-8c08-4884-9c0b-b09cb9df94a3",
   "metadata": {},
   "source": [
    "inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "id": "09680104-3052-4adb-b247-8ecaa4672a34",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'statmike-mlops-349915'"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "project = !gcloud config get-value project\n",
    "PROJECT_ID = project[0]\n",
    "PROJECT_ID"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "id": "b2a88a5e-0238-47ac-9e70-c761892c602d",
   "metadata": {},
   "outputs": [],
   "source": [
    "REGION = 'us-central1'\n",
    "EXPERIMENT = 'packages'\n",
    "SERIES = 'tips'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8da1824-09dc-4215-98cd-4471667a92a6",
   "metadata": {},
   "source": [
    "packages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "id": "7629a781-519d-48b7-a135-812b31dea62d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "import pkg_resources\n",
    "from datetime import datetime\n",
    "from google.cloud import storage\n",
    "from google.cloud import aiplatform\n",
    "\n",
    "from google.cloud import service_usage_v1\n",
    "from google.cloud import artifactregistry_v1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbc1edec-b957-4453-89e1-9aa68035602e",
   "metadata": {},
   "source": [
    "clients:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "id": "31447726-a55a-4ef5-bd2d-2465215d22a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "gcs = storage.Client()\n",
    "aiplatform.init(project = PROJECT_ID, location = REGION)\n",
    "\n",
    "su_client = service_usage_v1.ServiceUsageClient()\n",
    "ar_client = artifactregistry_v1.ArtifactRegistryClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e41b6f01-0018-4185-ac68-61e9a43a3070",
   "metadata": {},
   "source": [
    "parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "id": "f01a4cfe-8ade-4272-92a2-d78bc532cb89",
   "metadata": {},
   "outputs": [],
   "source": [
    "DIR = 'code'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27e41e2e-b2e7-4447-83fd-8ead073a8ce2",
   "metadata": {},
   "source": [
    "environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "id": "3d3912e8-7365-4bda-8105-88f8fa298524",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DIR exists?  True\n"
     ]
    }
   ],
   "source": [
    "# remove directory named DIR if exists\n",
    "shutil.rmtree(DIR, ignore_errors = True)\n",
    "\n",
    "# create directory DIR\n",
    "os.makedirs(DIR)\n",
    "\n",
    "# check for existance of DIR\n",
    "print('DIR exists? ', os.path.exists(DIR))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b5b13f5-0e0e-4d8f-b40d-6e1684da3432",
   "metadata": {},
   "source": [
    "---\n",
    "## Construct Python Package"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93d6c2db-f5b2-4298-8369-9b9ad6d09eb6",
   "metadata": {},
   "source": [
    "Use the temp dirctory created at DIR:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "id": "6b3ed43e-cbf5-410c-b9af-7a1e3b82a894",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'code'"
      ]
     },
     "execution_count": 228,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DIR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "id": "4fd56b82-afaa-4402-b6bf-2a22064eab6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 230,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir(f'{DIR}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df5efeb6-5e60-4bbe-b004-49a1d80687c6",
   "metadata": {},
   "source": [
    "### Create the folder structure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "id": "23e2c788-3ef6-4748-8f1f-b7024b2599e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.makedirs(DIR + f'/{SERIES}_trainer/src/{SERIES}_trainer')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "id": "5b08dcc3-ca10-4920-a4c6-393ed53086d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "code\n",
      "code/tips_trainer\n",
      "code/tips_trainer/src\n",
      "code/tips_trainer/src/tips_trainer\n"
     ]
    }
   ],
   "source": [
    "for root, dirs, files in os.walk(DIR):\n",
    "    print(root)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15cc1a36-e006-47fd-a980-875c4218dd60",
   "metadata": {},
   "source": [
    "### Add files to directory:\n",
    "\n",
    "The [05 - TensorFlow](../05%20-%20TensorFlow/readme.md) series has a model training file named [train.py](../05%20-%20TensorFlow/code/train.py) that will be used here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "id": "f62ff510-1136-4122-9a31-dbe32eccb31f",
   "metadata": {},
   "outputs": [],
   "source": [
    "shutil.copyfile('../05 - TensorFlow/code/train.py', f'{DIR}/{SERIES}_trainer/src/{SERIES}_trainer/train.py')\n",
    "with open(f'{DIR}/{SERIES}_trainer/src/{SERIES}_trainer/__init__.py', 'w') as file: pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "id": "29669ad0-0944-49bb-8d2e-dff5d3a8bc78",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(f'{DIR}/{SERIES}_trainer/pyproject.toml', 'w') as file:\n",
    "    file.write(f\"\"\"[build-system]\n",
    "requires = [\"setuptools\"]\n",
    "build-backend = \"setuptools.build_meta\"\n",
    "\n",
    "[project]\n",
    "name = '{SERIES}_trainer'\n",
    "version = '0.1'\n",
    "dependencies = ['tensorflow_io', 'google-cloud-aiplatform>={aiplatform.__version__}', 'protobuf=={pkg_resources.get_distribution('protobuf').version}']\n",
    "description = 'Training Package'\n",
    "authors = [{{name = 'statmike'}}]\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6517dae0-5b0b-4e63-9894-9b22964619b4",
   "metadata": {},
   "source": [
    "list directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "id": "4d2039fb-4857-4b1e-bdce-f65ba3a6df19",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "code/tips_trainer/pyproject.toml\n",
      "code/tips_trainer/.ipynb_checkpoints/pyproject-checkpoint.toml\n",
      "code/tips_trainer/src/tips_trainer/__init__.py\n",
      "code/tips_trainer/src/tips_trainer/train.py\n"
     ]
    }
   ],
   "source": [
    "for root, dirs, files in os.walk(DIR):\n",
    "    for f in files:\n",
    "        print(os.path.join(root, f))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ffeb127-98c6-408f-9cd2-746c2b9aabee",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Build the Python distribution archives:\n",
    "\n",
    "The build process creates both a `.tar.gz` source distribution and a `.whl` built distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "id": "c06245fc-e6d6-4871-ae53-1adc46c5856a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m* Creating virtualenv isolated environment...\u001b[0m\n",
      "\u001b[1m* Installing packages in isolated environment... (setuptools)\u001b[0m\n",
      "\u001b[1m* Getting dependencies for sdist...\u001b[0m\n",
      "running egg_info\n",
      "creating src/tips_trainer.egg-info\n",
      "writing src/tips_trainer.egg-info/PKG-INFO\n",
      "writing dependency_links to src/tips_trainer.egg-info/dependency_links.txt\n",
      "writing requirements to src/tips_trainer.egg-info/requires.txt\n",
      "writing top-level names to src/tips_trainer.egg-info/top_level.txt\n",
      "writing manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "reading manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "writing manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "\u001b[1m* Building sdist...\u001b[0m\n",
      "running sdist\n",
      "running egg_info\n",
      "writing src/tips_trainer.egg-info/PKG-INFO\n",
      "writing dependency_links to src/tips_trainer.egg-info/dependency_links.txt\n",
      "writing requirements to src/tips_trainer.egg-info/requires.txt\n",
      "writing top-level names to src/tips_trainer.egg-info/top_level.txt\n",
      "reading manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "writing manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md\n",
      "\n",
      "running check\n",
      "creating tips_trainer-0.1\n",
      "creating tips_trainer-0.1/src\n",
      "creating tips_trainer-0.1/src/tips_trainer\n",
      "creating tips_trainer-0.1/src/tips_trainer.egg-info\n",
      "copying files to tips_trainer-0.1...\n",
      "copying pyproject.toml -> tips_trainer-0.1\n",
      "copying src/tips_trainer/__init__.py -> tips_trainer-0.1/src/tips_trainer\n",
      "copying src/tips_trainer/train.py -> tips_trainer-0.1/src/tips_trainer\n",
      "copying src/tips_trainer.egg-info/PKG-INFO -> tips_trainer-0.1/src/tips_trainer.egg-info\n",
      "copying src/tips_trainer.egg-info/SOURCES.txt -> tips_trainer-0.1/src/tips_trainer.egg-info\n",
      "copying src/tips_trainer.egg-info/dependency_links.txt -> tips_trainer-0.1/src/tips_trainer.egg-info\n",
      "copying src/tips_trainer.egg-info/requires.txt -> tips_trainer-0.1/src/tips_trainer.egg-info\n",
      "copying src/tips_trainer.egg-info/top_level.txt -> tips_trainer-0.1/src/tips_trainer.egg-info\n",
      "Writing tips_trainer-0.1/setup.cfg\n",
      "Creating tar archive\n",
      "removing 'tips_trainer-0.1' (and everything under it)\n",
      "\u001b[1m* Building wheel from sdist\u001b[0m\n",
      "\u001b[1m* Creating virtualenv isolated environment...\u001b[0m\n",
      "\u001b[1m* Installing packages in isolated environment... (setuptools)\u001b[0m\n",
      "\u001b[1m* Getting dependencies for wheel...\u001b[0m\n",
      "running egg_info\n",
      "writing src/tips_trainer.egg-info/PKG-INFO\n",
      "writing dependency_links to src/tips_trainer.egg-info/dependency_links.txt\n",
      "writing requirements to src/tips_trainer.egg-info/requires.txt\n",
      "writing top-level names to src/tips_trainer.egg-info/top_level.txt\n",
      "reading manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "writing manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "\u001b[1m* Installing packages in isolated environment... (wheel)\u001b[0m\n",
      "\u001b[1m* Building wheel...\u001b[0m\n",
      "running bdist_wheel\n",
      "running build\n",
      "running build_py\n",
      "creating build\n",
      "creating build/lib\n",
      "creating build/lib/tips_trainer\n",
      "copying src/tips_trainer/train.py -> build/lib/tips_trainer\n",
      "copying src/tips_trainer/__init__.py -> build/lib/tips_trainer\n",
      "running egg_info\n",
      "writing src/tips_trainer.egg-info/PKG-INFO\n",
      "writing dependency_links to src/tips_trainer.egg-info/dependency_links.txt\n",
      "writing requirements to src/tips_trainer.egg-info/requires.txt\n",
      "writing top-level names to src/tips_trainer.egg-info/top_level.txt\n",
      "reading manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "writing manifest file 'src/tips_trainer.egg-info/SOURCES.txt'\n",
      "installing to build/bdist.linux-x86_64/wheel\n",
      "running install\n",
      "running install_lib\n",
      "creating build/bdist.linux-x86_64\n",
      "creating build/bdist.linux-x86_64/wheel\n",
      "creating build/bdist.linux-x86_64/wheel/tips_trainer\n",
      "copying build/lib/tips_trainer/train.py -> build/bdist.linux-x86_64/wheel/tips_trainer\n",
      "copying build/lib/tips_trainer/__init__.py -> build/bdist.linux-x86_64/wheel/tips_trainer\n",
      "running install_egg_info\n",
      "Copying src/tips_trainer.egg-info to build/bdist.linux-x86_64/wheel/tips_trainer-0.1-py3.7.egg-info\n",
      "running install_scripts\n",
      "creating build/bdist.linux-x86_64/wheel/tips_trainer-0.1.dist-info/WHEEL\n",
      "creating '/home/jupyter/vertex-ai-mlops/Tips/code/tips_trainer/dist/tmptia_dx0k/tips_trainer-0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it\n",
      "adding 'tips_trainer/__init__.py'\n",
      "adding 'tips_trainer/train.py'\n",
      "adding 'tips_trainer-0.1.dist-info/METADATA'\n",
      "adding 'tips_trainer-0.1.dist-info/WHEEL'\n",
      "adding 'tips_trainer-0.1.dist-info/top_level.txt'\n",
      "adding 'tips_trainer-0.1.dist-info/RECORD'\n",
      "removing build/bdist.linux-x86_64/wheel\n",
      "\u001b[1m\u001b[92mSuccessfully built \u001b[4mtips_trainer-0.1.tar.gz\u001b[0m\u001b[1m\u001b[92m and \u001b[4mtips_trainer-0.1-py3-none-any.whl\u001b[0m\u001b[1m\u001b[92m\u001b[0m\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "!cd ./{DIR}/{SERIES}_trainer && python -m build"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "087c9872-b835-48cf-ba6a-c7e365848933",
   "metadata": {},
   "source": [
    "list directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "id": "41cb8e6b-d08e-42c4-9142-f7d978416f46",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "code/tips_trainer/pyproject.toml\n",
      "code/tips_trainer/.ipynb_checkpoints/pyproject-checkpoint.toml\n",
      "code/tips_trainer/src/tips_trainer/__init__.py\n",
      "code/tips_trainer/src/tips_trainer/train.py\n",
      "code/tips_trainer/src/tips_trainer.egg-info/top_level.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/SOURCES.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/requires.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/dependency_links.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/PKG-INFO\n",
      "code/tips_trainer/dist/tips_trainer-0.1.tar.gz\n",
      "code/tips_trainer/dist/tips_trainer-0.1-py3-none-any.whl\n"
     ]
    }
   ],
   "source": [
    "for root, dirs, files in os.walk(DIR):\n",
    "    for f in files:\n",
    "        print(os.path.join(root, f))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4848a24-0ce3-4ae2-883e-710041c06c30",
   "metadata": {},
   "source": [
    "**Review**\n",
    "\n",
    "This single folder now has:\n",
    "\n",
    "- a single training file/module: {DIR}/tips_trainer/src/tips_trainer/train.py\n",
    "- a folder of training code: {DIR}/tips_trainer/src/tips_trainer*\n",
    "    - with a starting point of train.py\n",
    "- a source distribution: {DIR}/tips_trainer/dist/tips_trainer-0.1.tar.gz\n",
    "- a built distribution: {DIR}/tips_trainer/dist/tips_trainer-0.1-py3-none-any.whl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0876e703-10a2-4d34-85fa-74024f26fa1e",
   "metadata": {},
   "source": [
    "### Copy to GCS\n",
    "\n",
    "Here the folder structure for DIR will be copied to the GCS Bucket used across this project.  This section uses skills that are discussed in more detail in the [Python Client for GCS](./Python%20Client%20for%20GCS.ipynb) notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c1d8f4f-f04d-45c3-80ed-823d7612bd2a",
   "metadata": {},
   "source": [
    "List buckets in project:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "id": "e5e1f0b9-6edc-499d-8574-687226847d78",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Bucket: cloud-ai-platform-a68e7f3a-fac8-47f6-9f92-fff95c09cdb8>,\n",
       " <Bucket: statmike-mlops-349915>,\n",
       " <Bucket: statmike-mlops-349915-vertex-pipelines-us-central1>]"
      ]
     },
     "execution_count": 243,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(gcs.list_buckets())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44ae862e-aa14-4137-981b-40d940aecaa6",
   "metadata": {},
   "source": [
    "Get the bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "id": "fa9ab5fb-f424-495e-b773-260f7bb5bc8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket = gcs.lookup_bucket(PROJECT_ID)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edcc74d4-04e4-4395-898c-1c73114b0ae8",
   "metadata": {},
   "source": [
    "list files to upload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "id": "0287e7fe-3c73-483c-a5f1-35768fa61e1a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "code/tips_trainer/pyproject.toml\n",
      "code/tips_trainer/.ipynb_checkpoints/pyproject-checkpoint.toml\n",
      "code/tips_trainer/src/tips_trainer/__init__.py\n",
      "code/tips_trainer/src/tips_trainer/train.py\n",
      "code/tips_trainer/src/tips_trainer.egg-info/top_level.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/SOURCES.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/requires.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/dependency_links.txt\n",
      "code/tips_trainer/src/tips_trainer.egg-info/PKG-INFO\n",
      "code/tips_trainer/dist/tips_trainer-0.1.tar.gz\n",
      "code/tips_trainer/dist/tips_trainer-0.1-py3-none-any.whl\n"
     ]
    }
   ],
   "source": [
    "for root, dirs, files in os.walk(DIR):\n",
    "    for f in files:\n",
    "        print(os.path.join(root, f))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f85cabdc-528c-440c-bf43-7409a105a0d3",
   "metadata": {},
   "source": [
    "list of desired bucket object URIs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "id": "e40e4f90-4061-4442-8abe-6be5fbae47d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tips/code/tips_trainer/pyproject.toml\n",
      "tips/code/tips_trainer/.ipynb_checkpoints/pyproject-checkpoint.toml\n",
      "tips/code/tips_trainer/src/tips_trainer/__init__.py\n",
      "tips/code/tips_trainer/src/tips_trainer/train.py\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/top_level.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/SOURCES.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/requires.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/dependency_links.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/PKG-INFO\n",
      "tips/code/tips_trainer/dist/tips_trainer-0.1.tar.gz\n",
      "tips/code/tips_trainer/dist/tips_trainer-0.1-py3-none-any.whl\n"
     ]
    }
   ],
   "source": [
    "for root, dirs, files in os.walk(DIR):\n",
    "    for f in files:\n",
    "        filepath = os.path.join(root, f)\n",
    "        gcspath = f'{SERIES}/{filepath}'\n",
    "        print(gcspath)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e48dc5ff-aabd-4f06-a300-98ab03c07a62",
   "metadata": {},
   "source": [
    "upload files as objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "id": "5935e6e6-813d-4f3e-9e0f-c7d2a3c855ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "for root, dirs, files in os.walk(DIR):\n",
    "    for f in files:\n",
    "        filepath = os.path.join(root, f)\n",
    "        gcspath = f'{SERIES}/{filepath}'\n",
    "        blob = bucket.blob(gcspath)\n",
    "        blob.upload_from_filename(filepath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "id": "dfb199b8-9f3f-4d5d-bf9a-7727e71f97d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "View the bucket directly here:\n",
      "https://console.cloud.google.com/storage/browser/statmike-mlops-349915;tab=objects&project=statmike-mlops-349915\n"
     ]
    }
   ],
   "source": [
    "print(f\"View the bucket directly here:\\nhttps://console.cloud.google.com/storage/browser/{PROJECT_ID};tab=objects&project={PROJECT_ID}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33878a45-d6e3-4428-b9ca-79dc6004a7c2",
   "metadata": {},
   "source": [
    "list files in bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "id": "cad2c043-9d3c-4879-8a68-d4f319f0dcdc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tips/code/tips_trainer/.ipynb_checkpoints/pyproject-checkpoint.toml\n",
      "tips/code/tips_trainer/dist/tips_trainer-0.1-py3-none-any.whl\n",
      "tips/code/tips_trainer/dist/tips_trainer-0.1.tar.gz\n",
      "tips/code/tips_trainer/pyproject.toml\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/PKG-INFO\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/SOURCES.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/dependency_links.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/requires.txt\n",
      "tips/code/tips_trainer/src/tips_trainer.egg-info/top_level.txt\n",
      "tips/code/tips_trainer/src/tips_trainer/__init__.py\n",
      "tips/code/tips_trainer/src/tips_trainer/train.py\n"
     ]
    }
   ],
   "source": [
    "for blob in list(bucket.list_blobs(prefix = f'{SERIES}/{DIR}')):\n",
    "    print(blob.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be7499ba-3de4-497c-91d6-d7e376704376",
   "metadata": {},
   "source": [
    "---\n",
    "## Artifact Registry\n",
    "\n",
    "Artifact registry organizes artifacts with repositories.  Each repository contains packages and is designated to hold a partifcular format of package: Docker images, Python Packages and [others](https://cloud.google.com/artifact-registry/docs/supported-formats#package).\n",
    "\n",
    "\n",
    "- upload package\n",
    "- show listing at multiple levels - see doc"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1235d67-bc3b-4d8b-a1c0-7cc3d7bafa73",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Enable API for Artifact Registry\n",
    "\n",
    "Using Artifact Registry requires enabling the API for the Google Cloud Project.\n",
    "\n",
    "Options for enabeling these.  In this notebook (2) is used.\n",
    " 1. Use the APIs & Services page in the console: https://console.cloud.google.com/apis\n",
    "     - `+ Enable APIs and Services`\n",
    "     - Search for Cloud Build and Enable\n",
    "     - Search for Artifact Registry and Enable\n",
    " 2. Use [Google Service Usage](https://cloud.google.com/service-usage/docs) API from Python\n",
    "     - [Python Client For Service Usage](https://github.com/googleapis/python-service-usage)\n",
    "     - [Python Client Library Documentation](https://cloud.google.com/python/docs/reference/serviceusage/latest)\n",
    "     \n",
    "The following code cells use the Service Usage Client to:\n",
    "- get the state of the service\n",
    "- if 'DISABLED':\n",
    "    - Try enabling the service and return the state after trying\n",
    "- if 'ENABLED' print the state for confirmation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "id": "9d080a18-3e70-4518-ac69-105901dd8194",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Artifact Registry already enabled for project: statmike-mlops-349915\n"
     ]
    }
   ],
   "source": [
    "artifactregistry = su_client.get_service(\n",
    "    request = service_usage_v1.GetServiceRequest(\n",
    "        name = f'projects/{PROJECT_ID}/services/artifactregistry.googleapis.com'\n",
    "    )\n",
    ").state.name\n",
    "\n",
    "\n",
    "if artifactregistry == 'DISABLED':\n",
    "    print(f'Artifact Registry is currently {artifactregistry} for project: {PROJECT_ID}')\n",
    "    print(f'Trying to Enable...')\n",
    "    operation = su_client.enable_service(\n",
    "        request = service_usage_v1.EnableServiceRequest(\n",
    "            name = f'projects/{PROJECT_ID}/services/artifactregistry.googleapis.com'\n",
    "        )\n",
    "    )\n",
    "    response = operation.result()\n",
    "    if response.service.state.name == 'ENABLED':\n",
    "        print(f'Artifact Registry is now enabled for project: {PROJECT_ID}')\n",
    "    else:\n",
    "        print(response)\n",
    "else:\n",
    "    print(f'Artifact Registry already enabled for project: {PROJECT_ID}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3ccf37d-95da-49d1-94f8-7bdcc65e0338",
   "metadata": {},
   "source": [
    "### Create Python Package Repository\n",
    "\n",
    "Create an Artifact Registry Repository to hold Python Packages created by this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "id": "4994191c-fe48-4aac-99be-7624d4a7e54b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Retrieved existing repo: projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python\n"
     ]
    }
   ],
   "source": [
    "python_repo = None\n",
    "for repo in ar_client.list_repositories(parent = f'projects/{PROJECT_ID}/locations/{REGION}'):\n",
    "    if f'{PROJECT_ID}-python' in repo.name:\n",
    "        python_repo = repo\n",
    "        print(f'Retrieved existing repo: {python_repo.name}')\n",
    "\n",
    "if not python_repo:\n",
    "    operation = ar_client.create_repository(\n",
    "        request = artifactregistry_v1.CreateRepositoryRequest(\n",
    "            parent = f'projects/{PROJECT_ID}/locations/{REGION}',\n",
    "            repository_id = f'{PROJECT_ID}-python',\n",
    "            repository = artifactregistry_v1.Repository(\n",
    "                description = f'A repository for the {PROJECT_ID} experiment that holds Python Packages.',\n",
    "                name = f'{PROJECT_ID}-python',\n",
    "                format_ = artifactregistry_v1.Repository.Format.PYTHON,\n",
    "                labels = {'series': SERIES, 'experiment': EXPERIMENT}\n",
    "            )\n",
    "        )\n",
    "    )\n",
    "    print('Creating Repository ...')\n",
    "    python_repo = operation.result()\n",
    "    print(f'Completed creating repo: {python_repo.name}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "id": "110601aa-8053-4e2b-9704-27bb0ac9b613",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python',\n",
       " 'PYTHON')"
      ]
     },
     "execution_count": 254,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "python_repo.name, python_repo.format_.name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64badc78-c9a2-480c-b1cb-043644a86a64",
   "metadata": {},
   "source": [
    "### List Repositories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 255,
   "id": "29829561-fe69-4281-84e3-695dff80aeb6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915\n",
      "projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python\n"
     ]
    }
   ],
   "source": [
    "for repo in ar_client.list_repositories(parent = f'projects/{PROJECT_ID}/locations/{REGION}'):\n",
    "    print(repo.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b355abd-51fa-4ab1-98cf-171ca1eab784",
   "metadata": {},
   "source": [
    "### Upload Python Package Distribution\n",
    "\n",
    "This uses the `twine` Python package to upload the model to the repository. Not PyPI as twine was intended but to our customer repository in Artifact Registry by using the `--repository-url` flag.\n",
    "\n",
    "This command runs from out Tips folder so it needs to change directories into DIR and into the subfolder for the trainer project, then run the upload of the full `dist` subfolder which contains both distribution types (source and built)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 256,
   "id": "5c0525ea-6688-488c-a116-26a25a94008c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading distributions to \n",
      "https://us-central1-python.pkg.dev/statmike-mlops-349915/statmike-mlops-349915-p\n",
      "ython\n",
      "Uploading tips_trainer-0.1-py3-none-any.whl\n",
      "\u001b[2K\u001b[35m100%\u001b[0m \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.5/6.5 kB\u001b[0m • \u001b[33m00:00\u001b[0m • \u001b[31m?\u001b[0m\n",
      "\u001b[?25hUploading tips_trainer-0.1.tar.gz\n",
      "\u001b[2K\u001b[35m100%\u001b[0m \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.6/5.6 kB\u001b[0m • \u001b[33m00:00\u001b[0m • \u001b[31m?\u001b[0m\n",
      "\u001b[?25h"
     ]
    }
   ],
   "source": [
    "!cd ./{DIR}/{SERIES}_trainer && python -m twine upload --repository-url https://{REGION}-python.pkg.dev/{PROJECT_ID}/{PROJECT_ID}-python dist/*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df649960-3af6-4d85-9926-503760a2f10d",
   "metadata": {},
   "source": [
    "### Review Repository Contents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "id": "9a96bfe1-75fe-43e5-bae1-d05d765d9474",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python'"
      ]
     },
     "execution_count": 257,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "python_repo.name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e9cbb8f-2587-4f6c-84b0-6d0a9bd43034",
   "metadata": {},
   "source": [
    "List Packages in the repository:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "id": "271119e0-f5bc-4d2d-8819-ee1e7b590d59",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ListPackagesPager<packages {\n",
       "  name: \"projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python/packages/tips-trainer\"\n",
       "  create_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 502115000\n",
       "  }\n",
       "  update_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 771288000\n",
       "  }\n",
       "}\n",
       ">"
      ]
     },
     "execution_count": 258,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ar_client.list_packages(\n",
    "    parent = python_repo.name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c64f8353-ba7c-4170-9c8d-0ce83cd98c82",
   "metadata": {},
   "source": [
    "List files in the repository:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 259,
   "id": "d8bfa14c-2c2e-45d4-887b-a82f7966a176",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ListFilesPager<files {\n",
       "  name: \"projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python/files/tips-trainer%2Ftips_trainer-0.1-py3-none-any.whl\"\n",
       "  size_bytes: 3626\n",
       "  create_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 502115000\n",
       "  }\n",
       "  update_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 502115000\n",
       "  }\n",
       "  owner: \"projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python/packages/tips-trainer/versions/0.1\"\n",
       "}\n",
       "files {\n",
       "  name: \"projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python/files/tips-trainer%2Ftips_trainer-0.1.tar.gz\"\n",
       "  size_bytes: 3070\n",
       "  create_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 771288000\n",
       "  }\n",
       "  update_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 771288000\n",
       "  }\n",
       "  owner: \"projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python/packages/tips-trainer/versions/0.1\"\n",
       "}\n",
       ">"
      ]
     },
     "execution_count": 259,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ar_client.list_files(\n",
    "    parent = python_repo.name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42e2afa1-c758-419c-9e23-f1f17737725a",
   "metadata": {},
   "source": [
    "List versions of a package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "id": "0e3373f1-360f-4527-85f7-89a43fcab4ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ListVersionsPager<versions {\n",
       "  name: \"projects/statmike-mlops-349915/locations/us-central1/repositories/statmike-mlops-349915-python/packages/tips-trainer/versions/0.1\"\n",
       "  create_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 502115000\n",
       "  }\n",
       "  update_time {\n",
       "    seconds: 1663768642\n",
       "    nanos: 771288000\n",
       "  }\n",
       "}\n",
       ">"
      ]
     },
     "execution_count": 262,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ar_client.list_versions(\n",
    "    parent = python_repo.name + f'/packages/{SERIES}-trainer'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89cdde50-cf7c-4e5e-9f70-9a176ae6cf07",
   "metadata": {},
   "source": [
    "---\n",
    "## Using Training Code\n",
    "\n",
    "This notebook created versions of training code (script, folder, distribution) in multiple locations (local, GCS Bucket, Artifact Registry, GitHub).  Using these versions and forms in Vertex AI Training Custom Jobs demonstrated in [Python Training]('./Python%20Training.ipynb') which also uses many workflows with custom containers created by the demonstrations in [Python Custom Containers]('./Python%20Custom%20Containers.ipynb').\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5427bd1-3582-4a56-89e1-6d4956cc1b29",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "python3",
   "name": "tf2-gpu.2-3.m94",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-3:m94"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}