diff --git a/CHANGELOG.md b/CHANGELOG.md index cdafe977..519f952b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - added EFS removal policy to `mlflow-fargate` module - added `mwaa` module with example dag which demonstrates the MLOps in Airflow - added `sagemaker-hugging-face-endpoint` module +- added `hf_import_models` template to import hugging face models ### **Changed** diff --git a/README.md b/README.md index b5a8f23e..0699ccb9 100644 --- a/README.md +++ b/README.md @@ -23,13 +23,13 @@ See deployment steps in the [Deployment Guide](DEPLOYMENT.md). ### SageMaker Modules -| Type | Description | -|---------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [SageMaker Studio Module](modules/sagemaker/sagemaker-studio/README.md) | Provisions secure SageMaker Studio Domain environment, creates example User Profiles for Data Scientist and Lead Data Scientist linked to IAM Roles, and adds lifecycle config | -| [SageMaker Endpoint Module](modules/sagemaker/sagemaker-endpoint/README.md) | Creates SageMaker real-time inference endpoint for the specified model package or latest approved model from the model package group | -| [SageMaker Project Templates via Service Catalog Module](modules/sagemaker/sagemaker-templates-service-catalog/README.md) | Provisions SageMaker Project Templates for an organization. The templates are available using SageMaker Studio Classic or Service Catalog. Available templates:
- [Train a model on Abalone dataset using XGBoost](modules/sagemaker/sagemaker-templates-service-catalog/README.md#train-a-model-on-abalone-dataset-with-xgboost-template)
- [Perform batch inference](modules/sagemaker/sagemaker-templates-service-catalog/README.md#batch-inference-template)
- [Multi-account model deployment](modules/sagemaker/sagemaker-templates-service-catalog/README.md#multi-account-model-deployment-template) | -| [SageMaker Notebook Instance Module](modules/sagemaker/sagemaker-notebook/README.md) | Creates secure SageMaker Notebook Instance for the Data Scientist, clones the source code to the workspace | -| [SageMaker Custom Kernel Module](modules/sagemaker/sagemaker-custom-kernel/README.md) | Builds custom kernel for SageMaker Studio from a Dockerfile | +| Type | Description | +|---------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [SageMaker Studio Module](modules/sagemaker/sagemaker-studio/README.md) | Provisions secure SageMaker Studio Domain environment, creates example User Profiles for Data Scientist and Lead Data Scientist linked to IAM Roles, and adds lifecycle config | +| [SageMaker Endpoint Module](modules/sagemaker/sagemaker-endpoint/README.md) | Creates SageMaker real-time inference endpoint for the specified model package or latest approved model from the model package group | +| [SageMaker Project Templates via Service Catalog Module](modules/sagemaker/sagemaker-templates-service-catalog/README.md) | Provisions SageMaker Project Templates for an organization. The templates are available using SageMaker Studio Classic or Service Catalog. Available templates:
- [Train a model on Abalone dataset using XGBoost](modules/sagemaker/sagemaker-templates-service-catalog/README.md#train-a-model-on-abalone-dataset-with-xgboost-template)
- [Perform batch inference](modules/sagemaker/sagemaker-templates-service-catalog/README.md#batch-inference-template)
- [Multi-account model deployment](modules/sagemaker/sagemaker-templates-service-catalog/README.md#multi-account-model-deployment-template)
- [HuggingFace model import template](modules/sagemaker/sagemaker-templates-service-catalog/README.md#huggingface-model-import-template) | +| [SageMaker Notebook Instance Module](modules/sagemaker/sagemaker-notebook/README.md) | Creates secure SageMaker Notebook Instance for the Data Scientist, clones the source code to the workspace | +| [SageMaker Custom Kernel Module](modules/sagemaker/sagemaker-custom-kernel/README.md) | Builds custom kernel for SageMaker Studio from a Dockerfile | ### Mlflow Modules diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/README.md b/modules/sagemaker/sagemaker-templates-service-catalog/README.md index 163513df..d45ea112 100644 --- a/modules/sagemaker/sagemaker-templates-service-catalog/README.md +++ b/modules/sagemaker/sagemaker-templates-service-catalog/README.md @@ -26,6 +26,14 @@ This project template contains SageMaker pipeline that performs batch inference. ![Batch Inference Template](docs/_static/batch-inference-template.png "Batch Inference Template Architecture") +#### Huggingface Model Import Template + +This project template contains SageMaker pipeline that imports a hugging face model based on model id and access +token inputs. + +![Huggingface model import template](docs/_static/huggingface-model-import.png "Hugging Face Model Import Template +Architecture") + #### Multi-account Model Deployment Template The template contains an example CI/CD pipeline to deploy the model endpoints to multiple AWS accounts. diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/docs/_static/huggingface-model-import.png b/modules/sagemaker/sagemaker-templates-service-catalog/docs/_static/huggingface-model-import.png new file mode 100644 index 00000000..78033cb5 Binary files /dev/null and b/modules/sagemaker/sagemaker-templates-service-catalog/docs/_static/huggingface-model-import.png differ diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/docs/_static/huggingface-model-import.xml b/modules/sagemaker/sagemaker-templates-service-catalog/docs/_static/huggingface-model-import.xml new file mode 100644 index 00000000..5d8a06ab --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/docs/_static/huggingface-model-import.xml @@ -0,0 +1,2 @@ + +7H1Xm9s4svav2UvvwxwuSTEpUcyieHMe5pyj9Os/gN2eaXs83t3vzM7x7FruIIAgCBSq6n2rALX/hh/qTR78Lru2UVz9DUOi7W+48DcMQ3GUAr9gzfOthmbYt4p0yKO3KuTXCjN/xe93fq6d8yge3+veqqa2raa8+7IybJsmDqcv6vxhaNcvmyVtFX1R0flp/JsKM/Sr39be82jK3moZjP61XonzNPv8ZJR6n1/tf278PpMx86N2/VCFi3/DD0PbTm/v6u0QV1B4n+XSXeTXIZ6DG1mHylU4/E95baxP77018TbB9sfI8av5fbDEr/0a8djOQxgL8RgOeTe1A7hpeK98a/0/f8O57z5jjIfcr/KXP+Vt82mJhxH8frt3eW/iv4t3+Mbj3kdixrXfTHko+JN/aJvJz5t4+Gd6f7t7GvImveRTPPjV2zpPcTP9ExLqhraLh+ld77JpghrD/Q2TwBfopK3a9Pn3MQ7nIZ+ef/dr/9U2f4/iBVxO2rmJ9nGBQpT76eDXn5Z8nH8ZL6j3MYwkWZL6FOJR8okIsfBTQDAsKCJ0QjF+ELJvSiK9zeRoHH+zBP/SqIAd5GnzKW/GDmg6lLAUtnXXNkAeIygwhM8gAZl8IikC+0T4KP2JJRnyUxwkUUCRCRHE4R8rmvE5TnH9qYZWD1YJ1CAEibMkS38iMAb/RCQU+YlBWOSTH1N0wkZ4zBLsR6GAN9/Wj89Xv6HHny+9G8y/Yjy/WPPwz+jQ/96Q0L+0IaE/Dem/zpDaoIAwiiGVHwAo3+f1rsRAXcK8sp7du8zTGPSfh5/CzAfYW721+mx/mVJUmNYoLOXGk/Y/R9FaivQT+4s6/mKw4/T8jLVxBKD3vdgOU9ambeNX4q+1/ACFFsNHIKCUTTUcIQrexls+uR/eP2CTv5PvJWF7v2MvPD8UNDCDOp6gnN7qmml4uh8LH3qCxV+72kvPj6WvOyviaXq+Uxp/nlpQ9eu0Lm3bvQ/4TXJw9l9xjn/kpd4bDlFid8C03iTzgf+MYzzB9XI++gT8wxrIcQsGPDx3e6qANi7xVz18XurP7X65VWvzBurJO9MjyHdH9m4L71zk1y7etO/9ri/09sMw/hWP+x0N+wt43O+O/qfH/Qt43O8ur314ecH9JbhycJ4OE4El+fH15ywuQoYhiaPhpwgj0E8EG+JgHhFY4QinaTokCYxK/tDF/WJUKQwDP43TMIfTPMSghiJRkqYw5BNNBEC4NBKDobDRp4ANEpIKqDAggt+TzC9DWNf17yv+93aAK4WyLAsnCueAYZ+A+/s0PoF1bp8aoEz4BBHq3wCNUBJv6PgnICX6qx/5QaHyvwndSOrHQDf0rxKZf3/4P/HtJ779xLf/bnzDfnR8+xkK/n+CJUX/IGD5WV3/omCJ/QTLn2D5Eyz/48ASIFf3JUR6h/FYU6dPTfVau6fpaoj32j7h38+WvvXzBfS99Qp80BRvXzjv991RP/h882ci/qfiyvtafs4Urh/2U/H3uuzDXipBfeWdP0LP5/X8JxHiexL+CyDE94f/EyH+AgjxbxcKkZDAF6LJpwQPKSAUAgglQJBPCIqTFOIHLJYQ/1bX+u/GDiQg4tgPmU8IEZOfiBgsOEOxEMtIKglYgg7ofy92/KeA/J+Igaafxle/3P2glnfQeOLxd4Bx6cIvYVGalFRi25EvjNdRyhJl40Tpu5jYwZhi99QkD76Adz0gfyPBlQMs/R0jv6r4ukx/WYH+tgT7+LLi6zL9ZQX6dffoV89Hvx7gh4rflL7oHvnq+ciHAYIvEGLOExT44ZezUTAUhSieA7U9AL2D4WkDPMiXNGLNgIqbnb8b8QrsENQlQEne41cQmLyX3wUPex0zf1/HetsN9+/+OmLI3z9zlHQ4hjvmfePy/8B1Bx1MQ1vGnwcFrA8jGAYl4KPyqvpqsJDi5KFfcRVwiaB6ggE077+XqjiZYI9gBtBe95KAI++j/vAIjuNpngH1kT9mv+QSfpdFfWRL30fkP4QuvT8cJ/AvwmkU/S1/Iijyt/wJ/QP50++a4l+APP2FPPNP5vSTOf3XMafvcIR/djgYgiBwOMDNSvtYwgz4EzCQNyryz47kO17uR4uxfoA8xuek/2dT+mdyGhjzXf72ZVL/WzzkA035Bi/4Bce/mw35v8Hxr7Li34JxmvpGGgQn/91pkLdF+YnkP5H8J5L/RPL/BZJ/3738hNA/YN/8I8b8odvmf+AO9T9Itn8wjetx9Gf+Fa2mYhzkExW34zj/gdD7v9zJRvEvMfvrjezJH1KwNv/ujWz8LwLP3x39j2b9PwH65zb2X3cbm6/8plTBQh+FbzqS7zjWP2qtfiuVCE1ChKLiTzhDQY2PkU8MjhCfqDBOgLYQiP8eyvwwAOyH767xV9j9Xcn9Luh+Tn3PdcXt/f2an77AEWjtmE9vzjRop6mtQYN9aLwflukOyB9y0sn++t0c9wfE/mZ2/0cJej9Hub8Ncr+11f8HZqq/o/k/PID+13rY3yKXz4QkEkXRJ99nmU8EE8affNQPP5E+SyYgyiNi+g8mFj9AePuXDv7+BMj5S0Z+VAU3I6MczIZKp91Fv1UFnysOQOUObV3n8F4j7iBetND5vbUDTwm+vncXxocev4luIeg3fO/3I8SxT0ePnnbG5VfUM06k5Oid/12I++a++jf31r+1v/7NPfbf7rN/0Wzf+f7GE76u/FYd/dtK9LfNPm+W/7byW3XfOhnw9d3oN+5Gv7r79/flv9qbBv8kuB6/2a8H1wiBBhc/XBPyAXT0hjcNDNu/So+De8CL4NlvbbT/wji+2AX/hxTm+3vwIRgVNKgvEvb/4AyBv0dSoJDkGxzH7xwq+Oxg344U8KD4zcMFH7T/x6FFXycWPp6B/AYvYv44XvS7Rv+TF/1QvOg/Ma3wM+//n039fpfA/B/s4X/Hz/2kod+loZ/PXoA7OQDtiR9C26L8GuJ6E4zdl/zzH3LSf/xEE6Akws9hGU//a7I74l+S3CofledoykSi0y5lCFZa3r7/gYqfJPeHIrkUwuE4/a+RXIymUZT6LyK5I/4Dk9vPOUCM/LPJ7u8a/0+y+5ciu3FMEAHO+p9CIgg+EQggKgFLoZ8AnWPiEGXx+LPS/AeR3b80FfwWifgC5v8PKOF3vMFPSvgPM5P8nFfRH5KLDN56+sjSnJXJRAp1Vl3xJUfg49un7dNPlvbXYWk/U5H/ZCryTfl/OLKGfwbQP4+c/a7N/yRnfyly9jMT+TMT+aPRzw+E5f+AaH7Hs/1XE81vMsL/j8PP9HeZ4f/mb4b9oH/r658/SW0VD5WtnYDzryqtlo8hYlX7rdnbKeV/qKF/7oHrf5EzfEcf/gKs4buj/9E8w0/e8PNg9H/uwejv+Mkf52D0P5rEn8AzvrG0KB3EAK4+hQmwWMJH0E8+TUafGDSOcMBzMRLHfvBU1q/JyT8gm9X90tlH4vK7+vUzofUzofUfldD6Rf9/uJwWRuF/dk7rO7Dyw7PT/yb68p/ITX/mtP7zc1raF2TjT05r/Qmc+b86rUX829Ja/+F/Cv+fT49951Tox/TYdzT9L5geI/4iBOS7o//RPMxPCvIzPfafmx77E07P//vTY/8nOb4fOj32RzEV8t/FVP5PSMF3zgX+55IC8i9NCsifpOAnKfhJCv5cUvAnnJ/+SQp+xD0zZU5TYMIS/Au8f+gH/67Qq/yvd+Hy2k+/2n77nf9K7fsf/Pu8FfLW3//39sx+O/d5nwX5XAPeAw3xgUa9j1cqujiFO1iYxBbgB8Fxull6JyPleC4VOZPjjxwnwCu60Bpu1nkpx1k2wp3Bb44DzVLR5LgrLPAcJ4Jfos9x9nuZ45xUDD6W3VRsf20POueO1hf3H6MODOKX9t7+HPg67JX7pRP3fj2HJWUvCNzzWsgGuP5W5sQDtx7MLH0ri7Cf6zP90H67CocUtBfe+wX9gXK6T+LtOSZov34oW4eST39p/8tLd1TBtEvYq/XAT5V3J1++zCLvkkuR1LRVwUV500K8kyUCaeiwU0R/uPwacEfQFz8FjfG0y+hwlDLvnnMKtYsD9LpyIyd42QNXV93hdett8rwt8Vfud17s9HgfoWBd7tIayuvvNf3nXrxUeq5awHf7WsqnMsTUCZbhDy7uilA5IfCBLSwfVDxS+L287fcbz7tU+bCMvZWbz2X6rZy/l9+kDdo70glKWpRgd9yWPdxTA95ej3v/PPbAItj/9W2a/OzJ1QzL4V4mp+DOwiW5jnv7DLaHQrjuzxey/Kv+m9887/7F80A5+lj+LI/3Vyb6rroGskT+7wT9r72C+4Y+fl3aGv4Q5OxpSvsogWVyPCZPi4l+LJO5VTIfylX5fv97OUNNyf5QVl862n4oG5khjh/7G61y/VD2OqMsP/aPmI7+oey8DOf6a1k5LfYX4/GeOiJ+KEvPL8anOINjf7yeVab94flK1zpS+KHsdU5JfCirg42MX7TXkY/XT6hdfZy/MxvIR/l5T6P6WEZfX8mvM+2P/Rmts7ufX+SZfTF/0L/ufPG81138KE+jMsQvx+N8IT8e0T+OR+FH+4v7ndH4uP6KitmI/XF8wNa+uB+xpY/r1WVm9WF+0HfKDBdw/MIpR+DDgR8Sr5wBLkayQ0R7U14IcJ4M66rxFT3VoZ+T7dS8k8UlX9NrwT1V0Ara/PFdeJx2vb+bPy+N3ePutL/4/vQVQs8hvhGht6/lzU++v/hLB4Y8olDvBvuLfg8JtEntCETyPJFF5J6wX+56+wVMvuWEFcAUX3JKyhUQCC469+SEB3dbITDZnJnyGgfkctf5mJOvnA9dH5h/tPIMdxS5LD1cIDJV+sHnTgzXroeWOyPcmB5w7rJyqy7I0BshnOBw6sgRnFBxt5FjoEvURh5ggsLpCC+uoscZDH/SxYGzjvyNA9PmOdvmLU4yOafkH5zUcHeCj3SJ4R5XPl9lk/NavtXljgtEftYVngtLoBaKx8UiT+nKk0tagIfHK5fpB0U/1lzOHG7ciefK8eBwp4irw0Oonyiu1Q9Fen5w/fEwcGeMG8XDc7043MwdqPWCAjQSDitYqacoXPQrwSFXwVpVn8MeQpiqcKREK1TpLecoQphTTeHYq0CACUIWW4p8qlv8gRMvuk7zYio6ulHwiigmqXnlT63YcybGX3URWa0MyBuueGqrvLFK59WmeLuU7pzT8G4oZen9zvsPadJdiY9sidRdjE8fspQ+Gr4IoUg8vm7lePWvfIfIA0BafhJlYg1Qfg0VWQ8H/oUo1hplPK4rWRp7PLUqy5qYgObxtgInk14OPHO8ppl8kNpjkOaHw+lx7NMcXDyoxyO5FvTB4E4nvaQONnPyuIo8PJhTx9XUIWROpN7Qh0w8n9aWPVT62U97/tCl5yEdpMO0npl1vByex8tNn4wDVl5Sbn4caO7y5JYM0rLyKqZrL4jXq8ttqHAkrt364gS1vDIrAmjHQ9U51BdcXS1SrBPCq4qnOCFk19slJU9Crd+SlQqE4XFDUnoW1lI76SwvoIQGiR9/F+ir9lphkC1ypX7KeEGURT3JDoF4KXV0E1DRuIJ/kibeEbBUci+GpUHxR0XMH6ZhnEqx1c12uwjifLUApypE5Go9tpsgUrq1GVolcQ/7tBknSW7tfDMn6UrYDG+bkqU7TuYQkjc6i+GmUnq9nzLvJNXrvdj8pzQ97mCEWRhJiOh6RnySKMJFsgSX+fGh81klH9vHsBWOrLXeka8k+T56pVHjcoT4PN/2ciX60dbH8mj7NHCK8msMHtt8luljgGXrQTm0oZ1ttHK+hi/+hSoGERkbuipeGa0bPirZI9Z5slc6PV4yulO2a6JlbKeQerJsQGb9kbdT3eTH4ylNt4OwHo01Mw8SevTF7JUr9DEPc8c8H449k+Pm9Xx8pYWX3+wjcyyYg56cRKRMDuZwUtvqAEDjdE+r6ukqpySsT0/PPbVhPT2D9rSljWlG1IkaG/yQqmeBaUMzz8+q3vF5SZzvY9eYjX5Oj71qdu25G/vXczycX4/BN+fozIojb67kRSbGNn+5F32ddBPFLv44ESZxv5TrnOYUfpmJ5XRg/QspLttz97P8VbDXQDiUV3XcxEI6Xx/HbS6U5zVvn4/XObhO+kuwVPmKM69Z0Db1MCKeZcaqmqLiy1HVR4iuhcuqRYiFhT+rc4mfhChVyRVHX6l9kziiEIrjTbdJvQDxkHQLVhI82GqxW6NTYzEstydC+6+5v4HqlDm9tko76wzxQnLtLrKthadaDvwL/xCoVJuhpzlabKZRR8i/S67QQWDQlUKjmxXvl/Kox+ThKp6e+uAcWORK6TgqrKIuGGIk5qV1A3otOfbdNyJSPpZeY/STzIghauCTsoqpZIrTsbQL1zTQ0wOpOzPhzyrSseZoAEOYLJPsLkS59pZyus4lcrDsTi0RPLRy5xaUFG6tvGbacI4Wi+onlN/sy2QAFdZtbzLI6rjZzWZu6NWyUdbqKw1zxJNdoFbgmJkTOe7BSdm7K/mds0SuIcWmAzrjHxc0Y+/XCVDhqrsHmc+j3f3eez7rTNKddALSWVH36IVohTSum0VPh/DdeosXh9FcTEpGjJMeUpT2tcA8bDZrawV9lFXeyJf1gagwoB08kS8r2eo8C63K2m28Aq3LOmi9F9qUWNL5AttWdTH6ttrVdbP5ZdY32ID7CDt0GMxUBFI0DhhyDO78NN8JM6ineZOZOMCzBXG5ITx6K6GIVOg5G6OcLmHvvThXjUI6QyTFWKPrhp6VuxTFEgiw/SBaMtxp4lfMS0TQFNfY7MjcbZq49KjOHcUYU+m1WbPkKDEECMMSH+o155J5Mp6gruCgKCbQHXIGcWhTved9QlHT4nAo2wuSorEwt3qUHQ8iQThK5s+y0HrPbIqVGxEnOWce/Udxy63bqX40oJ+8Ns/Pdlxz0r/AB2xFce3V6wN7FPAxN6+lb8XL15qOk0uZ0tEOAl3p56bonahyvlmWd8Oqw8HOOwup4OI6T++BVB145x66CK0g/jysU07UpumVp4ap69xHvelQU+ASCMaep0aLI++Em015iIeOiRpihk/v+a5V88ygZKzN/bzqr3KL+wXZG/fuGle3873rsr7OqYDtMKohzpnZX81Opequz559QY2HHr8PlL+Fg3qYjDNGDDk2tz5zH4h55S48Ot767XFR7mM5P9fhSowUhZwCM5yMG5oPD35qaoy5RO0EZ0Hcg8KY7Z5ch5aehzt9GeZmOchMTSPO8gBNYKuBEpdlfy/BvL4E3LTA0VepWWPt4DKXYEULAb0axnalJZNxj1sZyCsT8RsNV0ALc+ZpDac5bInnkFzUccZegnudQgR9Be5NHSns9Uq0WeVI5DIYmiqDhyEwKWKukSoiTGCbkXVBbdpBIs9Bp8J11STFZOtBqdWIJYofTQONEULIR88zbghRNREh3l+ScwS6WgghSJdYEonwlUPTvwQEphVwsYwXqb2q7PZQyS5oIO2IO0q4tMutkqlI6CEMDhWFC8ObLJ4ybShTcyNaerAWHThF6cpIxUom0pMBKyw9C+3qMVSCqJp1YB0BxRdvYpcFz5fUAx2cC/Km7Q+qEvjfNUoTsACJS5haQzpooPBBJnhDhzBti78RbhxgcS2+M+c9/jh49QMw6zOMi9U74N8iC6u7IKrVJuK4DzRb/LW0hxbfLwe10QW/sH8O8PUDjO05G9aJGnz0CiOc0x6WwliL33txYDfFFcRjx9d15Q7X14cUiYxOAfZL2kaGTVvxrYx5avDheR9e/Oe8AXy9ZX5qr/5i/vvVU/2WqQLPfb0/A4xDfXsHx4FwGpQxx4tHzgLhCnfljyl3EHVREHWwFNLGcQl8IK+LMBbgP2aYOP0ogBBA5tf0dEiXvfUBFWGiQr8KXKoLvG6Lbwk2BJDWK+hGlMRc3HT9Ydl3UTk2on1tZaPV00IfFF5yLgZgIK9Hcz4c9PYubakn5zZgHWl3LnQUoDVxszqAiI5sINnJrFTbcTwXRlYPzKi8Omr9e/eKFAeL8IyM25uYPXwlP9bnsrvZlQeA5VT7bX8rO99v+nPdj8MNmQIfny81tY6auIUBoBHNGZk0G40CF1Mbn4gvQDndGhKAoqMW5cmA2Voou8CDnXwyfJYUzNv9IimR/weS0qGkjqDZVeTXXVI814R7a1HXAVsArLeUDB40F5/HZ6qntt7J2RiKp+NLNRneycNJPEu5jhCmu71JC8qu7FXrgd+EktIRQzLK6GjaneVIzt1BQXxaq6V39xpfrvoQN9CoiYjY7Q8pYUpZGx/zR2+WpOlUXfyovT5vKbPq+rjt/f450iY6DTExB8NhZSxpG5PjMxxMhLWA90mAWg45FIlVkXPSUvHwDUlhK4jVuc9SOsJvoC3HAwxGufUKJPV4v/hQYOy0f4PY6P0bNISNgST1EnyDSGX9/L187l2HDYAOrjb8vhy4x/HXVv+/37B3XAdOgYdGg6XA3FYRJtnAakF2x2n0TJ7kqrwWTFWUyDPPU64FL+4MXtwh/2PLtxbxiWvJZ0HfWJQ85fMDAXGdCeOmlYeA3w4dtRs4Akb23T6H54WAG2OKQEDnKvi4xyYLcl2fIPRKQIDR6rOpbUO11bcV8pLuNuV9nvHRHYGgf1oy5nIfXs6t4J8l5BrOC8V6doswwCIAmB37c/s0ty2FrOHRQARjJ3q79IXmxxx9CmQxLV6nYoE398QRylN0mHzvO78QJPMwof4QNXon5heGT6nAjLAz20OJl2dld+JxFntkhrPnrskSL4SCy26szLfhZG7XynKDCG4bT9qmYTfGTCY54fF1fW0uNulgcBDE8+xxhT0c7oab3+dkgrVCD/iNHtzo+Q59omZwqUak9fLE0xEwKZ5NRNEuMy3kGtan7i8G4uHpMgiyaPQxiGVJb2Aw2FPhNI4z2ce1G5AgIRZWoOpiVSpHumAMDmgfH0CE4xUsgZDJXeJ5rp41XEyljRavnoWGmbKhOS0YBaIL8pQhT8yC+Yg4YTjsvtSYQjyoHtwgqKEBmRmvulBOEoIzEdbtPHCpz02ebIpKEeiA2LsYLeTIaCQvGLAokbsUeIvicwi9NWhx7BuW7qZxR+URcrYnOt8XOm7y/krMi8AuZ819PRemHMgyIFwTsqQ7DUWWqI6nZVF/3Ht1ImOCb8S8y4iaMba4vlUMCuG8EYOncjwFiLOvgnVhLXpGLuw8rI0GbyJMzTrRgeY+eXLWXhhKG61CO9pqowTc2ADc8QnpczancL9cuqKQNagvRkvEkHuJ+bPAHqAhH4VoRF16NE/Y84S7+KEsmK1e2pEH6kSbY7+cbLceNIbjB6bIhlAgNRdO/XhtIFO3oStL6YaFIV7l0y34Nd+feF3g12gTcaMMjqqPJi+UUuEkYyVY3TnDR3S4Nkb5IBVCyZf+skLGg1lAH4VTMSVi6jqDlyxQ/uKmjGGadC6lllB53AqNevlhe3aDu4sBtzsHuk6uZEZERF83lwMqqVHTQiLqkgpF2g5DJELC9hfCS8Tr4MdQq7yJMMoVP2FpwgjHQ8MILuIpJ+WaVvnjOghHEPUaLoVXc/DQrmdzIM6DHMYUtB6w3IWWhV4aGhkkba9NUrekXqhZA05lQITymy5GeCBQSQsqgBaoMZtrO0+oigmTWAmdwe3ZMwXavZE6JPk9fzVLDByGWKwZc4vOgbNIDlYeQtp+Ma8G4VifgfK84bR9gw5NUE8Rx6jVhb6HwFfTLVG82dVtvlEkWNlBgw8srE0jWDSbfMbCLdyBPpR7IbYytuwZvD/f1ERSDp5OPiz+3hMFXPyLgibOGyflPXZb6dV1t6eTwMwoCc0YqknjxR6rJ9LCcAHPQNJ81VQkWO0Oea7BGd66e6yp4eGRBCJZ1eQoFlTWs4Tn1eUc5DozkpcZuA++I4dxgTJK4aBA4B260GXEIgNRNtaIqGdLZNDZK+JfnrR8PMc0MHMKfr7hGseMBSxSUv3XADwEubRzzpRTmfSy0hHKxU9GgK8Jiic2kyENEVJDVHmRGdCqHOO7PXI9e0Tw1SISBMk2iYSMHBgHMxvuHegae+5OjXrfJr9bzk8mK23b5jYubzXDAGpsUefz66Z/WNhO5iJzKI7ag0EmiZP5XssSfbLsnJhQnoUBCfiWuny5sVDSckh5JRK7b75RYJIKfbljJLlydwoUy9yie8NcJ4F0lugGJG6SmxJvqkjcjKZgDHfRVuhDfLgMx2ZFKhMVSVRenyoirduDYOubqXWu7UaN72eHIGaLuyFgD4c0hn3+t7sVRb6MOYpWXWOwVvFW3eLHcqGeA3HrqRBFoZ5zAqXG1g3rnHVWocvdtlJE3uKk3X1yAusvmcPA3W6u2e6TY5ndxiecF2PVqsg3giSVzp2A+1jvFd7XKVVWuPc07xMajLIDMdepHpHg7tkkLmEHxA2MWNLylLjrhCkzCqfBg1Fugz95lz2FUBkvkSDPN+EVo7eelTT6ukD9gUubUQqJbsJ6sdsTUXAxK0QbNSD0wCRwBXQqb8GFSy9dcizyjjjn3W2S5ylRdC4ubTeJo5AIIoHX9YO5qoUB6AevdQsLLeolRYZ7oaB73lgV/JyhrT6G/ryzFBjVwkriK+fBy57De0Uq9Ja7QteNAH/Bo2eY3UsTInr27FO350sb5ppmX2BkncFB045CGx2yG1i2Eed6S5QzHlIQkByOkfSExl2w3OQAHEBMbckBm1LvdBLKhmJNH+KwJi7bOeJfz+bcvYX5082nImID9sbTVHCBz7Fm8Sa2z3xcN4rF46uLMjgZlWqeo8L9QvsQPImmuplmOJoxeRqw1/oilJE9HfBohyyCIjRoYXwVuNmyotCzKsPjomRdrbyWonyTzn3ooN3X9XAP6yQkFmVOZpuFGlBD0T+UmjcMQ+dsaHiwhvtmmX/VMGkhqzeUEnMvKiGlo4+LyyT3BbhQ5GKr/FwEeM8k63zLpMRi4ZA6yvOoxZefSykc42ZMbGc49ARNK0l2jc6QujnUpi5sE7DdYC8ZHRoxzCHJYuHKxVPNPPnmpif29YQ6p8k3OB341orMMXLHB2sgmOO1niMcqeZ4VJOUJwpGxZOMopjZIglouSCQkCC12FiYp2gWDC4wlg8NEVGZQuiM85xOLULzD0w7pQ9G2FEbKs79NiWLmw1E9VzH1nPrkkbc7hBAIMOshFkahFUXuF7QMUvHRy9se1JuoacL7dYLYAMauluwtLS9jM3Hy4wXM+2lTNhniGeV0l3WDdaw0QRZwUJBLD7Kg3+F6vx4k/7of8MCyOQ3pqLz72gIdfkjHg4kJOm+D4FO1NaZgWQBMzohm+B5jvHBk4U7wxRKRTEovRAE8/AtQmAueJIXd/egY09ptooQZ+AC2M3OymVl45nQnxhDkyqc6AAavnCheSyerRzSJCu0xSU9omH8BNPSpa8lmBafJMdP8AfrxYuRqKxAu+kroymo1lqwJwQpaK0X6sLu7OkqkpkD7AISgXLSQmsqx+15P9Elyh7gKBKhmohkTPNUvRJPmoWWVj2UVFufc3NP0TvtQqC8ahUUTwY7FVHq5DIp2g8l3if9/fFYmmRsk0MKOQnHrP9qfNX6VTd1PVqzVTzMBpzAUeuhToRqjJgkAt/6yHxzqBb3MhdhXbh4p4jPINRa9dTIJ+hwfXUmtJ3zxpMfWVUECODjIs3F+QUVrFkS+ths1Pl5YmBKluPdqwUjGIIhGw1oQsPE2gMuDmCGr8V4Qned++U4WiofYi+SVwAM7AzF19gmIWbANIJVlULchrwdq3zhSXcgaC6U14MVsD25hiqEwfiLbC+0spUDerzgMK04Ogj7YLULomW326xUxn7Q96a4s7mHM+2bA4Qa/WItnALiWvZTgLc3oWEqNHUY5dcwUIZWd3VQKVz2s38KAvPZRELmM2j7ptisBf1O/jkGTZc8Il6TxilpYCeSisEYQ/OPlUOP+OMwXq38HCvta0Bd/2pjOHlx6kkuzoNdmOkwTad40F7QOAMDapzgowC9VnRPHjKcMUCi7oVVQnS7R+DgsDgl5HZq/bSIIwJ9BBaZIgNdTKgrlzjTHgk6NJGNXmxNlIcXNfRC7VMzeaDHeLiiiY5njYEglkHQV4HEZPJFN82+JFmTv/LDUGMPdIzsZAMuHsunisUJEDQQyWF9vi5ULHYBhiYRe4+45NkBRYSkR1PftbEb5tKXTT1zbO+k6Rx0lipgNhOWMFKRiFNDtEuSQUE+m5MkBXdLlil0NvgTOaSE93BuxsRRN5l/NUW+QBCTZjHciHSz1tldl+BuN518piEvKqtohETyGr7ax7kBQRDLVhCMJPNyCe+MADoV9aaVH1BmvV+fy7Be4B2um2CQWGPZIsa9SW19UywbhyCDlEuoHDk2DGEpuAIcUGjWTUYoYojBWJ9gCetMBP26d4l0L/iMbUfWKf3YgZ4P83B87pTxBgHZGSOC8zprpftHIOmN7WLBDfrRkwpCOMytC/JZbI1e3VCzhrkAqLGEwg6RlGgufQVzwlf4ZH+XBNoxlpah2mysR7rHEq6cyI0lZdnCeayZ4SP5DD73wHgphUnDa8WOKYEe5xcIEww2Mfu11x2zpjt69/Bk8Qu9XVKXhTHxZYUpgi0hTkZXupAvSHHXbsjLUjE8mYhrmZHnrQ+5hiePxICE/XOQvCrv48PqqK/eikroRRfvEk3USZZPm6oJCO70M2InifUwkmsImA5w4vbzbJ6jiBEuR80xoY6fFwiYUgE9CPyAB49mTKQ+EkcTkxyCJzJfQksJfOvKVnIP10jMj728UaVD0ns87UOH5kOEhD5i2qA01oO8KPdJvYhhfqOWU5DvtuQX0RPamrg/Dlmgp/AqQNNZS1odIHisRJPplQwQvM1q6wwS8DOtoqkEIslqvRHkOl+23WceDhGkarxzSoDd3GEuSFg2QpQKYowSVlyOqZgt5q3F0rMYMiAuplg4vIArFCkBxqEJ9e1pyeZ6e+ikNzjAinHYSw9jCqpBZUAhFAmpnHam9nVpAU9yCA97w2lSY4wEugTWGKbxBmd1twWElifga19EuzN9DirZMZr6a3Yah6lKZ+e1TQ9CgEOPXjW8PAcxMDB86rLZ2NJ8qYV7pCbmZNQwJrCaB/RBfMv6bHEaQXCmSDbjZrgfyvUCJePU8ZQRCKjriBsA3iGBYJEqrCBNoY5HixJhdNCSjtAbuJOVcte/R04rXxZEpRHQKytnyupFSG5YpFpOYdPb1L6lQkiJUyW4V9Y+Mj5XVVHsJGskDUmc5EQphFTf+CZlLHtFrfiyOwrIOHdwRYX8XyP5kAV89vkDQy0YuShFW0PIgKJ+JRmDX+FKPw/LAUbwtloviO1VhkkrnHJSARHZwybZboqG6KlpEMKLcg48AwP8pTbY0BuYjnEZMhaeFjXxWHXf5bXy83B3qQvZwPPk3B0yh/t1YwFu0Bb0qzULIRuKxHg26cU0+njK77pn4vLu4+C+4JToTigwNxZu+j99hR0fiXrbbgnbTC7MBkhH9e6QCuXUIPQatHGCyFYkkptTysq0aWabrlAcbZbTbPRgBWNRdQjpnM5XqY8qw30heEFvNy+5ilSBYYIUl7G4yL5PWA/8md8IRS/aN/jk1HiozLIKKQpzUT+MPWp4ajlUvLRfGG+CBnVYaDle/KZ1VnRIKPKitIiIJMI2RWcSF5BthBM+l0HBvJqLUpbsM+5g1bbnGSaeOKCXba/QYj/r/KpGFBDz2eTiHwDjgSyTiaVkw1OSO9LoHnKmmtkaRNI752Sbu0tNU2d1gcoBieoRgjNpxapZjW2PE7jUdo5wXU3VfF1D7nwtG4wbLDTQ2QCRoeY+KdR7YWONqPldrQgtuV2hMaI307LvrEXeIUmf3oTCSziBURWjaRgI+a9dBxd7fG0R95AmuIwXtneF864Pel7CFXMDFyZmAEdYTZvhqIKAkwoi5kp1HXGBZEKGzrN7lREvc4852UJ4n2+AgCm+TYdMyM/X3t1ACB8udxoGsUxlaggODzPZeFp3wdmC43N7GIQl+cbe92jsoB5LU/bdzAbxLS0MLnPE6RDfnujAtgrbK5tHbKiHM/22dNN1auubUxRjc7050LT4VYTyjproTiwX03+dNvKpKg2CC1IC96pwWoeLSLtaxr4ZTZIlmyI+LrV2Hyj7HHd5XUcElBEhdrvuZxokeshVuygENtzw6bHmLUuDsGhin3ALZN2P/p0IpFHt3naYS7BQrIZqCnVGB0tX44W+BfblvM2nxyWpDbR49At6Oz853djlbT/LGivZAjL70Z+z+KQGjEyRMMTcWTD3ugaJHV1qk5o853YoRwM/Jy8pweECjpD+iyASI+4LIyYCCFuCU4me1zzWm+XYYmB9HrtgGuw1nBnrxGVl+Xg4BvpoWn6HiN2tehwju1At+R2oGGj94mS6nhZQkUxzrEFinkaqLgOAXuhsrY/5HgSQt9h0k2qELmuOlkWbbgfSYicnX7IRFLtD3svATnzK8rQEz8C4GrtBpPjsZ4yhhvMpD44DrzTpg79pPRxOhanqEwMKIxBmB0Nc5cFS7bCBmDA3EuM18MMdM09FOnM7A+Qc+loMx/xpNOfxmiKdHkZPSBzlxoi6NnKhg94uSCvfLg5RkcrzlgSTC3jFbcYOILiRNv92JfUqYYlIQ9vs/Jq1A5p46ON5XDALuORYWPIhuIv0Y2Idfmqe4Z4BYWSoAYICDYB/RiAouasvtYu7FQ8DSZlIJzjeDmpHDiybcEt+v2wFcUPwzRBYqGWp+UiYgwwn3Wur8arQUHGfWICm6FnrLgkNOYjHBEThxa1D3a/5dDq/AD2EQiGVOwRo8JZsyD6Qv8oez+5vksw2B1OFbPAL5gj5kmuDGSu7nl16rB+jrpi8CBIXLO8gsb0v66Wq62Woht6tzehJx/Q1hAOLkhEqKYr3TF0lPHQad34aS6Y0BQASO1NengoDmB22uMEd30r1eN1ryZIJrzFJVhlzQTIBwjDsUqpg0PRIhDkF2jU8Kta77xLe13m1KA3BEKnoGLW+lT7vL7ZQAG65nppenWZakq8MjDX4YJGDp0bU86K+SkmHCFrzLpK5UEuLYM3UfhBs9ip76fXWsDQVsz7SnUz0Rl96xI9cnK0LaAoT/CmdTtLgjIyLT4RCojKW++O4lGqwLOzVoRLiAixHrcLCiRwZRff8VujR9ZyHZRmjeWhYmRUE+G1l/BjiCOBtHHPxC+LE3O8UXtd+d2CXcF2QGPqxVk22LJlg5MuIbJLDUPE5EExTI641DK52gHmy68tP4vtoWYvckZjFc2anPODRaCiqQHJMdrwINh1idacbuGhFdqXfuo0WRYdN1svGFrEcGudI1jSzYWyZPmGYnxjDgt4LZnnht2vVQWp6sBvf3JNK2uXN3+Ug4kobuIHnTfNCW9r2RLpyUW6qOF7ma/+oeLKOZDnaVjWDiGXD1JQULUSR8ICDJ9JLV/YIXxz0i3a/HHRRlomCVzRkczsx9A1jEMTWEDnHsR8cR3FcxJ0jtw5NxoVJ+1v4quroHhQX+mwl8yk4X97yyVLPWu0qO0h4ydAM8CEKh8S+Y3aS47JlwDY7tnnNPOnLkqJEkiE5zOryp0YXonEfbygcGKHABsoEdGW9ekVo5NQwEH1LvmK6xHCWvmGVkZnF0ya6y7iTv0p9T59AyrTkBUSkal86xtfytx2Di7c/KJMR1L27jliXlS15mcj1wYZjKSeUDhsiDvZQrqGTUkVPmMO5OfUoMXVYAgCg9ZxeuvXzfZDUOO6etyLmEl7J8JXRDnFGWT4215mgAP0tuAITH8eNeuzir25vUUQHmR6vNZAQpy6jb8z+bmkRFiVsqL32dAmw0ILA/aCr+c47nCmG6TRvqZh6L5cNWGd4PsfJmLBJ1bOiyQP6aZ4V2JEN9ctYvW0pYmWENjQjS1JczPCO+yYbU3GfUGrTeF0lKTKSeJoxePCDbbwN4ZQ7nuqEEZFFcbcbE1EBEy5F1JVzFUvRqTLJ6WJS+oZLZGiU19kOhIvcca8qqXniuLmse71eneVQLVxr3lS972+pHXICdajyosiKuyzp+fF67guOY9NzcnflRL6rRLEt6AKibfk8HQ5YvqABtQzk+Xl+Duf+znfxq6MmepljLxEx0/YjXlxw+VEt/a2Kzw6rUWzL3GffZ7NzKDI93hrIAHkTqqzbcK/N+hZL+RzLbikvdX+t/I1tqgpIaLcizEsd9hQLbpmsL192h3HVuhIl6LZJX87kzBJDqe0jphuEhcFMOvh4Pl2ii88Ftv7geh8BPVZ1FYtla0SYZbcUUkkhSXQM3Fg80aKfDOjLTx8yHe1sj4VJU0CsPA27aaKBDoSm3+2XV45ek3KsaAuBS0IsEWecGe4PcvfDzaA0zHzeQxNMiACsJ250KLXncbzmAv6YIC7oAxszCrNGQn5oLMo5PuHyKtC1ZdVWQAY5eicaAPAGP5tLNtQN2YFMcuG1o7z9Jkc2VOw302kcRC5Ibt4zwlr2YmfYQl2IA/Lc+PVSk489BoAb1+TCiVoTn44Frkswjymf39zGgrE4bHWAg1TsHQ8F1oIpt/SwpA0RlsJ27PbjYcIr2EM4ONNFPrRAE1HofcrHCbbnIHb2yazwTOwebRQll5evJec9lyWdhLeUiBLUW14jv25EuHB+4oUoY17LNMJtGMgKxMQ5Rkekh+4LrTat8QXqwZ1ObHrwSYGATSYx4XBiXS9hImib8fBHmboopxGur6FgY/Ic+iwNrVvOGHJ1QtCWPF/IgB2wKXxGcQ15zCusXtFANeMknXU9lEvgzPQvcsEVd6Js7mS4Bb7tCVYYnh60NZqsfQf6tWkP4C1flXdS6ULp832dzBlqq9JsZ4ZPpGZdeAI6GBnrV2rrnFfzpHPhhhA3YV155UinDmqut1Nh3I8glOuDqDq/4FBplylODhU0tGq6Krb00WIYhgsopJmW0dkE0fPT/3V7/WkRrUFKg0p5ML6eYioZdZeEMUfuRnd5z6Tm8ICJRBNw+bdh04jmjct1QULv73SUguq4VWfIsVR2km/FHZ6A4XMUuMoYbSx4pIKDaWhJ2y40oP23Rn2E0P9DjZBIL8n6ZFTgWNlFIcTfI2nc8V2xV34/r8T9QtbAFwxa3ncBoXpBZsYiJ7jLwSt3RXdDLGG4Zd+0SPD0phvxUYQgiZsaCStLEETFjUztgVvTc3IyTq9G65YVLuS1WIcTGU8rtFFmj+SOTcEblZLfphNtFvy+1ytdGATqdFlpR1SKlFNSJKxF3dv+drZej6pkjXrfjjsW+tf7B1A/gh1jOBTTMGcRiCLKDnhuokIaHZJzvdUL6SYCiMxZQkX7y3YhtKVjz6gs1DdqUq72MzGqFUjxfe+Pdy0i1uAfkJAecCu7tI26ZLRb3ffctbnq3g0u63pbiLW6OXskty0cLdoULljtfXmip25AzXh5QkAGAa9eKQoMp6IzXGBUoa9Q+rzvwmSRJABvtW5MSHuaNBet3mCj8qSfJ+5298Yb5Kh8JyXIIRl3zaHjht650gFpJJ4H0bdYXuPOSkU/iKXHZYVsivKCHNpvZvd3vJkD06UNz3zqZoITDYj4ahCAjkTDyEzxcuYpg2kOhxqo1Z0h/Q/VHS5d8bkqMAK9rmz7SjQFpvz3uQuvGQDtcIQfEYnlM4jqL8kKdZw4ww8tTUnCo+0lWF8EspxYLeXsu/May/yo9OZOPTkmHugU2WPQ42VIJosK5KNqMbdYCfbNgZsRpCDy7I4wUVklxe0R84ooydGo3hqnwO15T9CbLgndasdaOvBTIeRETE4sCNx1hcd27iercmAIQMHjSdKJHvapcTvV8ytU5vadmlhehKkqH8wmzYcdCTxTQwYihoHyvpvG2dCkA2gm166eEnEpTGaabjXhb5NehjrPyhg0bk7bqGKfWJEfYOb21SEti1/q5LkHVIHbS0kh8JtLzxeociZONmyYbG+yDXbUhvXektX7fliTmxxViYeEvk33u09r0K3ST6HAoe4luzx96JKJfZNcDohSJhk14WLEXYcOjuXEDtB1eNHzRPXh+VGwB6gf0o2cLz1bq6/kZbpiRy1nsgwY7KTq0Jc+QRwl3A3yjt6hY+ZHx0Eb5pxUQkBjwHkqjEUsolvwTJlcds8G/QCIMFQiRXdfE/jkiWgZD/4/rPzgV+p2itygvntL/wqbwWUNuBw8T1C7J3jhxGgyjxp1HWIZn03b11d6TJx5D919L+2rmhLqhT4tO0S4K7Ms0AfQwHT8qkeMbGb9Zr5USDBMdweF/i2fBVcjcJSKhKzApHIGEdHVwab4njPlkFyX7CZ2XvaKSmwfc8Ga1BugFlCodvr1CaunSzRq9ZBeLIJSkJRsGbd7a35IG44AMS609nyKYR+iSTIYk7vUO7epFkyObupG7B7ZQCEdaLBp/53ERIIMNL7SLYim+PQxdKpGTwlyG25e/HreZrjndoZ/Q4JvqIbq3qlNAB0NV79RG47/p7e7jS+8f7RgHcuX6K5IVBfowrX0ikoZxcc1QYDOql20XhL7eCla6SEvFAcPdPGVAMPow9xSuoPL5NQ9Vi2qjm9HoYAF+8Vj6e4dfqueSTx5TabBNFVC7ubrFvsnvi5FeCmk6XzJ33QIfzt3pM4aSq/kdm6yPeWSGNABsfc34OIr9lwJa3CIYVfVvXML6DjUopsqMVB1hsm5RjP2nuBKEOgLGK27qbAvxrVONHhmkrrj/UVlXuDUGNrCw4B80Z8ZAymIgiqC25F1dAvGo3i2ogBEgr6qerSf8MesGMtLGQ3bTAm6Ud21MgvGR+g6MwxLKYvcTc6rkThFmgy2ctXa4eY6LPwwiHS5k/klrhmn5bIKvzwFvB3VzceN51IsawFmQq9e36SDe9xU328CjT9mlHi9aejmAkDeoJ+TMOhKeQ1EoiwwSjHVsJdOzyYe5deAUbKMDuEStOa+EHCt1zvkAY/nrMbQHgW2X4j4XoA7yhzDJn27zF7LJWXFcS+m83KS5bGBHRlxPhNeAEBw3sTBZgzvxEpqsv9xC7FGKUwr48t1juK0ZyQa0Z4wJAyTl2Xb+Sjb10VnUHRJl9Ugyp2Dz5D7KIsgzV5H3iCnzfR4UFq4SgPV3a7HOcQ9eNjm5kuH7t4bnf2Y7e72NNEg8ZRTSbQDe08yyCueAgECfoyJWALl0JhdD3aQS3c2Q4mdn8FQc/ITrrGm4wiBLj8R6lRNxGPBI1yQb9ztJWDKzWcOxXbJzvPprmSJC28zd5yN95O8j903XEJijZ3X67Y8cRBAn/CyherI141zWoClc3Z6VBPdpW/LmsHJ7m6Ek+E5ZWmct+0UqliA6Ar6Ko771rU+bNDzP0mwhBJbkVHjCdalcg6DChax6JpbfEag/omyvFaRfa4gYE1z04e4sGRbXYdkcgGRKdGrtGHXCInhE+CZlhs1BGU7tMTcl7QjIK6LcRc7PZX4l7NOgMhnYyC+LG0yuEV2A6ZPS4fFGnTHrM6Mi2o5k+xHzlcnvstbpK69gUNKOQQjW8PPD0uHhgj8jXkkudMm64acZTMO7sNTxh5GcTlJR+IOFa/NyzHp9tMkzUoyEqaAn5x/IQRk08dbebelXm3Xw6ME0fV2fLkEdE+M6CyYGulIAwLCMVHYgQzca3DHss0cMuBFRNMK2Q6AMqsCJ7m7wiNeRJsrXMNXVs1yFTf6Eg3eK35zeNzxUFYUrkH0EMc5klPGUGNWCxAqiUGs5Hg7m5uKnKzN3RVcuYREdxBGn1a8+W0/BXjVRAkTj08zoQeuHlROVYPMMMVMvz8Ijq7fjhQrmzgNaj1FsvvygEudWMQBanzfOqO1+hkR4xrlIzOXce9F7ZmnpamDbSGaOokvXcnTF2RwRk2Z2wm83GNZvAAheyXFvRnv0fCc+yoxapfYGoZ2lgu99ZArJWqSNDR8l2kBFQxufi3IoGoJHfQK/DHjuOSTaiKZXbqgZOgamklKQWqyNfQpySCUZXtYu3RMIQgu6UNoP9jQ4A8Liq8h5H1QDZOtn3ybMCsGpxdqZJIiX2gBEiUv8tT9HBJvsJe3VLJ0h4Z0rP/FCPxvv55VHNYL8Uwm6OWFJKXEMdIFd26fw2yi+U2Y2HvAH9j4yvpE+HxpB+KQXYC7a1sb3ZHzPjBVH93ENUokxzFWRjlb1JFlB8ZJsOaN5Wm8uwULMUTqtMTEvJ9sjy87oWO4Yhswc9+yApxGjSrash5i+mJRsKpsCL2RE1uJAtiGf2H84LAuXnSLn6j5fMkunpWxOlsrvz15fADc24gvTPbCz50dqhDTEps828HmL4hvTeh9nqYy6fBDrJNNVas2hYLVi15O43UY1S1w12kmGdyhrU5ZaaR3LRsoVOPfDR3nV74pHWJyEZQIUFOx2VNW6Uymd5AW0ABEGLRGJoGmr1RlisdZGOP9XFXCsWSb1/QkX9kFv2m1S+WjtbIDi7YSdSKsWp0Va4RGXbjITJ2OWVFy3l0+CLiUaHihrU8sc5sGI17BS9l3/w7PxOw69BbgnhwLmz+cEB+DfO/KTrfZD9XzbeKZlY/whdUGF0ocXs0tW41nk3lVjzSpdIWDn+TkJy+nqvjU0f7+CRFOEa4KVEViiBnZPylDPB5uhPxosyWM91M4kyMfsmhEW+Iuy3ZfTkFShrV4Jh6C0oYOrmXuVsS+FV58/FXfsQdxs8qhWeoloAfrcmDG18Pazg+AqvqtQQZyRmlTQVybvEAr02+O1ndtoVqIfyhyPz5b3avoE+t5G/vLMXsOnKhgqj05yGJFIfIkkOB+FEFQRhe+7J91vowUMLCUEUVFCjNGUx2VbbfpVL9OQROwjmYxPU3NEZ5dDKpl5MkkT8/uSSlMBTRitXdyc4InaK4wnbQfLjm563HspjtzpIqrdxriq9RFGlAlDSIR6XmvFCqQe+TkM0wnpB5zUqHHkxS4lSqvB8iS2henRfAoO/+Y45anaANoKZo8N8PaJuHa7gMv4OlRuFm1QSf2WBhIJm8eIPrMEaZFHnULBnworvyJrviUBGNeoLchPJVYHAoTOwB60Kb39TzFl9gNGrfCoUk4LnoDczp4VnV7OFdvP5rim5NE90r8xFGExu57tlm59Sjt3esWK6si2U6qcO4cJITnGUc93oOXRWpoGd9G5iKRj8C82EEEVf4+CkWYMFwqm2is4PgK88qne/OSDwSj+VUZoZN30F4HlO2ykFxvR7MBoJ10pwQp2KUlCwAiCos7xUl7uaRSLtmydkTMHO888lpx38DK02N0Cls4aWEZ+ZQ8k+OSZYwJ4kbRVrL/196XdTlqLN3+mvNYZzEPj0IIkIQAMYuXuxgFiHmGX39Jqtoud7fLw9enXd2utr3cIClJMiMjduyMiCSNu4668AM4MpfG7uyYpdMBbBUz57ltu0yoVv1lWaF63Ric/e4Qp+ZtkNsVgRqTfEwGC6hILHhoU86MmaQG2uXYOLZmz9oLWJYVbohXNMwj92VjTKPk0A3+2ZTVR61DR3oMXUygE6Dq748BW6cxcSgdVzhheqThQrGuowzrILV9uRPO9XCNr8rQFbLPFlI0mBTQ+oio7JR7VtI65V4kf8Cowy71drxwPF5xeTVAQ4uczzIooWLuy/u+LrHdrsFs4LCsDlNcczaDS6dnT7jEBw0tmEDAB7IpcHZaxHOLN4YzVWimR5Nn26sDjjQEAfw2LhM06QhtsQ9VMJR5YlFB5SCPWxLHiXFMjsOVUe87s4RMaxBree4csTcwS9K2APSDhZ7QkFT2qeAcMXvqsM5Tt32ovHhELi8sQHMSWzwj1wAZcd2QeFAz5lSWErQPWIpQCKlsP6OVmWo4YA2kEBg58DJsa+XIdXBgJs94v2hZ26z0JAIzg98JzTPxCyV1Xk20jiq6MqTfcQxrVmkMNu4QoDf4Yi53+2CjwzWFGxAjdlQAfkOylm24vrQsADuktAeAelwx2AjzJDTfUG71bJJqSy6gjDwzUv+kxulZuWSuSqgYSVk92ZVhPsTqdMYIAFsBtmtsoEhSVwwJPNjDbUYjN5oTBlObW7IwmkmUEdALzcDA0JwaQoPYK2lfPcbTdrsBDnqSX5U45+xr+HbTHiV4WyAmsI+7yMbVJtPoeRX6eEywsJpNE5ewGNpcasnM2h2Cm7hBzyzUEEGDj4BHeM5WOzr4Te9N5THL8WRAlebBF79vYDdTqaVvOH0cym7BEkpFrBhxSVysHWNHh8B2X81n+QrvruSGQsB1AdsyCQ+nlsRAmF3ZWBIxypRhAL4IaLpqNzuxGLJquD3dHArG9arZP+xu6ysbcET3w6RKAUkfoPq83+zrOSCjfIuc2ZhR5RBqOHcnkNNEezRpuFryQGAOlAVIehQFiSWMic1zscLWBiKsbfuABIG21QloUBy4QjqRDLH9TMs3z7vqDFNfrDSURKlTQiM2e04mLm047QQBPjVkdi/i/mb2YjCdeGxuzv79To+ggZIFjzxS/RI+QESTKYlRCRMVUeBebhm3YaebCnfFLbpfaN7r/zyzbJIOXj2jNhpoV1yYCixOqC3K39u+ddUKAbE4FNBq4Q7wINYwmhQIoToFJuWADgKy6iTdHq6G9ahYnJb7Ad+niLDNHRZNGWRh1DViRUu6dNHQ7ZYthmoFx1BDHUvh0TmZn7LH8Bpt0FOO0ixpVTLa34ArQNZQgcsmGd2HRYGbES2h4QZvuZwyYc9gtQEEcqSPbRQ3kc3VLT1dWq+9kJFOUOYit+Eh8KAbr2EpOm1LXX7GRB2SELRHpG4l9tht9RNcQtw5kcEPOEE3JwXsBgO/veOHvQfEsVceSb8apfDoSHw/PsAgr6AxWvujYCm1Q1bD2zTa/eikCxCGWApFFE4DN/B5eDic5eGWdOb5At9orbCqW86C7jC0agW4EF4T6wKzdZ+PbnfTLGm8xNMYXemaUqlzeF5B+oOrQEFWZlV9j21+lI4+00jn9WcqOTwAtVQ2RcFMDmYdR4o4AAvhL3gICzqZaiSiTb1KQc5UwEJjr15gH3BDUwHNQEE34NsGAQfmCkiM4m6pDG7XDiSyi1ZltdoZ2h1O0h6WdU3S0x6Wd2w8lC46LKNyM3ViifNs2Djeq4k7q9U8XrpOaC7XGeCNx8bvCDoAMQZMsBVSXd340EyCjDmrsqpX1dYgSAgVvPEw6mhqigTlUIUtTUGta8ADM6Gy59tU7RzNMrDk8UId34TBhvz9/pKWAFtZK5hGPE1Ss9aJ72Wo+MEANUg6ctidr12QExz50qZamG5kFtiJrM6FvCZXidAxzPO5GUEoI9jOLxSOY46EwJ0tp+flqFl097wTmGTYrwsteZxOYIv1JaXIxFNKk54zZxJX8gq86u1t1UFBTe9gwRAjjTyD1IE+jY9TQRStMjV4DtuGQ1sDLhI+LhapcgtiC+Y51KdF0VcWWA5IkZppzIFWzwkOz4gRnStWohkb6vhkUhv+PnI8Y4NkkR29MRiorkb2bpKUhYxDOCAntDIdO0gu4yR05yxrU6APOCrZQvmadkCv98hIhrw1iY66DodHMA90jMRzXQTHkVYhbkUJErQj8PpOAR/N+BS8OBD7znXxG+oQMmO3LlvrtA4pI4MBvTlaBd3YW4CZo1J5WG/8MWs3UPgwGHNfX3b+zhwVQChOkVBdnLY5xk0MVhe6WHy7AjUFyY81apEbR+NNN0mJ5dwNEafAbmANHUzM14sHfFr/KEfHYXsnfRSoFG67u8POXOqdRcE3nhfqA3EBpp4Lzh0jrAZaGzMF3Xb99JhWtlx5WPNmkT/OEhmc6V2nxFmkXWOyD4wbnZGRnQx2cx5bWr5z3W1KJAYRwgp3SoxD52LX8KZDs8sZcc2TDEuWSo3EUtJoGVCeTBBpZgkrLOCiGhitKaVUcvXysHOv7Pu6d5zu6JgqQuQSRYKp2VnDolG+i3KNXDe4gB45BMD2YQvExweE7Q6DodtSmBnJ/jqpU4fcmh6NMptoMPXWHIAeO7qS/EDnxchOgeg34bCq0ZkoMt8xMNjPMqNp0Mxp4MGjjKgK/rTfH8FEtM0UkAegIpsVybTifsp62nhYZ1TVHN7HDBFdbFCQnAPlc0dRoq4eAkIdBaAMCsQgM0S+nE97v+sI32nIbQcHuDdHmHZURFm9ZsKYquBEl8/Jo1SubVvsB6eh6I6lFNDXYemY2/12dZx0ZGxqCS82F4gHbIMpRCApiK77q7aSPHHuBjQqo1bY5frA3gI8NXOoatRAtNGoQIARzN0uIEwJz3q/joZzGDcIe1U4eAjPHOtZZInSZ8zd3mJIeMSY1XNbA2kk/IJqzekBmdMNFTK9hoyaTkfVxm4w79CLURm+NgRhRRsWjbJ+irMLITSW16xAJ2rg/uZNUS6gtmDqdsqDJi9eDd1UdFrRcMqSYOFNDbxow0Hqwml/lQawfTKxtdJYh1JL63Q+I1wNnQD/w1DUtqM2klEokOWIHlG1IB93gu+W1f1pIt0xO3FqpwfGCh0ZaYMv9kvbmR3fNZV6eo5uLJKBUYAMxDwFnY1lus/WpLBsR7vw6lHb4pHex0l5S5S27nF6Ny5X1DgGam/GM2svA9ZDUsfCTgkLZG3j5UKGWQBTchBdME/1HviNDfjoFqjb2qywDNoLE3uLHFRGVPmG0Kzlrz5SS6dpuYSHpr90AwErz1lLQDQffcDnJn9yBUXvlcqGHEYR/LHxr49Ls3q+zfic83hlpxOw5sB4WL1CIuQzG4S7ygbHWcfKRjfq0fwUVJLczWQ5XAo1cw1hXX7A41EUdrWFbrdlzfOpx0rTbkusT6yBCy4wj0M5rdGdHj8UHRZ8kzZMLMNw6iw5WtfwBaAu8Ygk7lGgNPQ5OEeIZWfIyRxuptQDaVWJqUQapO+ss93V6qzkgi8MVmMv0uqvXm3rbq+QEbg20jBUxNYTpiEuEGmItilKIrF4fppSKSGgjVzp+OruntLLkbg0DWbZIEpikY/PO2WP0/7eTiMDVy7lUV540eWFkCo9OsonQQD47l6Xql84ACgII1VDvXUCo7f5ggLdoDayn8HiVssUCNqwxdCUCqPELbFECdFvTiLFzBEN4qizG5+4YkCQYeNaJUb0qKMSMH0knGoVOO6RrZ5MbAg2rDS2KmtJbwoNVncCu0I/iAaM5oTwDLEfLgqQzXSHiFKJFAfeSHFgmHqcZNEG7JsM3HBA76l1oo1saA57SzZwgrrDqaYNdRjcb9BGSYSDKuSQ79OofMRG+/GQdzK+ugwqJbmCUMr5yWNzdcjBWODNIwKvc088XSzdou6Vi04mO3yMDBhBBmISyjzXUJJqy7GkNbdvlDw6pxXAV6cVVxquBXJtpJzFRsqc3JOXQ13C97YAxLhPYSk81B1eB2h8hdotrbs6SfaxDAxKcx9odp+XWTjRO4XYmWSD3agzMsQEG0ZotjEiIKxARzg7MGC8UBeDEvRuAHMF0xv+5YYdxGexh5q7k5jWHPm8t58P1/nKPKh9TS/BaCVklSbDmQ6p8HQfLWXpiBTU81gVDw9VPY7XqTVdyMEAINOUotUapmKLYWQ+8CuWvtlhzZWenx0XWPHhGcYjUHGFE3EYwm/0cI20btHBSDIDUe6ogzzpkKvj16iyqJua1gvsymxlcAss1D6+b73l3OLn0trd0ppBG7pREN9DMS/FAqR/RCxCWMFxx8pgdVfCwt19/5F2k40zqxnk4EZrrmhDOgedRCD82DTK3aGuHfBzBWXiWsnscC3rpHNrzgMlqpcwVAy2sgkR830TavNF0moIPnMaZIukYaD9IPbp1J8vYDI7nZGMc96K3cmTtvxmSFk9lS/LeyjxHIhZ1gM83M8qE9PsHYbhCJFf8qJ3W8rgf54zwKutzswDSsguSgRg9uq2uhzoqmODbgx6ZcYbvFhVEZIX9qUDwRMKiuwvNLvVEgqiGqgrGC2H+2CsdvoUrX5E6i1IEzdqthPzK8bvLqki2JAgRSlU3Q4Xeb+BrM2/sC7bM4FAmcVyqutMhul1MGAItopAIJsxSrj0UqbaI+9Xu3lCYF2lLYQmGT/yZZbggZ9WEkrVN9cc2NTYpGgW5cIDYWN9T15FNmQXU6Bs6ro90aWP0DDZLhpyHb7VGeqVMQIb1IwlFye3JOt+v7qEgSCArUtKEefVusiPx6ICXnK3Oar/ASFnq+s909AcRXsrDcJSoa8AL2WYZCTRDuyVqatPcFk9yq7ih9PwME4n5KEDzaw05lTTJ0FVF65Bz0Tf4zMae1PMhYvo2sIshyiYP23QnSP0MKrjbi7aEDQf0iZ1AB3bqkTm1swHnGyGSq1Qu/0QShdpDpRYAuanke6oyalieIpTNzhZtVGUMw9BtubtgyZf7pas9vuRyDstSVbxu2enwWyBxmYY3VZPh4t/Z88oKDdhgCIxozBVmCUntoLMEO+5BhWRJ/psj2JDEnZ9bSsA3Hz9IaaQ163Ob20d5Ao5dJl8qQlQ72ZGRU50lw7EBbHT6kYix6YGGCyUcuCKx+d6kjiozgmlkcFWA756UnS1KjA0Jtp+s64XBmtWP8ptSSHWE2z2bQhEYjw0X8DIdb1VMkCWN9LA8gJb0D29zTpBXVuRx2GZ0pYu7sNNFva6hWITzNBX6qq3enZbWp3Y9xslGhYnXiGvD2GRNY6+y2ciIx9iAUkywo+oBDvEqQwTK48tDcnaS1EvASJ5Erd6wwFthcxZf6BnCwMlIJJr3QVI0ERXHhmO04lHTd3Mk7vrWfXZzKourtoosw7nSSSBA2vXTWO1yNq6Z5sQopA1PbswQWmU5SpYAhfT6SSwe3Jq7uqWO7pv7r1DtIQv6ea2r0ZljtfiKSDND+D9m0PK5Mjzl0HIQaOWRWybne/zjCIm5CyNCqXbVXhftdzS0t1S4weo3FgU9LouDmJPdZ0zxWzM7TAjwszmXGfVgd43+eHs4hc5zWmW4gm9rGN6KPQIRVaU19cnlcOPOwEzF/BYLIk7TmznqSE6u7A2huVxOlvHSxqhBcES+xbmjdw3XM4dxDysUCFpJtZZVZc2CXjue1dl9UJOG2V97gN/i0uCrJm5dPDlod0I/IZxhtjTvUfcFlTPcgXfD41Ip7Y6AENQR4cW5Yc5IgK3wrPioGbWkegKhbFTQQhB7izD++DFO/ZS8wz+6ENN94JOXoF/hzrIrXZncge7dI5kkQ4ODWNsEBhkL6O3hzvXordKaAwho6pOX3dn8mv5MUDzBtSxvphcAJSKj8ukB2J+4YGIFYSkvcGiVELCgjlqCQTvL0GHO4AseZxfUBEjp0RUI+VWWEedE5YsaCwfoE2fA2qBByoeqykRQVNBvQ1SAbUP/awM/BEnomshRRlchZVOmEETjD6GHZUtKHP1NIZ7q6vYvVkau5MXsMuSNSfLLmGgk4L+cIbOiH6tIME1vG1TDRjoJCTQ6K6MaKByx9twBP5VsqFGpcGbxbaGndwQFqs4EaEeZOJFjzIbPIrtyOWwpiORi8ijSwDdqKSfYniP5hiDrKtuS1VWZBEGryafJo43K5BnUEeymWEx7Ok6rt5a8M53P1Bwk3Awh7pZNOmL7izMRZSyAUuztk7AmAalViZq5REPl2qMpglASn6QT1HcNIcAGhDLukxkSybaXoGAmLqHrCkDsbm5A0eh/awABkbVTGGfqpPIT+iB5XW01vobZXwelXrvK+V8d69bQL3TrZi3RWWFxAmT7JoNenljP8lcyusgSY4JcLzoHaSIyOVRUARx3hNFh5u6i1cX+kHxNkUPpeGeUu+BLbRLwMGtYHCD7y0ogaWiaqU+LG0Fp+DKPDRSixFzeq5Oq78/Iu0ilCV3kRv0RDCqZaoGomazuqpSjiCHUAPxhkbr0qvYaNMVMVvKwabFzoTkDLsQHJrahUl7mzAR+47Qlja5G+8m8Zq6jq1qwEk4pdwCHVfPjTRVy6tg6FQOokfOa29msKcYrpp747KOr6LoeHiwRVYV1YrsRl5BTeJRECZW1cWmxHrk2CmoVvThlqW12yTGCwGqBKu6ccl6HEQJotqaiWc1FarRLaAI813PPehEiRd+NKax+gw7GNOb8GNNooo5Uq2kVbWPhvaNmiD02nmUqpMl7NzDNqVYOhwwk4iqYsSgHtkPtZgDwgkTgAXDYGKnrMpSaG2PT+idzFhiBOTaRXz1cdoZgebqtRZOnC6YdYO7YUKlWcTZd/JaBsdsEHnJC7WjF98p4cEWku9RYmf1W0GWY566ze2RaGcylRmUB9tPjGQqhyGpI4uli/h0sEKfJmjgFjrbPm1ehoqS2A4R48DHHV3z2E3SKcX92q9Xf3IUVwdOiAsGKw5OFtey0OO974UmOMCdm1GJUhWiy2i3SnDXAveqjlpowKaxRqOTTk4MB17BsKY94/Y1M8V7OVxP9yg/E/VAjhaoc8jJHgptumz9j2xx7OjlXBNihpze67kor2EXFR4VDMj1VJGJN9mUCBteYmdt4jsnfI6xiZY31Z7QXKRQqn/BWJmSaGuCRDrvTurq09JSASvmXMi+mcE20vMZsYHbFehRfOkGdX9HSUuur6t1nvjQeBjlVlyOgntLADrSEL29bZM4cLRQ8LAg6mAMUC2H4S5iD8uzhiTVMdyStZPJz1cBRsV7IVT5cEcxwEe1V4IEac6ggBwj0Jg4Hx0PhHlzWUSI94OwWlQK88FMHHgXO/XVfUBpZYYeg6U8RyonChyJodT2U2JcrMFZ1qUN3N5iUQpFGzi68HeiIOJehNoYFxRss7vt+ksK25WdCIk9CthMCegJLNsDki/I407KW0DdbT6PjW4G1zZHLaphkdOswA2VTzJTGw6o6IPNtlqB+UoiTdlBdRuG6UmKwmFQaXmy4U6k4Tayk0ZrmyiQo2qLxQAKSmq6HddjYDxrdVwR2r5PuojPC+1xxkORoBVI8tPqguF4YKaQYhNnbI9ShvXIuXufgZ2gyDfriSDP6Hk+o7gpHJcsi/EmEEzDXE4wohd4ttCqIRKTBLvmVRlL3NJqI63nazlPE+1O5GLu8ttWlNqAp2A1fhCCMBcwgyDiFlXniDpjijlw3RKWN/EYGjHYzuRCyeYfSJCxuyWnkwnr4NY2CmSPeOfa77cMvczXGs5iDCrNwZ4EI9y9iaOuQRhD7EjuIKhtdpdRmEvgw9O3gbXJszACPoAbK89lsXurhsvU5Fbkg/3R2r8nA8TGyl24tHQsr64Hf4g6rX+giPnY+MnT4iUC41xLo+7QE825dE8qM69PbHO57RPsdg9SE1Y4pDtKpb9Rry7QZVNGCeKgIPLeAgsOdXhgmjxz3jqzHxdKqfiFrSWYV64ghopxGmLfYZJ/vQbl+TBogCN2VdyQZT/Ph1C3G0CunEFEiBtTNxcKJbAv95xnMJgAfBzyTvYTv9mCFnVeoWNv5CvmAVsbyLmfQc+6gLtFrRntXfjGkxd6Ii/4FgNWu/YdndgTku0P0RasoGG47+EiFz7njLEe2OIJUiojc7LfSl9IoPLP7jajlVlo9PUBzbRG6g82jolkQBLIo3MRE90DWI8LdL9M/N45Catsqlvug5LS1aozB2OIs8YoEnDqTQFZpN/yhkNlOY50ZOCHsF8c7DjIe+MmEqTpO5NO41dQS8QM0EDUB0QeAAC7XJici4iGkSW4WqyNdVDJ4miBwREUfgVrXaZDTU16RunE3UNel9IdrB2PPsM1BeV6sA9moa4Ifrh64uRVATzJ0Wnjr/m+2LLlxQoqIEWd+N2jKJuSjhh2VeRQ55/WNazvJG+5p6v15xu9jmhXgRoxc3CynFUY5vx8RQVgpxjeqq6YcjSxwu6yoOckzW7n+EQKvoBHg2/uKeqwwnUBh/EMsE8ifblysc+DdZBsbqQt23Z6dxREOxRo2JinQmV3phOwiKybXhDRdlbjpKEcKK7cO5WZ0vG8LmoMYH7mFFj9aLq6W5HilFJh59DVjcjqFovbshITNdntSvHwoM9IhTlz8GiGWUaK7HEjYKEMuhUB+L3W3auTQoHRwcyRRiM0oDCXWP2xLh+qdNwK4euUEfGqhcHawVW9QIMnGgeoEQFp2etboBElKciGNgdqfRmAK7ClzkDUPidaGA/gATrR6CbVmiwMe1JuZzcnorYGiwpWVmRADuT6ixSPJzOz7GqKxaJA0oYmEZt2BTBe0V6gc29aKN2LVylKPD24MDzhOlpzUehMvxdKAzXrhO2UbaecZe0RoRQ1BEwje1Cy60S1E+4gZDuRDxgxjIu5gvYbtz+WI2fcdgJUawOhYCp2AEzl8SibnA879arO1ks1ANTDVj7yDJQhlk+Wq3UGjeRd2XhwFPz53AAbmSjsy+ywDbid+cq1K6S1vNUPeZztw6Fgj9jMbhjWGziUPKKrcdA9nJQXT3dSGiXPKp9HRs0sm701OC60c0vD2MuRyqVSzQdyBjv/qxCTvpDLqwnDW8XwI7XAw/A0bP3g6NzOKZEo2JCtWx94J1ZU9ScFx5pRn0DGwLKue14ZIZ1uoRinB5uUciA70HOJJJiLCgts9bKhapIDS9HAo9T7oc+2upCRddOn3SJnjhlaLrVV4gIG/xEtqJthBqUTsibZEIF3mjzkU9srC+dTp4M5ZEeGiyDVlh+uCSpDC6inbmcnEMyAJ+0eRIBs0bTJiqtKqIblSDsBuTuaVJZEuy2PiS6mVQOAepkMcByvNJJhGSjdzICJYYYMqGJpyyJDC6AcF0jBoRVf3+79eCIdKRoMdNwQYSP1coq8NNTCtRWNBLVfR8gldDclpnOE3+H5SrEXJ0Jbh9+2LVeLZ/IkpflyVsXoltO9Za9xORhsAJh4JTnB9pUW+6aJIFc+5QJ+N+nnykxb3hP/XFcPxuIsHRTeqF3HFcBZEzgv+IHaKRiYijpoqhWLHCfHonygrZADANYzBzZzmWaORmse2Nlc8bNjWn0xbmXBbUqfLhEWY3PDTSPEsQsKEi1tTCjsEYwwV9xJzAZRzJxTxBZyUCMmyxRxFuAbxrs0V5YNHY49V2Juxscxdr4cidPJ0pR7TYeVlROZpWcnsiNjkFF5dV723jVY0BvBJHlbD7TEcvfrkgYniOzo5NxDRqUaD2w2oanqZAJdbT0GdG9rrOvDh/INRcu+9Uiji6fpZKyUFbZffQx74jDgRe+VSV3BHHt1u5Ie5wZ+xNcZyAZN724NjRBpliS5PuMZW4cLe2yoZiHHZPaEJfBYtNzi2kUKQO+7uM25XUae40x4znNJHbjw6kuAyYngFwpfbhQebs43L4tEPzC9e7LVx9iSt3ZyF0GcHd2HMY02Sa5WLU/rhz5aJlvRor2IeVprRyoZbdF+vkmJJrWntpJz5NRagyzaxIpMmUeMPSB+iEYQsksrq00WsHN33iuFp8CS7Qk6WJwGgrjejh51JDWAUJ8HtOqUOKUU8oh39LGL6dsY4TAkDZhHKVYhNTcLnHDCkIdyY0agMmQCl6C71PTm7bxpR9KnM6jswgUgUICTwVqclRGFbOwScaJdMcKNEMIzC7na6WFT85meKcArgBr3HBeZttm57pawduoDhNKA4hwemx7p5uPDJY8jtyPZpDBp9a4w9qnW9UmyAh0HdSs5ZquTSXlceMyVolJjXsAGKlBGEKzHjVvwwoarmE7cL6FIzrZRmfi0hRemCmyQANoXNqxjIKRDEO7RKHJ4FOv6TRSP46KWuy0plISR1VkqQ4MSHT1LbHj1phMY7kLQgftW3HJSsktQhDLIF+IazumCBrFR8fqcMM7clfPfrxlu3O+YdPrDCrj/h+uv5eozkHaI7UKJcNWiP+Wx0Y//oEzc5eA8afiXQ5rB0cjh9HyuMvxyvLbb/HLsu7NvjzlxeiqyZaxmzVYgZ5me0JfTxJvVVlX3xg3C4DctuG0bduDcZzNs2uezmNcP0FeHOvNhmYdds44f9PJwHCX/iz///OVgb/jlwHBoTIIufr4HKrmCW3GY3OOXPpL0p6c+X99/afvXg7q/cg731w4df+s4agRqwyb55WD6p+H1uw3Qb/vQfOV5Ly//9QPD/0Trz7/umqS4i0kXNi44FttfW/hltt7u/rc6Nj1I3HXO86chaftfOryxOwiO0zjx5KNB9IT5iP/kYRS9XkJkRFCu59Pk6zk5qscvJuEv9SoI2+RePCXFdoY3GGLOL/OqLNYBAephhcIU5OHRE05g4Eh6mHyicQp/Cr0o8Ag8wrzQ/7ZD085tF+ZPOTgufZ0moAYwHKVxmnzCEAp9wiICf6IgGnpyQ4KM6AANaYz+roOyqgiYpuHoKUJ9Yh0UbB0UD4KeIBjFCcj1aAScjPP1QfmlD+M4/ndE/1s2mzWhaYCvIfASCPK0KoWndl7lenoq1geiHTj//Vu+4W9GfJXEKn5qu6b3u74Blh7ysDB0feoJwkL8CQvXCacIepXFACcij8ZIj/T+py/45hIV+vt9/ZRz/fACpOSv9QSBIEBfQIDR2Lrhx6s2WfuQud5LW3+mE8Z+cTxrYW3eO3f7DkOiBKRwfQ8lAeG+j6Ow/xQgGPyE0T66rocAzA5KkqSPYwgRvX4P4HF+VWF++vQrmv3TR/m0D7Psl8vSS9cV8Xz58ncEeh647WVflPw6TH6SAev1yZysj0v8Jz92i+LThCXBW/YRoV+Zuq0PoN05e2kxDO6h9nJZNl1c3svCzQ6/3mUaMJSbUYV+a7TTsOtmLVnAT92+K9dbv7YglmX1G+MOHvQ3Tfvr1blHaZUnI4tq6yYdd5hnncCxu98MATRhtorHEH7WwpfW/OWnSpkA9+4TcoBR6De4AfvMFnduc1/N4fOPfiNZr3rxV1DCW5P+A6CEt7v/gRJ+AJTws2j3/zUWIHAYJwkEeiIxbx1cEgrXrtDBk0d7EU54hO9h/1sswGRu8ZDWiQbnfXxFlbyhWr/VXH05KgEc+RBBhE8oRQCJD6EnCoWwJ8IPQTk4DHJfPK93Y4Nd/0U3/mp5f3fkftfstrG7tdfn2W5rD2WA95v4q+UEPVDKNumetalXdl2Zr1/Yusa4/uO+meT9OurN1hoabX9etbHLVjWzftYBI/zaZpd9ty7zcB22Ynv3zaa/F+/7k6/9pav9yaq+drWJzwzY/8HVfkPy370F/ddq2C8tl0v5OBQEwZPr0tQTRvnhkwu7/hPu0ni0uppYSKLvzXr+uz3Q72ByvhX2+66Gh8jWj5kgWd+GuHebin6+5X26sV9Fbl/meQJ+q4YVsBclUH7P31uf4n3+220wXrX4Vevmr+36L+2+NnHN1eSrIBt3Z9fq9DP28AsQiPOGiauAk7OpRpxZ/13V2f75P3z9dA/u/BfBv3Lza/fIL2/CX35t/R/8tSd8fvNr98gvb8Jffg1cfer1b29+7R6Jf9njz38Nf+XX8Ge/Xv/9HcsdrVL4Cges/3BgPkBJ4CAJf/MZxpLrh68+Y5NmbejZ3hTAcQftJVn26jfrH4yh1/urRikf4dcQR+C28S/kwB9CmK/jE/flyl97BRbUa8AC3vCFY4CRT9cvQgYe6W6e1HoRJRPoB/MJWeXTpgz/644tAv33k4I9+qBDzHr5/LfPvvZK+t8PLPqcWniFi6iv4CLq2+Gi3130H7joXeGin5FW+Nh8+Mmh3+8BmH9gD+INPfcBQ9+EoUpSgQW+Tju0W0175PpgbRFuDux64bXVb/HnH2LSP36itlpJiOn9R9j9n8Fui/4W5Jp7mxITrI4RTOQaU4PbEVRS/wC5PwrIJaAdipJ/DeQiJAnDxL8I5LboOwa3nzhABP/eYPd3F/8H2P2hwG4YYpiH0u6Tj3neEwatQMWjCfhphXNU6MM0Gn4Smp8I7P7QUPBrIOI3Zv4fgIRvaIMPSPiHzCTTJyBM/xtwkd5zS69RWnoj9elxKunWOcRNf85IYbA+UNqPg9I+qMg/SUU+C/+7A2voJwP6/cDZ7675D3D2Q4GzDybyg4l8b/DzFWD5B4DmG5rtXw00v4oI/0b8M/raln3T+Od1CpvZfrm/XdzAxYqgXi7Z6fWH7Pz6SlnfYzWGYPC2m/9IMPWTg8C6cduxFcnJT5iU0Mn/G5+/9hyo/Ici+n1jrv8iaHhLIN4HbvijkMw3Juj9hGR+gIaPqOgfNyr6b6QnoN84ivBb2ec/0ibfAWp8ZYJh0gtXg/XkR+u6xVwIfnJJPHii4DBAV6iL4CjyztmsX/nJb0BoVb809hq7/K6i/+C0Pjitn4rT+kX+3x2thRDo96a13sB37wGefoCYnxihftBaPz+tpfwGbHxnZus7OK//bmbrV0X9rZmtKensV39/xWutV7/SWuBifnXxOan14zNkb0SGvmbI3hD1H5Ehg94PBPkjn/Y7hO5+MGQfDNkHQ/bXGLJvHGH3vRiyf4Rvf9cM2TcDK/D/Cqz8I7jgjfDAnxgXwD8OLvgO8ZsfuOADF3zggr+GC+DvM1cfuOCHwQXYT4ULYJ8nDdGQj6tiZpzDGF8y8/AlLuA64c7RZcuk6nLk4kiYdgfux8UF2I+DC96YoA9c8IELPnDBP4MLfncj6H3jgjf0+L8UF/yJBH73Hl7cx6bTv1lkTbs2mj83+hpz/K62/wir+UHCanYMQu/+WlgNDBPsOnn/nrCaX4X/3cXU4NB3TxV7A+G9B4D6AWN+Yoz6EVPzc8fUfB27/JXefKPYmu/gxv67Y2uQn4qWeiPy/zUt9YZU/Yi01Cd5eAdW/12mZnzQUh+01Act9YdW4P2Z1HdJcr9rWuqvkEa/IwvkB2/0wRt98EbfqcQQ9tn5q3/yUBj02/FIb6mB9wEqP6DHT4wrP6ikn5tKelu9vDfE+c+Dtb9D4rx9QM1HgtTfNPSvKaNM3z0M7UT2DB/pTVzmYxNW/wRl9PLTz85jJcjP6krTnxnnZ6n94kDWXdO486uvvcD7338ORH71Odzf+/76l+ce/LqGvuHpsOiPcobL291/bzryA8Z80GM/Lj32R5zSG4r+X8op/XGokxrek7bbtOEFLKH/VZjT787NB131QVd90FX/e7qKhJDfwjn8u4c9vaGf3wPO+6Azfng64wPf/sT49s33+BLH/BUx+0YhT98BgL5ztgyMX1l2r0kBoCWepwQ9/H8= \ No newline at end of file diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/pipeline_constructs/build_pipeline_construct.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/pipeline_constructs/build_pipeline_construct.py new file mode 100644 index 00000000..9da4e612 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/pipeline_constructs/build_pipeline_construct.py @@ -0,0 +1,288 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +from typing import Any + +import aws_cdk +from aws_cdk import Aws +from aws_cdk import aws_cloudwatch as cloudwatch +from aws_cdk import aws_codebuild as codebuild +from aws_cdk import aws_codecommit as codecommit +from aws_cdk import aws_codepipeline as codepipeline +from aws_cdk import aws_codepipeline_actions as codepipeline_actions +from aws_cdk import aws_iam as iam +from aws_cdk import aws_s3 as s3 +from aws_cdk import aws_s3_assets as s3_assets +from constructs import Construct + + +class BuildPipelineConstruct(Construct): + def __init__( + self, + scope: Construct, + construct_id: str, + project_name: str, + project_id: str, + s3_artifact: s3.IBucket, + repo_asset: s3_assets.Asset, + model_package_group_name: str, + hf_access_token_secret: str, + hf_model_id: str, + **kwargs: Any, + ) -> None: + super().__init__(scope, construct_id, **kwargs) + + # Define resource names + codepipeline_name = f"{project_name}-{construct_id}" + + sagemaker_pipeline_name = f"{project_name}-{project_id}" + sagemaker_pipeline_description = f"{project_name} Model Build Pipeline" + + # Create source repo from seed bucket/key + build_app_repository = codecommit.Repository( + self, + "Build App Code Repo", + repository_name=f"{project_name}-{construct_id}", + code=codecommit.Code.from_asset( + asset=repo_asset, + branch="main", + ), + ) + aws_cdk.Tags.of(build_app_repository).add("sagemaker:project-id", project_id) + aws_cdk.Tags.of(build_app_repository).add("sagemaker:project-name", project_name) + + sagemaker_seedcode_bucket = s3.Bucket.from_bucket_name( + self, "SageMaker Seedcode Bucket", f"sagemaker-{Aws.REGION}-{Aws.ACCOUNT_ID}" + ) + + codebuild_role = iam.Role( + self, + "CodeBuild Role", + assumed_by=iam.ServicePrincipal("codebuild.amazonaws.com"), + path="/service-role/", + ) + + sagemaker_execution_role = iam.Role( + self, + "SageMaker Execution Role", + assumed_by=iam.ServicePrincipal("sagemaker.amazonaws.com"), + path="/service-role/", + ) + + # Create a policy statement for SageMaker pull + sagemaker_policy = iam.Policy( + self, + "SageMaker Policy", + document=iam.PolicyDocument( + statements=[ + iam.PolicyStatement( + actions=[ + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:PutLogEvents", + ], + resources=["*"], + ), + iam.PolicyStatement( + actions=[ + "ecr:BatchCheckLayerAvailability", + "ecr:BatchGetImage", + "ecr:Describe*", + "ecr:GetAuthorizationToken", + "ecr:GetDownloadUrlForLayer", + ], + resources=["*"], + ), + iam.PolicyStatement( + actions=[ + "kms:Encrypt", + "kms:ReEncrypt*", + "kms:GenerateDataKey*", + "kms:Decrypt", + "kms:DescribeKey", + ], + effect=iam.Effect.ALLOW, + resources=[f"arn:{Aws.PARTITION}:kms:{Aws.REGION}:{Aws.ACCOUNT_ID}:key/*"], + ), + ] + ), + ) + + cloudwatch.Metric.grant_put_metric_data(sagemaker_policy) + sagemaker_execution_role.grant_pass_role(sagemaker_policy) # type: ignore[arg-type] + s3_artifact.grant_read_write(sagemaker_policy) + sagemaker_seedcode_bucket.grant_read_write(sagemaker_policy) + + # Attach the policy + sagemaker_policy.attach_to_role(sagemaker_execution_role) + sagemaker_policy.attach_to_role(codebuild_role) + + # Grant extra permissions for the SageMaker role + sagemaker_execution_role.add_to_policy( + iam.PolicyStatement( + actions=[ + "sagemaker:CreateModel", + "sagemaker:DeleteModel", + "sagemaker:DescribeModel", + "sagemaker:AddTags", + "sagemaker:DeleteTags", + "sagemaker:ListTags", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:model/*", + ], + ), + ) + sagemaker_execution_role.add_to_policy( + iam.PolicyStatement( + actions=[ + "sagemaker:CreateModelPackageGroup", + "sagemaker:DeleteModelPackageGroup", + "sagemaker:DescribeModelPackageGroup", + "sagemaker:AddTags", + "sagemaker:DeleteTags", + "sagemaker:ListTags", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:model-package-group/{model_package_group_name}" + ], + ), + ) + sagemaker_execution_role.add_to_policy( + iam.PolicyStatement( + actions=[ + "sagemaker:CreateModelPackage", + "sagemaker:DeleteModelPackage", + "sagemaker:UpdateModelPackage", + "sagemaker:DescribeModelPackage", + "sagemaker:ListModelPackages", + "sagemaker:AddTags", + "sagemaker:DeleteTags", + "sagemaker:ListTags", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:model-package/{model_package_group_name}/*" + ], + ), + ) + + # Grant extra permissions for the CodeBuild role + codebuild_role.add_to_policy( + iam.PolicyStatement( + actions=[ + "sagemaker:DescribeModelPackage", + "sagemaker:ListModelPackages", + "sagemaker:UpdateModelPackage", + "sagemaker:AddTags", + "sagemaker:DeleteTags", + "sagemaker:ListTags", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:model-package/{model_package_group_name}/*" + ], + ), + ) + codebuild_role.add_to_policy( + iam.PolicyStatement( + actions=[ + "sagemaker:CreatePipeline", + "sagemaker:UpdatePipeline", + "sagemaker:DeletePipeline", + "sagemaker:StartPipelineExecution", + "sagemaker:StopPipelineExecution", + "sagemaker:DescribePipelineExecution", + "sagemaker:ListPipelineExecutionSteps", + "sagemaker:AddTags", + "sagemaker:DeleteTags", + "sagemaker:ListTags", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:pipeline/{sagemaker_pipeline_name}", + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:pipeline/{sagemaker_pipeline_name}/execution/*", + ], + ), + ) + codebuild_role.add_to_policy( + iam.PolicyStatement( + actions=[ + "s3:CreateBucket", + ], + resources=[sagemaker_seedcode_bucket.bucket_arn], + ) + ) + + # Create the CodeBuild project + sm_pipeline_build = codebuild.PipelineProject( + self, + "SM Pipeline Build", + project_name=f"{project_name}-{construct_id}", + role=codebuild_role, + build_spec=codebuild.BuildSpec.from_source_filename("buildspec.yml"), + environment=codebuild.BuildEnvironment( + build_image=codebuild.LinuxBuildImage.STANDARD_5_0, + environment_variables={ + "SAGEMAKER_PROJECT_NAME": codebuild.BuildEnvironmentVariable(value=project_name), + "SAGEMAKER_PROJECT_ID": codebuild.BuildEnvironmentVariable(value=project_id), + "MODEL_PACKAGE_GROUP_NAME": codebuild.BuildEnvironmentVariable(value=model_package_group_name), + "AWS_REGION": codebuild.BuildEnvironmentVariable(value=Aws.REGION), + "SAGEMAKER_PIPELINE_NAME": codebuild.BuildEnvironmentVariable( + value=sagemaker_pipeline_name, + ), + "SAGEMAKER_PIPELINE_DESCRIPTION": codebuild.BuildEnvironmentVariable( + value=sagemaker_pipeline_description, + ), + "SAGEMAKER_PIPELINE_ROLE_ARN": codebuild.BuildEnvironmentVariable( + value=sagemaker_execution_role.role_arn, + ), + "ARTIFACT_BUCKET": codebuild.BuildEnvironmentVariable(value=s3_artifact.bucket_name), + "ARTIFACT_BUCKET_KMS_ID": codebuild.BuildEnvironmentVariable( + value=s3_artifact.encryption_key.key_id # type: ignore[union-attr] + ), + "HUGGING_FACE_ACCESS_TOKEN_SECRET": codebuild.BuildEnvironmentVariable( + value=hf_access_token_secret + ), # pass secret + "HUGGING_FACE_MODEL_ID": codebuild.BuildEnvironmentVariable(value=hf_model_id), + }, + ), + ) + + source_artifact = codepipeline.Artifact(artifact_name="GitSource") + + build_pipeline = codepipeline.Pipeline( + self, "Pipeline", pipeline_name=codepipeline_name, artifact_bucket=s3_artifact + ) + + # add a source stage + source_stage = build_pipeline.add_stage(stage_name="Source") + source_stage.add_action( + codepipeline_actions.CodeCommitSourceAction( + action_name="Source", + output=source_artifact, + repository=build_app_repository, + branch="main", + ) + ) + + # add a build stage + build_stage = build_pipeline.add_stage(stage_name="Build") + build_stage.add_action( + codepipeline_actions.CodeBuildAction( + action_name="SMPipeline", + input=source_artifact, + project=sm_pipeline_build, + ) + ) diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/product_stack.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/product_stack.py new file mode 100644 index 00000000..3c148579 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/product_stack.py @@ -0,0 +1,216 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +from typing import Any + +import aws_cdk +import aws_cdk.aws_servicecatalog as servicecatalog +from aws_cdk import Aws, Tags +from aws_cdk import aws_iam as iam +from aws_cdk import aws_kms as kms +from aws_cdk import aws_s3 as s3 +from aws_cdk import aws_s3_assets as s3_assets +from aws_cdk import aws_sagemaker as sagemaker +from constructs import Construct + +from templates.hf_import_models.pipeline_constructs.build_pipeline_construct import BuildPipelineConstruct + + +class Product(servicecatalog.ProductStack): + DESCRIPTION: str = "Enables the import of Hugging Face models" + TEMPLATE_NAME: str = "Hugging Face Model Import" + + def __init__( + self, + scope: Construct, + construct_id: str, + build_app_asset: s3_assets.Asset, + deploy_app_asset: s3_assets.Asset, + **kwargs: Any, + ) -> None: + super().__init__(scope, construct_id) + + # Define required parmeters + project_name = aws_cdk.CfnParameter( + self, + "SageMakerProjectName", + type="String", + description="The name of the SageMaker project.", + min_length=1, + max_length=32, + ).value_as_string + + project_id = aws_cdk.CfnParameter( + self, + "SageMakerProjectId", + type="String", + min_length=1, + max_length=16, + description="Service generated Id of the project.", + ).value_as_string + + staging_account = aws_cdk.CfnParameter( + self, + "StgAccountId", + type="String", + min_length=1, + max_length=16, + description="Staging account id.", + ).value_as_string + + prod_account = aws_cdk.CfnParameter( + self, + "ProdAccountId", + type="String", + min_length=1, + max_length=16, + description="Prod account id.", + ).value_as_string + + hf_access_token_secret = aws_cdk.CfnParameter( + self, + "HFAccessTokenSecret", + type="String", + min_length=1, + description="AWS Secret Of Hugging Face Access Token", + ).value_as_string + + hf_model_id = aws_cdk.CfnParameter( + self, + "HFModelID", + type="String", + min_length=1, + description="Model ID from hf.co/models", + ).value_as_string + + Tags.of(self).add("sagemaker:project-id", project_id) + Tags.of(self).add("sagemaker:project-name", project_name) + + # create kms key to be used by the assets bucket + kms_key_artifact = kms.Key( + self, + "Artifacts Bucket KMS Key", + description="key used for encryption of data in Amazon S3", + enable_key_rotation=True, + policy=iam.PolicyDocument( + statements=[ + iam.PolicyStatement( + actions=["kms:*"], + effect=iam.Effect.ALLOW, + resources=["*"], + principals=[iam.AccountRootPrincipal()], + ) + ] + ), + ) + + # allow cross account access to the kms key + kms_key_artifact.add_to_resource_policy( + iam.PolicyStatement( + actions=[ + "kms:Encrypt", + "kms:Decrypt", + "kms:ReEncrypt*", + "kms:GenerateDataKey*", + "kms:DescribeKey", + ], + resources=[ + "*", + ], + principals=[ + iam.AccountPrincipal(staging_account), + iam.AccountPrincipal(prod_account), + ], + ) + ) + + s3_artifact = s3.Bucket( + self, + "S3 Artifact", + bucket_name=f"mlops-{project_name}-{Aws.ACCOUNT_ID}", # Bucket name has a limit of 63 characters + encryption_key=kms_key_artifact, + versioned=True, + removal_policy=aws_cdk.RemovalPolicy.DESTROY, + enforce_ssl=True, # Blocks insecure requests to the bucket + ) + + # DEV account access to objects in the bucket + s3_artifact.grant_read_write(iam.AccountRootPrincipal()) + + # PROD account access to objects in the bucket + s3_artifact.grant_read_write(iam.AccountPrincipal(staging_account)) + s3_artifact.grant_read_write(iam.AccountPrincipal(prod_account)) + + # cross account model registry resource policy + model_package_group_name = f"{project_name}-{project_id}" + model_package_group_policy = iam.PolicyDocument( + statements=[ + iam.PolicyStatement( + sid="ModelPackageGroup", + actions=[ + "sagemaker:DescribeModelPackageGroup", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:model-package-group/{model_package_group_name}" + ], + principals=[ + iam.AccountPrincipal(staging_account), + iam.AccountPrincipal(prod_account), + ], + ), + iam.PolicyStatement( + sid="ModelPackage", + actions=[ + "sagemaker:DescribeModelPackage", + "sagemaker:ListModelPackages", + "sagemaker:UpdateModelPackage", + "sagemaker:CreateModel", + ], + resources=[ + f"arn:{Aws.PARTITION}:sagemaker:{Aws.REGION}:{Aws.ACCOUNT_ID}:model-package/{model_package_group_name}/*" + ], + principals=[ + iam.AccountPrincipal(staging_account), + iam.AccountPrincipal(prod_account), + ], + ), + ] + ).to_json() + + sagemaker.CfnModelPackageGroup( + self, + "Model Package Group", + model_package_group_name=model_package_group_name, + model_package_group_description=f"Model Package Group for {project_name}", + model_package_group_policy=model_package_group_policy, + tags=[ + aws_cdk.CfnTag(key="sagemaker:project-id", value=project_id), + aws_cdk.CfnTag(key="sagemaker:project-name", value=project_name), + ], + ) + + BuildPipelineConstruct( + self, + "build", + project_name=project_name, + project_id=project_id, + s3_artifact=s3_artifact, + repo_asset=build_app_asset, + model_package_group_name=model_package_group_name, + hf_access_token_secret=hf_access_token_secret, + hf_model_id=hf_model_id, + ) diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/README.md b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/README.md new file mode 100644 index 00000000..2e986925 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/README.md @@ -0,0 +1,12 @@ +# SageMaker Build - Train Pipelines + +This folder contains all the SageMaker Pipelines of your project. + +`buildspec.yml` defines how to run a pipeline after each commit to this repository. +`ml_pipelines/` contains the SageMaker pipelines definitions. +The expected output of your main pipeline (here `training/pipeline.py`) is a model registered to SageMaker Model Registry. + +`tests/` contains the unittests for your `source_scripts/` + +`notebooks/` contains experimentation notebooks. + diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/buildspec.yml b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/buildspec.yml new file mode 100644 index 00000000..5603158e --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/buildspec.yml @@ -0,0 +1,18 @@ +version: 0.2 + +phases: + install: + runtime-versions: + python: 3.8 + commands: + - pip install --upgrade --force-reinstall . "awscli>1.20.30" + + build: + commands: + - export PYTHONUNBUFFERED=TRUE + - | + run-pipeline --module-name ml_pipelines.training.pipeline \ + --role-arn $SAGEMAKER_PIPELINE_ROLE_ARN \ + --tags "[{\"Key\":\"sagemaker:project-name\", \"Value\":\"${SAGEMAKER_PROJECT_NAME}\"}, {\"Key\":\"sagemaker:project-id\", \"Value\":\"${SAGEMAKER_PROJECT_ID}\"}]" \ + --kwargs "{\"region\":\"${AWS_REGION}\",\"role\":\"${SAGEMAKER_PIPELINE_ROLE_ARN}\",\"default_bucket\":\"${ARTIFACT_BUCKET}\",\"pipeline_name\":\"${SAGEMAKER_PIPELINE_NAME}\",\"model_package_group_name\":\"${MODEL_PACKAGE_GROUP_NAME}\",\"hugging_face_model_id\":\"${HUGGING_FACE_MODEL_ID}\"}" + - echo "Create/Update of the SageMaker Pipeline and execution completed." diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/README.md b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/README.md new file mode 100644 index 00000000..1f7850d8 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/README.md @@ -0,0 +1,8 @@ +# SageMaker Pipelines + +This folder contains SageMaker Pipeline definitions and helper scripts to either simply "get" a SageMaker Pipeline definition (JSON dictionnary) with `get_pipeline_definition.py`, or "run" a SageMaker Pipeline from a SageMaker pipeline definition with `run_pipeline.py`. + +Those files are generic and can be reused to call any SageMaker Pipeline. + +Each SageMaker Pipeline definition should be be treated as a module inside its own folder, for example here the +"training" pipeline, contained inside `training/`. diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/__init__.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/__init__.py new file mode 100644 index 00000000..ff79f21c --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/__init__.py @@ -0,0 +1,30 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +# © 2021 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. This +# AWS Content is provided subject to the terms of the AWS Customer Agreement +# available at http://aws.amazon.com/agreement or other written agreement between +# Customer and either Amazon Web Services, Inc. or Amazon Web Services EMEA SARL +# or both. +# +# Any code, applications, scripts, templates, proofs of concept, documentation +# and other items provided by AWS under this SOW are "AWS Content," as defined +# in the Agreement, and are provided for illustration purposes only. All such +# AWS Content is provided solely at the option of AWS, and is subject to the +# terms of the Addendum and the Agreement. Customer is solely responsible for +# using, deploying, testing, and supporting any code and applications provided +# by AWS under this SOW. diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/__version__.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/__version__.py new file mode 100644 index 00000000..660d19ee --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/__version__.py @@ -0,0 +1,26 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +"""Metadata for the ml pipelines package.""" + +__title__ = "ml_pipelines" +__description__ = "ml pipelines - template package" +__version__ = "0.0.1" +__author__ = "" +__author_email__ = "" +__license__ = "Apache 2.0" +__url__ = "" diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/_utils.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/_utils.py new file mode 100644 index 00000000..12a5b559 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/_utils.py @@ -0,0 +1,93 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +# © 2021 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. This +# AWS Content is provided subject to the terms of the AWS Customer Agreement +# available at http://aws.amazon.com/agreement or other written agreement between +# Customer and either Amazon Web Services, Inc. or Amazon Web Services EMEA SARL +# or both. +# +# Any code, applications, scripts, templates, proofs of concept, documentation +# and other items provided by AWS under this SOW are "AWS Content," as defined +# in the Agreement, and are provided for illustration purposes only. All such +# AWS Content is provided solely at the option of AWS, and is subject to the +# terms of the Addendum and the Agreement. Customer is solely responsible for +# using, deploying, testing, and supporting any code and applications provided +# by AWS under this SOW. + +# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"). You +# may not use this file except in compliance with the License. A copy of +# the License is located at +# +# http://aws.amazon.com/apache2.0/ +# +# or in the "license" file accompanying this file. This file is +# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF +# ANY KIND, either express or implied. See the License for the specific +# language governing permissions and limitations under the License. +"""Provides utilities for SageMaker Pipeline CLI.""" + +from __future__ import absolute_import + +import ast +from typing import Any, Dict, Optional + + +def get_pipeline_driver(module_name: str, passed_args: Optional[str] = None) -> Any: + """Gets the driver for generating your pipeline definition. + + Pipeline modules must define a get_pipeline() module-level method. + + Args: + module_name: The module name of your pipeline. + passed_args: Optional passed arguments that your pipeline may be templated by. + + Returns: + The SageMaker Workflow pipeline. + """ + _imports = __import__(module_name, fromlist=["get_pipeline"]) + kwargs = convert_struct(passed_args) + return _imports.get_pipeline(**kwargs) + + +def convert_struct(str_struct: Optional[str] = None) -> Any: + """convert the string argument to it's proper type + + Args: + str_struct (str, optional): string to be evaluated. Defaults to None. + + Returns: + string struct as it's actuat evaluated type + """ + return ast.literal_eval(str_struct) if str_struct else {} + + +def get_pipeline_custom_tags(module_name: str, args: Optional[str], tags: Dict[str, Any]) -> Any: + """Gets the custom tags for pipeline + + Returns: + Custom tags to be added to the pipeline + """ + try: + _imports = __import__(module_name, fromlist=["get_pipeline_custom_tags"]) + kwargs = convert_struct(args) + return _imports.get_pipeline_custom_tags(tags, kwargs["region"], kwargs["sagemaker_project_arn"]) + except Exception as e: + print(f"Error getting project tags: {e}") + return tags diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/get_pipeline_definition.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/get_pipeline_definition.py new file mode 100644 index 00000000..53535920 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/get_pipeline_definition.py @@ -0,0 +1,74 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + +"""A CLI to get pipeline definitions from pipeline modules.""" + +from __future__ import absolute_import + +import argparse +import sys + +from ml_pipelines._utils import get_pipeline_driver + + +def main() -> None: # pragma: no cover + """The main harness that gets the pipeline definition JSON. + + Prints the json to stdout or saves to file. + """ + parser = argparse.ArgumentParser("Gets the pipeline definition for the pipeline script.") + + parser.add_argument( + "-n", + "--module-name", + dest="module_name", + type=str, + help="The module name of the pipeline to import.", + ) + parser.add_argument( + "-f", + "--file-name", + dest="file_name", + type=str, + default=None, + help="The file to output the pipeline definition json to.", + ) + parser.add_argument( + "-kwargs", + "--kwargs", + dest="kwargs", + default=None, + help="Dict string of keyword arguments for the pipeline generation (if supported)", + ) + args = parser.parse_args() + + if args.module_name is None: + parser.print_help() + sys.exit(2) + + pipeline = get_pipeline_driver(args.module_name, args.kwargs) + content = pipeline.definition() + if args.file_name: + with open(args.file_name, "w") as f: + f.write(content) + else: + print(content) + + +if __name__ == "__main__": + main() diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/run_pipeline.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/run_pipeline.py new file mode 100644 index 00000000..f0e12338 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/run_pipeline.py @@ -0,0 +1,106 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +"""A CLI to create or update and run pipelines.""" + +from __future__ import absolute_import + +import argparse +import json +import sys + +from ml_pipelines._utils import convert_struct, get_pipeline_custom_tags, get_pipeline_driver + + +def main() -> None: # pragma: no cover + """The main harness that creates or updates and runs the pipeline. + + Creates or updates the pipeline and runs it. + """ + parser = argparse.ArgumentParser("Creates or updates and runs the pipeline for the pipeline script.") + + parser.add_argument( + "-n", + "--module-name", + dest="module_name", + type=str, + help="The module name of the pipeline to import.", + ) + parser.add_argument( + "-kwargs", + "--kwargs", + dest="kwargs", + default=None, + help="Dict string of keyword arguments for the pipeline generation (if supported)", + ) + parser.add_argument( + "-role-arn", + "--role-arn", + dest="role_arn", + type=str, + help="The role arn for the pipeline service execution role.", + ) + parser.add_argument( + "-description", + "--description", + dest="description", + type=str, + default=None, + help="The description of the pipeline.", + ) + parser.add_argument( + "-tags", + "--tags", + dest="tags", + default=None, + help="""List of dict strings of '[{"Key": "string", "Value": "string"}, ..]'""", + ) + args = parser.parse_args() + + if args.module_name is None or args.role_arn is None: + parser.print_help() + sys.exit(2) + tags = convert_struct(args.tags) + + pipeline = get_pipeline_driver(args.module_name, args.kwargs) + print("###### Creating/updating a SageMaker Pipeline with the following definition:") + parsed = json.loads(pipeline.definition()) + print(json.dumps(parsed, indent=2, sort_keys=True)) + + all_tags = get_pipeline_custom_tags(args.module_name, args.kwargs, tags) + + upsert_response = pipeline.upsert(role_arn=args.role_arn, description=args.description, tags=all_tags) + + upsert_response = pipeline.upsert( + role_arn=args.role_arn, description=args.description + ) # , tags=tags) # Removing tag momentaneously + print("\n###### Created/Updated SageMaker Pipeline: Response received:") + print(upsert_response) + + execution = pipeline.start() + print(f"\n###### Execution started with PipelineExecutionArn: {execution.arn}") + + # TODO removiong wait time as training can take some time + print("Waiting for the execution to finish...") + execution.wait() + print("\n#####Execution completed. Execution step details:") + + print(execution.list_steps()) + + +if __name__ == "__main__": + main() diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/README.md b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/README.md new file mode 100644 index 00000000..11c532c6 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/README.md @@ -0,0 +1,15 @@ +# Deploying HuggingFace LLM on SageMaker Pipeline. + +This SageMaker Pipeline definition creates a workflow that will: + +Retrieve the Docker image URI for the HuggingFace Language Model (LLM). + +Create a HuggingFaceModel instance with the specified role, image URI, and environment variables (model ID, GPU count, input/output lengths, batch processing limits, and access token). + +Register the HuggingFaceModel for deployment through the RegisterModel step. + +Configure the content types, response types, and instance types for inference. + +Specify the model package group name and set the initial approval status to "PendingManualApproval". + +Create the SageMaker Pipeline instance with the RegisterModel step and pipeline parameters. diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/__init__.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/__init__.py new file mode 100644 index 00000000..ff79f21c --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/__init__.py @@ -0,0 +1,30 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +# © 2021 Amazon Web Services, Inc. or its affiliates. All Rights Reserved. This +# AWS Content is provided subject to the terms of the AWS Customer Agreement +# available at http://aws.amazon.com/agreement or other written agreement between +# Customer and either Amazon Web Services, Inc. or Amazon Web Services EMEA SARL +# or both. +# +# Any code, applications, scripts, templates, proofs of concept, documentation +# and other items provided by AWS under this SOW are "AWS Content," as defined +# in the Agreement, and are provided for illustration purposes only. All such +# AWS Content is provided solely at the option of AWS, and is subject to the +# terms of the Addendum and the Agreement. Customer is solely responsible for +# using, deploying, testing, and supporting any code and applications provided +# by AWS under this SOW. diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/_utils.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/_utils.py new file mode 100644 index 00000000..933302ae --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/_utils.py @@ -0,0 +1,90 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +import logging +from typing import Any, Dict, List + +import sagemaker.session +from botocore.exceptions import ClientError + +logger = logging.getLogger(__name__) + + +def resolve_ecr_uri_from_image_versions( + sagemaker_session: sagemaker.session.Session, image_versions: List[Dict[str, Any]], image_name: str +) -> Any: + """Gets ECR URI from image versions + Args: + sagemaker_session: boto3 session for sagemaker client + image_versions: list of the image versions + image_name: Name of the image + + Returns: + ECR URI of the image version + """ + + # Fetch image details to get the Base Image URI + for image_version in image_versions: + if image_version["ImageVersionStatus"] == "CREATED": + image_arn = image_version["ImageVersionArn"] + version = image_version["Version"] + logger.info(f"Identified the latest image version: {image_arn}") + response = sagemaker_session.sagemaker_client.describe_image_version(ImageName=image_name, Version=version) + return response["ContainerImage"] + return None + + +def resolve_ecr_uri(sagemaker_session: sagemaker.session.Session, image_arn: str) -> Any: + """Gets the ECR URI from the image name + + Args: + sagemaker_session: boto3 session for sagemaker client + image_name: name of the image + + Returns: + ECR URI of the latest image version + """ + + # Fetching image name from image_arn (^arn:aws(-[\w]+)*:sagemaker:.+:[0-9]{12}:image/[a-z0-9]([-.]?[a-z0-9])*$) + image_name = image_arn.partition("image/")[2] + try: + # Fetch the image versions + next_token = "" + while True: + response = sagemaker_session.sagemaker_client.list_image_versions( + ImageName=image_name, MaxResults=100, SortBy="VERSION", SortOrder="DESCENDING", NextToken=next_token + ) + + ecr_uri = resolve_ecr_uri_from_image_versions(sagemaker_session, response["ImageVersions"], image_name) + + if ecr_uri is not None: + return ecr_uri + + if "NextToken" in response: + next_token = response["NextToken"] + else: + break + + # Return error if no versions of the image found + error_message = f"No image version found for image name: {image_name}" + logger.error(error_message) + raise Exception(error_message) + + except (ClientError, sagemaker_session.sagemaker_client.exceptions.ResourceNotFound) as e: + error_message = e.response["Error"]["Message"] + logger.error(error_message) + raise Exception(error_message) diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/pipeline.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/pipeline.py new file mode 100644 index 00000000..25775259 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/ml_pipelines/training/pipeline.py @@ -0,0 +1,119 @@ +import json +import logging +import os +from typing import Any, Optional + +import boto3 +import sagemaker +import sagemaker.session +from botocore.exceptions import ClientError +from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri +from sagemaker.workflow.parameters import ParameterString +from sagemaker.workflow.pipeline import Pipeline +from sagemaker.workflow.step_collections import RegisterModel + +logger = logging.getLogger(__name__) +ACCESS_TOKEN_SECRET = os.environ["HUGGING_FACE_ACCESS_TOKEN_SECRET"] # read token from secret using boto3 +SECRET_REGION = os.environ["AWS_REGION"] + + +def get_acess_token_from_secret(secretid: str, secret_region: str) -> Any: + # Create a Secrets Manager client + session = boto3.session.Session() + client = session.client(service_name="secretsmanager", region_name=secret_region) + + try: + get_secret_value_response = client.get_secret_value(SecretId=secretid) + except ClientError as e: + raise e + + # Get the secret value + secret_value = get_secret_value_response["SecretString"] + return secret_value + + +ACCESS_TOKEN = get_acess_token_from_secret(ACCESS_TOKEN_SECRET, SECRET_REGION) + + +def get_session(region: str, default_bucket: Optional[str]) -> sagemaker.session.Session: + """Gets the sagemaker session based on the region. + + Args: + region: the aws region to start the session + default_bucket: the bucket to use for storing the artifacts + + Returns: + `sagemaker.session.Session instance + """ + + boto_session = boto3.Session(region_name=region) + + sagemaker_client = boto_session.client("sagemaker") + runtime_client = boto_session.client("sagemaker-runtime") + session = sagemaker.session.Session( + boto_session=boto_session, + sagemaker_client=sagemaker_client, + sagemaker_runtime_client=runtime_client, + default_bucket=default_bucket, + ) + + return session + + +def get_pipeline( + region: str, + hugging_face_model_id: str, + role: Optional[str] = None, + default_bucket: Optional[str] = None, + model_package_group_name: str = "AbalonePackageGroup", + pipeline_name: str = "AbalonePipeline", + project_id: str = "SageMakerProjectId", +) -> Any: + sagemaker_session = get_session(region, default_bucket) + if role is None: + role = sagemaker.session.get_execution_role(sagemaker_session) + + # parameters for pipeline execution + model_approval_status = ParameterString(name="ModelApprovalStatus", default_value="PendingManualApproval") + + inference_image_uri = get_huggingface_llm_image_uri("huggingface", version="0.9.3") + llm_model = HuggingFaceModel( + role=role, + image_uri=inference_image_uri, + env={ + "HF_MODEL_ID": hugging_face_model_id, # model_id from hf.co/models + "SM_NUM_GPUS": json.dumps(1), # Number of GPU used per replica + "MAX_INPUT_LENGTH": json.dumps(2048), # Max length of input text + "MAX_TOTAL_TOKENS": json.dumps(4096), # Max length of the generation (including input text) + "MAX_BATCH_TOTAL_TOKENS": json.dumps( + 8192 + ), # Limits the number of tokens that can be processed in parallel during the generation + "HUGGING_FACE_HUB_TOKEN": ACCESS_TOKEN, + # ,'HF_MODEL_QUANTIZE': "bitsandbytes", # comment in to quantize + }, + ) + + step_register = RegisterModel( + name="RegisterModel", + model=llm_model, + # estimator=xgb_train, + # image_uri=inference_image_uri, + # model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts, + content_types=["text/csv"], + response_types=["text/csv"], + inference_instances=["ml.g5.2xlarge", "ml.g5.12xlarge"], + # transform_instances=["ml.g5.12xlarge", "ml.p4d.24xlarge"], + model_package_group_name=model_package_group_name, + approval_status=model_approval_status, + ) + + # pipeline instance + pipeline = Pipeline( + name=pipeline_name, + parameters=[ + model_approval_status, + ], + steps=[step_register], + sagemaker_session=sagemaker_session, + ) + return pipeline diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/setup.cfg b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/setup.cfg new file mode 100644 index 00000000..6f878705 --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/setup.cfg @@ -0,0 +1,14 @@ +[tool:pytest] +addopts = + -vv +testpaths = tests + +[aliases] +test=pytest + +[metadata] +description-file = README.md +license_file = LICENSE + +[wheel] +universal = 1 diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/setup.py b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/setup.py new file mode 100644 index 00000000..a27183bb --- /dev/null +++ b/modules/sagemaker/sagemaker-templates-service-catalog/templates/hf_import_models/seed_code/build_app/setup.py @@ -0,0 +1,78 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# SPDX-License-Identifier: MIT-0 +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of this +# software and associated documentation files (the "Software"), to deal in the Software +# without restriction, including without limitation the rights to use, copy, modify, +# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A +# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +import os +from typing import Any, Dict + +import setuptools + +about: Dict[str, Any] = {} +here = os.path.abspath(os.path.dirname(__file__)) +with open(os.path.join(here, "ml_pipelines", "__version__.py")) as f: + exec(f.read(), about) + + +with open("README.md", "r") as f: + readme = f.read() + + +required_packages = ["sagemaker"] +extras = { + "test": [ + "black", + "coverage", + "flake8", + "mock", + "pydocstyle", + "pytest", + "pytest-cov", + "sagemaker", + "tox", + ] +} +setuptools.setup( + name=about["__title__"], + description=about["__description__"], + version=about["__version__"], + author=about["__author__"], + author_email=about["__author_email__"], + long_description=readme, + long_description_content_type="text/markdown", + url=about["__url__"], + license=about["__license__"], + packages=setuptools.find_packages(), + include_package_data=True, + python_requires=">=3.6", + install_requires=required_packages, + extras_require=extras, + entry_points={ + "console_scripts": [ + "get-pipeline-definition=ml_pipelines.get_pipeline_definition:main", + "run-pipeline=ml_pipelines.run_pipeline:main", + ] + }, + classifiers=[ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "Natural Language :: English", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.6", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + ], +) diff --git a/modules/sagemaker/sagemaker-templates-service-catalog/tests/test_stack.py b/modules/sagemaker/sagemaker-templates-service-catalog/tests/test_stack.py index 53666e89..de5c3805 100644 --- a/modules/sagemaker/sagemaker-templates-service-catalog/tests/test_stack.py +++ b/modules/sagemaker/sagemaker-templates-service-catalog/tests/test_stack.py @@ -76,7 +76,7 @@ def test_synthesize_stack(stack: cdk.Stack) -> None: template = Template.from_stack(stack) template.resource_count_is("AWS::ServiceCatalog::Portfolio", 1) - template.resource_count_is("AWS::ServiceCatalog::CloudFormationProduct", 3) + template.resource_count_is("AWS::ServiceCatalog::CloudFormationProduct", 4) def test_no_cdk_nag_errors(stack: cdk.Stack) -> None: