Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switched to new GPU SKU because NC6 is deprecated #2462

Merged
merged 5 commits into from
Jul 18, 2023
Merged

Switched to new GPU SKU because NC6 is deprecated #2462

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5b8b446
Switched to new GPU SKU because NC6 is deprecated
jeff-shepherd Jul 17, 2023
c636abf
Run black
jeff-shepherd Jul 17, 2023
40e7578
Updated credentials for remaining V1 notebooks
jeff-shepherd Jul 17, 2023
09b2413
Updated gpu-cluster in bootstrap.sh
jeff-shepherd Jul 17, 2023
1a90b0a
Merge branch 'main' into jeffshep/v2ncclusterupdate
jeff-shepherd Jul 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-deploy-pytorch-mnist-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-deploy-scikit-learn-diabetes-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-sdk-experimental-automl-model-testing-classification-tsi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,14 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_AE_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
./v1/scripts/update-azure-extensions.sh
shell: bash
- name: attach to workspace
run: az ml folder attach -w main-python-sdk -g azureml-examples-rg
run: az ml folder attach -w main -g azureml-examples
- name: run classification-TSI.ipynb
run: papermill -k python classification-TSI.ipynb classification-TSI.output.ipynb
working-directory: v1/python-sdk/experimental/automl-model-testing/classification
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-sdk-experimental-automl-model-testing-forecasting-tsi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,14 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_AE_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
./v1/scripts/update-azure-extensions.sh
shell: bash
- name: attach to workspace
run: az ml folder attach -w main-python-sdk -g azureml-examples-rg
run: az ml folder attach -w main -g azureml-examples
- name: run forecasting-TSI.ipynb
run: papermill -k python forecasting-TSI.ipynb forecasting-TSI.output.ipynb
working-directory: v1/python-sdk/experimental/automl-model-testing/forecasting
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-sdk-experimental-automl-model-testing-regression-tsi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,14 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_AE_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
./v1/scripts/update-azure-extensions.sh
shell: bash
- name: attach to workspace
run: az ml folder attach -w main-python-sdk -g azureml-examples-rg
run: az ml folder attach -w main -g azureml-examples
- name: run regression-TSI.ipynb
run: papermill -k python regression-TSI.ipynb regression-TSI.output.ipynb
working-directory: v1/python-sdk/experimental/automl-model-testing/regression
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-deepspeed-cifar-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-deepspeed-transformers-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-fastai-mnist-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-fastai-mnist-mlproject-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-lightgbm-iris-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-pytorch-cifar-distributed-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-pytorch-mnist-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-pytorch-mnist-mlproject-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-scikit-learn-diabetes-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-scikit-learn-diabetes-mlproject-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-tensorflow-mnist-distributed-horovod-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-tensorflow-mnist-distributed-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-tensorflow-mnist-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-transformers-glue-1-aml-finetune-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-transformers-glue-2-aml-comparison-of-sku-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-transformers-glue-3-aml-hyperdrive-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-train-xgboost-iris-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
Expand Down
34 changes: 0 additions & 34 deletions .github/workflows/python-sdk-tutorial-automl-with-pycaret.yml
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-tutorial-dataset-uploads.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: install azmlcli
run: az extension add -n azure-cli-ml -y
- name: attach to workspace
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-tutorial-deploy-local.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: install azmlcli
run: az extension add -n azure-cli-ml -y
- name: attach to workspace
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-tutorial-deploy-triton.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: install azmlcli
run: az extension add -n azure-cli-ml -y
- name: attach to workspace
Expand Down
4 changes: 2 additions & 2 deletions ...lows/python-sdk-tutorial-metrics-binary-classification-metric-and-confidence-interval.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,14 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_AE_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
./v1/scripts/update-azure-extensions.sh
shell: bash
- name: attach to workspace
run: az ml folder attach -w main-python-sdk -g azureml-examples-rg
run: az ml folder attach -w main -g azureml-examples
- name: run binary-classification-metric-and-confidence-interval.ipynb
run: papermill -k python binary-classification-metric-and-confidence-interval.ipynb binary-classification-metric-and-confidence-interval.output.ipynb
working-directory: v1/python-sdk/tutorials/automl-with-azureml/metrics
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-sdk-tutorial-train-on-low-priority-aml-compute.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_AE_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: Run update-azure-extensions
run: |
chmod +x ./v1/scripts/update-azure-extensions.sh
./v1/scripts/update-azure-extensions.sh
shell: bash
- name: attach to workspace
run: az ml folder attach -w main-python-sdk -g azureml-examples-rg
run: az ml folder attach -w main -g azureml-examples
- name: run train-on-low-priority-aml-compute
run: papermill -k python train-on-low-priority-aml-compute.ipynb train-on-low-priority-aml-compute.output.ipynb
working-directory: v1/python-sdk/tutorials/train-on-low-priority-aml-compute
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-tutorial-using-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: install azmlcli
run: az extension add -n azure-cli-ml -y
- name: attach to workspace
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-tutorial-using-pytorch-lightning.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: install azmlcli
run: az extension add -n azure-cli-ml -y
- name: attach to workspace
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-sdk-tutorial-using-rapids.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: azure login
uses: azure/login@v1
with:
creds: ${{secrets.AZ_CREDS}}
creds: ${{secrets.AZUREML_CREDENTIALS}}
- name: install azmlcli
run: az extension add -n azure-cli-ml -y
- name: attach to workspace
Expand Down
2 changes: 1 addition & 1 deletion cli/endpoints/batch/deploy-models/imagenet-classifier/deploy-and-run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ az ml model create --name $MODEL_NAME --path "model"

echo "Creating compute with GPU"
# <create_compute>
az ml compute create -n gpu-cluster --type amlcompute --size STANDARD_NC6 --min-instances 0 --max-instances 2
az ml compute create -n gpu-cluster --type amlcompute --size STANDARD_NC6s_v3 --min-instances 0 --max-instances 2
# </create_compute>

echo "Creating batch endpoint $ENDPOINT_NAME"
Expand Down
4 changes: 2 additions & 2 deletions infra/bootstrapping/bootstrap.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,8 @@ if [[ ! -z "${RUN_BOOTSTRAP:-}" ]]; then
"$SCRIPT_DIR"/sdk_helpers.sh ensure_aml_compute "cpu-cluster-lg" 0 4 "Standard_DS15_v2"

echo_title "Ensuring GPU compute"
"$SCRIPT_DIR"/sdk_helpers.sh ensure_aml_compute "gpu-cluster" 0 20 "Standard_NC6"
"$SCRIPT_DIR"/sdk_helpers.sh ensure_aml_compute "automl-gpu-cluster" 0 4 "STANDARD_NC6"
"$SCRIPT_DIR"/sdk_helpers.sh ensure_aml_compute "gpu-cluster" 0 20 "STANDARD_NC6s_v3"
"$SCRIPT_DIR"/sdk_helpers.sh ensure_aml_compute "automl-gpu-cluster" 0 4 "STANDARD_NC6s_v3"
# v100 single GPU cluster for pytorch 2.0 based notebooks
"$SCRIPT_DIR"/sdk_helpers.sh ensure_aml_compute "gpu-v100-1GPU-cluster" 0 4 "Standard_NC6s_v3"
# v100 GPU cluster for deepspeed cli examples
Expand Down
2 changes: 1 addition & 1 deletion sdk/dotnet/AzureML-Samples-CSharp/Resources/Compute/AMLComputeOperations.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ public static async Task<MachineLearningComputeResource> GetOrCreateAMLComputeAs
{
ScaleSettings = new ScaleSettings(4),
VmSize = "Standard_DS3_v2",
// VmSize = "STANDARD_NC6",
// VmSize = "STANDARD_NC6s_v3",
}
},
};
Expand Down
5 changes: 4 additions & 1 deletion sdk/python/jobs/configuration.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,10 @@
"except Exception:\n",
" print(\"Creating a new gpu compute target...\")\n",
" compute = AmlCompute(\n",
" name=gpu_compute_target, size=\"STANDARD_NC6\", min_instances=0, max_instances=4\n",
" name=gpu_compute_target,\n",
" size=\"STANDARD_NC6s_v3\",\n",
" min_instances=0,\n",
" max_instances=4,\n",
" )\n",
" ml_client.compute.begin_create_or_update(compute).result()"
]
Expand Down
Loading