forked from Azure/azure-sdk-for-python
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-python …
…into azure_purview_scanning * 'master' of https://github.com/Azure/azure-sdk-for-python: (550 commits) del useless files (Azure#18528) delete existing useless files for trafficmanager (Azure#18525) Define new replacer to replace keys in recording (Azure#18294) [purview] add purview nspkg to ci (Azure#18523) ignore analysis for swagger readmes (Azure#18520) [purview] add azure-purview-nspkg (Azure#18518) [AppConfiguration] Appconfig consistency (Azure#18493) [Container Registry] Improved samples (Azure#18263) [Container Registry] renamings (Azure#18492) [ServiceBus] internal code rename and sample readme update (Azure#18516) [EventHub] update link in sample readme (Azure#18517) Post Process Event Names Script (Azure#18419) [Tables] use etag from entity if match condition is given (Azure#18271) adding operation-location to headers that are scrubbed (Azure#18514) [Tables] Adds support for AzureNamedKeyCredential (Azure#18456) [Tables] delete_entity takes an entity instead of row and partition key (Azure#18269) [Tables] Removed TableEntity attribute wrapper (Azure#18489) [EventHub&ServiceBus] Bump uAMQP dependency (Azure#17942) [ServiceBus] add keyword override support to update_ methods in mgmt module (Azure#18210) Add compatibility switch to disable CAE (Azure#18148) ...
- Loading branch information
Showing
7,449 changed files
with
1,176,678 additions
and
401,145 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
There are no files selected for viewing
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| jobs: | ||
| - template: /eng/pipelines/templates/jobs/smoke-test.yml | ||
| - template: /eng/pipelines/templates/jobs/smoke.tests.yml | ||
| parameters: | ||
| Daily: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # Azure SDK for Python Conda Distributions | ||
|
|
||
| ## Local Environment Setup | ||
|
|
||
| Follow the instructions [here](https://docs.conda.io/projects/conda-build/en/latest/install-conda-build.html) to install `conda` and `conda-build`. | ||
|
|
||
| ## CI Build Process | ||
|
|
||
| There will be a `CondaArtifact` defined in the `ci.yml` of each service directory. (`sdk/<service>`) | ||
|
|
||
| A Conda Artifact defines: | ||
| - The location of the `meta.yml` | ||
| - Which packages will be pulled into the combined artifact | ||
| - The name of the combined artifact | ||
| - Any other necessary details. | ||
|
|
||
| ## How to Build an Azure SDK Conda Package Locally | ||
|
|
||
| ### Set up your conda environment | ||
|
|
||
| You will notice that all the azure-sdk conda distributions have the **same** version number and requirement set. This is due to the fact that the azure-sdk team pushes our conda packages out in waves. To support this, all versions are set via a common environment variable `AZURESDK_CONDA_VERSION`. | ||
|
|
||
| We keep this environment variable set properly across all our builds by using a common `conda_env.yml` when creating our build environment. This environment definition ensures that: | ||
|
|
||
| 1. Our channel `https://azuresdkconda.blob.core.windows.net/channel1/` is added to the set to download packages | ||
| 2. The environment variable `AZURESDK_CONDA_VERSION` will be set exactly once. | ||
|
|
||
|
|
||
| Reference the `conda_env.yml` in your local build by pass `-f <path to conda_env.yml>` when you create your conda environment. | ||
|
|
||
| ``` | ||
| conda env create --yes --quiet --name ${{ artifact.name }} -f $(Build.SourcesDirectory)/eng/conda_env.yml | ||
| ``` | ||
|
|
||
| ### Create Your Build Directory | ||
| Given how Conda packages are comprised of multiple source distributions _combined_, the buildable source does not exist directly within the azure-sdk-for-python repo. Currently, there is _some_ manual work that needs to be done. | ||
|
|
||
| To begin, check your `ci.yml` for a `CondaArtifact`. Each these artifacts will become a single conda package. Let's use `storage/ci.yml` as an example. | ||
|
|
||
| ``` | ||
| - name: azure-storage | ||
| meta_source: meta.yml | ||
| common_root: azure/storage | ||
| checkout: | ||
| - package: azure-storage-blob | ||
| checkout_path: sdk/storage | ||
| version: 12.8.0 | ||
| - package: azure-storage-queue | ||
| checkout_path: sdk/storage | ||
| version: 12.1.5 | ||
| - package: azure-storage-file-share | ||
| checkout_path: sdk/storage | ||
| version: 12.4.1 | ||
| - package: azure-storage-file-datalake | ||
| checkout_path: sdk/storage | ||
| version: 12.3.0 | ||
| ``` | ||
|
|
||
| - `name: azure-storage`: will be the name of the "combined" sdist package that we generate. | ||
| - `meta_source: meta.yml`: this is the path (relative to the service directory) to the target conda package meta.yml. | ||
| - `common_root: azure/storage`: when generating the combined package, where will we begin combining? This is tightly bound to folder structure within the generated sdist. | ||
| - `checkout`: the `checkout` setting is a list of target packages that will go into the combined artifact. These targets will be individually sparse cloned, and copied into the conda build directory. Currently, this is a **manual step** in your local build. Reference `eng/pipelines/templates/get-tagged-code.yml` for exact details on how CI does it. | ||
|
|
||
| Before we continue, you should be aware of two primary locations that are necessary, but not referenced directly in the `ci.yml`. | ||
|
|
||
| The `build` folder and the `output` folder. The `build` folder (`$(Conda.Build)` variable in CI) is where we will... | ||
|
|
||
| - store the cloned package code | ||
| - generate the combined sdist | ||
|
|
||
| To locally repro without magic given a specific `checkout` artifact: | ||
|
|
||
| ``` | ||
| <cd sdk-for-python> | ||
| git checkout `<package>_<version>` | ||
| grab the entire <package> directory under the <checkout_path>. place into your `build` folder. | ||
| ``` | ||
|
|
||
| Given the `storage` example. This is what your `build` folder should look like prior to invoking `build_conda_artifacts.py`. | ||
|
|
||
| ``` | ||
| <build directory>/ | ||
| azure-storage-blob/ <-- the package directly ripped from specified tag | ||
| azure-storage-file-datalake/ | ||
| azure-storage-file-share/ | ||
| azure-storage-queue/ | ||
| ``` | ||
|
|
||
| ### Create the Combined SDist | ||
|
|
||
| Once you have a directory assembled, invoke the script to build. The below command is formatted for visibility, recombine the lines however necessary for your chosen shell environment. | ||
|
|
||
|
|
||
| ``` | ||
| python `build_conda_artifacts.py` | ||
| -d "<output folder>" | ||
| -b "<build folder>" | ||
| -m "<resolvable path to sdk/storage/meta.yml>" | ||
| -r "azure/storage" | ||
| -n "azure-storage" | ||
| -s "storage" | ||
| -e "<resolvable path to repo root>/eng/conda_env.yml" | ||
| -c "<resolvable path to sdk/storage/ci.yml>" | ||
| ``` | ||
|
|
||
| ### Generate the Conda Package | ||
|
|
||
| Locally, from the anaconda prompt, set the environment variable `STORAGE_SOURCE_DISTRIBUTION` to the location of the generated sdist. After that: | ||
|
|
||
| ```bash | ||
| export STORAGE_SOURCE_DISTRIBUTION=<path/to/generated/sdist> | ||
| cd <meta.yml directory> | ||
| conda-build . --output-folder <conda.output> | ||
| ``` |
Oops, something went wrong.