Not able to run az ml folder attach

[rcavalieri819]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
az ml folder attach was working with the previous az cli version 2.29.1 vs 2.30.0.

To Reproduce
az ml folder attach

Expected behavior
Instead of a success, we get:

2021-11-02T13:36:50.8211470Z [command]/bin/bash /agent/_work/_temp/azureclitaskscript1635860209071.sh
2021-11-02T13:36:51.8133956Z ERROR: __init__() got an unexpected keyword argument 'async_persist'

Environment summary
Use azure-cli-ml from Azure DevOps agents

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

Issue Details

---

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
az ml folder attach was working with the previous az cli version 2.29.1 vs 2.30.0.

To Reproduce
az ml folder attach

Expected behavior
Instead of a success, we get:

2021-11-02T13:36:50.8211470Z [command]/bin/bash /agent/_work/_temp/azureclitaskscript1635860209071.sh
2021-11-02T13:36:51.8133956Z ERROR: __init__() got an unexpected keyword argument 'async_persist'

Environment summary
Use azure-cli-ml from Azure DevOps agents

Author:	rcavalieri819
Assignees:	-
Labels:	Service Attention, Machine Learning, customer-reported
Milestone:	-

[yonzhan]

route to service team

[dkmiller]

Relevant service team has been notified and is actively working on a fix.

[Mossaka]

@rcavalieri819 To unblock you, could you please downgrade the azure-cli/azure-cli-core version to be less than 2.30.0?

You can refer to this comment on how to downgrade the azure cli. #13331

[ranikamadurawe]

This is an Issue effecting one of our pipelines as well. Specifically when running the az ml model deploy command. Is there an ETA for the fix?

[jhliberis]

@ranikamadurawe I have managed to workaround this by editing authentication.py in .Azure\cliextensions\azure-cli-ml\azureml\core. Removing the 'async_persist=False' parameter seems to have done thr trick.

[nish112]

I tried @jhliberis method but was not working for me and then i tried downgrading az cli to 2.29.1 as suggested by @Mossaka , and downgrading worked for me. I am using self hosted agent and i ran below command for downgrading:
sudo apt-get install azure-cli=2.29.1-1~bionic

[apthagowda97]

Update:
sudo apt-get install azure-cli=2.29.1-1~bionic won't work in Azure Cli Task in Azure Pipelines.

Now Azure CLI task is running with v2.30.0 in all the pipelines, I am getting the same error everywhere.

Please help me find the fix or rollback to the previous version in Azure CLI Task.

[virvirlopez]

im having the same error, any updates? please

[sgutwein]

Seeing the same error by running az ml datastore detach and az ml datastore attach-adls-gen2

azure-cli                         2.30.0
core                              2.30.0
telemetry                          1.0.6
Extensions:
azure-cli-ml                      1.33.0

[virvirlopez]

Our error is in our releases in azurecli with the command:
az ml workspace show -g X-AML-RG -w X-AML-WS --query keyVault -o tsv
ERROR: init() got an unexpected keyword argument 'async_persist'

[guidodizi]

Getting the same error here, any news on updates or quick fixes?

[Jerry1309]

we were able to fix the issue in DevOps using the following (installing 2.29.2)

            sudo apt remove azure-cli -y && sudo apt autoremove -y;

            sudo apt-get update;

            sudo apt-get install ca-certificates curl apt-transport-https lsb-release gnupg;

            curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null ;

            AZ_REPO=$(lsb_release -cs);

            echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list ;

            sudo apt-get update;

            apt-cache policy azure-cli;

            sudo apt-get install azure-cli=2.29.2-1~bionic;

[sbohlen]

Thanks, @Jerry1309 this works great as a work-around, thanks for posting/sharing so we can all get temporarily unblocked!

Silly comment, but could you edit your formatting so that the first/initial line is also included in the code block region? This initially didn't work for me b/c the COPY CODE feature of the site doesn't include the first line of your script and I missed porting it in my first-pass attempt at duplicating your success for our own pipeline :P

(just a suggestion to save others some time troubleshooting if/when they also attempt to repro based on your post, thanks again!)

[kgejara]

Thanks @Jerry1309. I have blocked with this error when I tried MLOps Poc, my error is also same, I was able to resolve the issue with the workaround you provided. Script lines worked the exact same for me. Only change for me is, I need to add the "--allow-downgrades" option in the last command when I notice a downgrade error, and I tested in 'ubuntu-18.04'.

[Jerry1309]

Thanks @Jerry1309. I have blocked with this error when I tried MLOps Poc, my error is also same, I was able to resolve the issue with the workaround you provided. Script lines worked the exact same for me. Only change for me is, I need to add the "--allow-downgrades" option in the last command when I notice a downgrade error, and I tested in 'ubuntu-18.04'.

Glad to know it worked for you, you might have missed the cli removal part.

Option 2 in this documentation helped me https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt

[apthagowda97]

Still getting errors while running sudo apt-get install azure-cli=2.29.2-1~bionic;

E: Version '2.29.2-1~bionic' for 'azure-cli' was not found
/home/vsts/work/_temp/azureclitaskscript1636613378792.sh: line 17: az: command not found
/home/vsts/work/_temp/azureclitaskscript1636613378792.sh: line 19: az: command not found
##[error]Script failed with error: Error: The process '/usr/bin/bash' failed with exit code 127

Any help?

[Jerry1309]

Still getting errors while running sudo apt-get install azure-cli=2.29.2-1~bionic;

E: Version '2.29.2-1~bionic' for 'azure-cli' was not found
/home/vsts/work/_temp/azureclitaskscript1636613378792.sh: line 17: az: command not found
/home/vsts/work/_temp/azureclitaskscript1636613378792.sh: line 19: az: command not found
##[error]Script failed with error: Error: The process '/usr/bin/bash' failed with exit code 127

Any help?

did you add the cli software repo by running
AZ_REPO=$(lsb_release -cs)
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" |
sudo tee /etc/apt/sources.list.d/azure-cli.list

if sudo apt-get install azure-cli=2.29.2-1~bionic is not working for you try with

sudo apt-get install azure-cli=2.29.2-1~focus

hope you are following all the commands including the removal of the current version from the Agent

apt-cache policy azure-cli
running this will list all available versions for your distribution. you can pick the right 2.29.2 version from there
if your agent is ubuntu 20.4 bionic may not be the right version mine was 18.04 and bionic worked

[apthagowda97]

Yes, Thanks. It solves that error by using sudo apt-get install azure-cli=2.29.2-1~focal; but it asks for az login again.

I guess I need to write that step manually again.

Update:

Finally it works!!

sudo apt remove azure-cli -y && sudo apt autoremove -y;
sudo apt-get update;
sudo apt-get install ca-certificates curl apt-transport-https lsb-release gnupg;

curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null ;

AZ_REPO=$(lsb_release -cs)
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" |
sudo tee /etc/apt/sources.list.d/azure-cli.list

sudo apt-get update;
apt-cache policy azure-cli;
sudo apt-get install azure-cli=2.29.2-1~focal;

az login --service-principal -u $servicePrincipalId --password=$servicePrincipalKey --tenant $tenantId

1. Remember to check the compatible version of Azure CLI (focal, bionic, etc)
2. In the advanced section of Azure CLI Task enable Access service principal details in script

[caiomsouza]

I had the same issue with version 2.30.

It did work with 2.29.

I used the command below to change from 2.30 to 2.29.

python -m pip install -U --force-reinstall pip pip install azure-cli==2.29

[wfranceys]

This is blocking for us (we are on MS hosted agents using az-cli 2.30.0, we cannot downgrade azure-cli-core to 2.29.0 as there is incompatibility between az-cli 2.30.0 and 2.29.0 see here)

[pragasv]

Downgrading to azure cli to 2.29 is causing new issues in downloading azure artifacts. As azure-devops extension is not compatible with it.
Any intermediate solutions ?

WARNING: The 'azure-devops' extension version 0.22.0 is not compatible with your current CLI core version 2.29.0.
This extension requires a min of 2.30.0 CLI core.
Please run 'az upgrade' to upgrade to a compatible version.
ERROR: 'artifacts' is misspelled or not recognized by the system.

[dkmiller]

@pragasv , you need to do something like this:

pip install --upgrade --force-reinstall --no-cache-dir "azure-cli<2.30.0"
az extension remove -n azure-devops
az extension add -n azure-devops --version 0.21.0

[pragasv]

@dkmiller thanks for the comment, when i tried the approach you suggested i ran into a following issue for the same service connection. Is this a known issue ?

ERROR: Unable to get 'ResourceType.MGMT_RESOURCE_SUBSCRIPTIONS' in profile 'latest'
##[error]Error Code: [1]
##[error]Error: Azure login failed
##[error]Script failed with error: ERROR: Unable to get 'ResourceType.MGMT_RESOURCE_SUBSCRIPTIONS' in profile 'latest'

also i was able to get the pipeline for the following version configurations now.

azure-cli                         2.30.0

core                              2.30.0
telemetry                          1.0.6

Extensions:
azure-devops                      0.22.0

[KoenDeCausmaecker-TomTom]

@ranikamadurawe I have managed to workaround this by editing authentication.py in .Azure\cliextensions\azure-cli-ml\azureml\core. Removing the 'async_persist=False' parameter seems to have done thr trick.

Thanks to this suggestion, I monkey-patched azure.cli.core._profile.Profile.__init__ to accept the old async_persist argument:

import azure.cli.core._profile

profile_init_orig = azure.cli.core._profile.Profile.__init__

def profile_init_new(__self, cli_ctx=None, storage=None, async_persist=False):
    profile_init_orig(__self, cli_ctx, storage)

azure.cli.core._profile.Profile.__init__ = profile_init_new

[jamescross91]

This still isn't fixed, and updates please? @dkmiller perhaps you can help?

[santiagxf]

@pragasv, Downgrade to version 2.29.1 as indicated in Install specific version - Microsoft Docs . In my case most of the Azure ML CLI v1 are failing with the last version of Azure CLI. I won't suggest pip install directly as you may break other things. You can refer to this GitHub Action that installs an specific version of the CLI if needed aml-cli-install

[rammada]

Im facing the same error __init__() got an unexpected keyword argument 'async_persist' in my Github actions workflow. I
cannot downgrade to CLI version to 2.29 since I'm using OIDC authentication for logging in to azure and OIDC requires at least 2.3 version of CLI. Please suggest a work around for this.

[Jerry1309]

Im facing the same error __init__() got an unexpected keyword argument 'async_persist' in my Github actions workflow. I cannot downgrade to CLI version to 2.29 since I'm using OIDC authentication for logging in to azure and OIDC requires at least 2.3 version of CLI. Please suggest a work around for this.

this issue got resolved with latest version of azml cli extension

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!