Could not load file or assembly 'Azure.Security.KeyVault.Keys, Version=4.4.0.0'

[DonvanMeel]

Description

OS Version: 10.0.17763 Build 5936
Image Version: 20240707.1.0

After the upgrade of the Azure Pipelines Hosted Windows 2019 with VS2019 image to image version 20240707.1.0
We have a problem in our pipeline that results in

"Could not load file or assembly 'Azure.Security.KeyVault.Keys, Version=4.4.0.0, Culture=neutral, PublicKeyToken=927421***e12e44c8'. Could not find or load a specific file. (0x80131621)"

After exploring this problem en the changes of Image Version 20240707.1.0 we have found the powershell module that is causing the problem. It is the SqlServer Module version 22.3 When we unistall this version and install the older version 22.2 everything is working again.

We use the classic pipeline with the Azure Powershell task version 5.*

If we call the next scriptline then the Error is raised

Invoke-Sqlcmd -InputFile $ScriptCreateSchemaIfNotExists -Verbose -ConnectionString $DbConnectionStringToIdm2Database -Variable schema='[OU-IDM]', schemaCheck='OU-IDM'

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

OS Version: 10.0.17763 Build 5936
Image Version: 20240707.1.0

Is it regression?

yes, OS Version: 10.0.17763 Build 5936 Image Version: 20240630.1.1

Expected behavior

We dont get an error after calling
Invoke-Sqlcmd -InputFile $ScriptCreateSchemaIfNotExists -Verbose -ConnectionString $DbConnectionStringToIdm2Database -Variable schema='[OU-IDM]', schemaCheck='OU-IDM'

Actual behavior

We get this error:

"Could not load file or assembly 'Azure.Security.KeyVault.Keys, Version=4.4.0.0, Culture=neutral, PublicKeyToken=927421***e12e44c8'. Could not find or load a specific file. (0x80131621)"

Repro steps

See description

[kishorekumar-anchala]

Hi @DonvanMeel , We're looking into the issue , we will update you soon after investigating. Thank you !

[kishorekumar-anchala]

Hi @DonvanMeel ,

I suspect this is the driver version issue, could you please try by modifying the driver version?

[DonvanMeel]

Which driver version do you mean ? Beacause I have downgraded the SqlServer version from 22.3 to 22.2 and then it works again. See the description.

[kishorekumar-anchala]

@DonvanMeel , I have seen it , but now the version was changed to 22.3 .

22.3 version will not support some of the dependencies , we request you to update version from 4.4.0.0 to compatible version with 22.3 and also check with Azure SDK versions.

[kishorekumar-anchala]

HI @DonvanMeel ,

Could you please provide your comments ? Thankyou !

[timhoekstra]

We're seeing the exact same error from this 22.3.0 version since a few days when calling Invoke-Sqlcmd on both ubuntu-latest and windows-latest.
I'm a bit confused as to why this call is looking for the 4.4.x.x version of the dll as 22.2 includes 4.3.x.x and 22.3 includes 4.5.x.x in it's package as far as I can tell.

I'm not sure when the images were updated but I first noticed it around the 10th/11th of july. Some stages in our deployment pipeline were randomly failing at that time.
We call Install-Module -Name SqlServer -Force every time to make sure the module is installed before we execute our scripts that deploy our services to azure so I would not expect any wrong dll's to be loaded yet.
What I could determine is that for the stages that were passing the SqlServer module was not installed yet. Those failing had the 22.3 module already installed.

Does this need to be reported to https://github.com/microsoft/SQLServerPSModule/ as well?
There was some mentions of a possible breaking change as far as I can tell from reading the recent issue threads.
microsoft/SQLServerPSModule#88 is open, this could be part of the hotfix mentioned if the issue is on the powershell module side of things. Will look into the dll's that are loaded and see if that is indeed the issue. We do do some calls to keyvault before we invoke this function.

Edit: C:\Modules\az_11.3.1\Az.KeyVault\5.2.0\Azure.Security.KeyVault.Keys.dll is indeed loaded, which is 4.3.x.x?

[kishorekumar-anchala]

Hi @DonvanMeel ,

We reverted to the previous version and it will be available with next rollout . Thank you !

[kishorekumar-anchala]

Hi @DonvanMeel ,

Reverted to old version , Could you please try with latest version . Thank you !

[kishorekumar-anchala]

The reverted version(22.2) is now available with latest rollout. Thank you !