Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[eng/common] Helper function "Install-ModuleIfNotInstalled" is not threadsafe #6514

Closed
mikeharder opened this issue Jul 11, 2023 · 0 comments · Fixed by #6521
Closed

[eng/common] Helper function "Install-ModuleIfNotInstalled" is not threadsafe #6514

mikeharder opened this issue Jul 11, 2023 · 0 comments · Fixed by #6521
Assignees
Labels
EngSys This issue is impacting the engineering system. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team.

Comments

@mikeharder
Copy link
Member

mikeharder commented Jul 11, 2023

In pipeline "Autorest Regen Preview", the following command fails intermittently. I think this might be a race condition, where multiple parallel threads (39 and 40) are installing the same module at the same time, and it's possible for one thread to see a partial install.

   1:7>Project "/mnt/vss/_work/1/s/azure-sdk-for-net/eng/service.proj" (1:7) is building "/mnt/vss/_work/1/s/azure-sdk-for-net/sdk/healthinsights/Azure.Health.Insights.ClinicalMatching/src/Azure.Health.Insights.ClinicalMatching.csproj" (40:9) on node 2 (GenerateCode target(s)).
    40>_GenerateCodeForTypeSpec:
         pwsh -NoProfile -NonInteractive -executionpolicy Unrestricted -File /mnt/vss/_work/1/s/azure-sdk-for-net/sdk/healthinsights/Azure.Health.Insights.ClinicalMatching/src/../../../../eng/common/scripts/TypeSpec-Project-Sync.ps1 /mnt/vss/_work/1/s/azure-sdk-for-net/sdk/healthinsights/Azure.Health.Insights.ClinicalMatching/src/.. 
    39>_GenerateCodeForTypeSpec:
         Installing module powershell-yaml with min version 0.4.1 from https://www.powershellgallery.com/api/v2
    40>_GenerateCodeForTypeSpec:
         Installing module powershell-yaml with min version 0.4.1 from https://www.powershellgallery.com/api/v2
         Using module powershell-yaml with version 0.4.7.
    39>_GenerateCodeForTypeSpec:
         Using module powershell-yaml with version 0.4.7.
    40>_GenerateCodeForTypeSpec:
         ParentContainsErrorRecordException: /home/cloudtest/.local/share/powershell/Modules/powershell-yaml/0.4.7/Load-Assemblies.ps1:28
         Line |
           28 |  …           return [Reflection.Assembly]::LoadFrom($assemblies["core"])
              |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
              | Exception calling "LoadFrom" with "1" argument(s): "Could not load file
              | or assembly
              | '/home/cloudtest/.local/share/powershell/Modules/powershell-yaml/0.4.7/lib/netstandard2.1/YamlDotNet.dll'. The system cannot find the file specified. "

https://dev.azure.com/azure-sdk/internal/_build/results?buildId=2906603&view=logs&j=a59e8b2b-145b-5c08-bac3-c8d41476a686&t=b360b5d6-342a-59db-d57c-25f0340a37f5&l=4073

I suspect the root cause is one of the Install-ModuleIfNotInstalled commands in azure-sdk-for-net:

https://github.com/search?q=repo%3AAzure%2Fazure-sdk-for-net%20powershell-yaml&type=code

One level deeper, I believe the root cause is using Install-Module -Force, which can overwrite the module and cause a parallel Import-Module to fail.

https://github.com/Azure/azure-sdk-for-net/blob/3ab9a7ac57f8a7672079c1ffba1ad8f95cfa5a1b/eng/common/scripts/Helpers/PSModule-Helpers.ps1#L86

@mikeharder mikeharder self-assigned this Jul 11, 2023
@chunyu3 chunyu3 added the EngSys This issue is impacting the engineering system. label Jul 12, 2023
@mikeharder mikeharder transferred this issue from Azure/autorest.csharp Jul 14, 2023
@github-actions github-actions bot added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jul 14, 2023
@mikeharder mikeharder moved this from 🐝 Dev to 🔬 Dev in PR in Azure SDK EngSys 🚢🎉 Jul 14, 2023
@github-project-automation github-project-automation bot moved this from 🔬 Dev in PR to 🎊 Closed in Azure SDK EngSys 🚢🎉 Jul 15, 2023
@mikeharder mikeharder moved this from 🎊 Closed to 🔬 Dev in PR in Azure SDK EngSys 🚢🎉 Jul 15, 2023
@mikeharder mikeharder changed the title Intermittent failure loading powershell-yaml [eng/common] Helper function "Install-ModuleIfNotInstalled" is not threadsafe Jul 15, 2023
@benbp benbp moved this from 🔬 Dev in PR to 🎊 Closed in Azure SDK EngSys 🚢🎉 Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
EngSys This issue is impacting the engineering system. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants