UseShouldProcessCorrectly Unable to find type

[jborean93]

Steps to reproduce

1. Create a script with just

Invoke-Pester

2. Install Pester with Install-Module -Name Pester

3. Then run pwsh -Command 'Invoke-ScriptAnalyzer -Path test.ps1' multiple times until you get the error

Expected behavior

Passes with no exception

Actual behavior

It usually succeeds normally but can sometimes fail with the exception

Invoke-ScriptAnalyzer : Unable to find type [Pester.OutputTypes].
At line:1 char:1
+ Invoke-ScriptAnalyzer -Path "/home/jborean/dev/test_script.ps ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (/home/jborean/dev/test_script.ps1:String) [Invoke-ScriptAnalyzer], RuntimeException
+ FullyQualifiedErrorId : RULE_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeScriptAnalyzerCommand
 

writeErrorStream      : True
PSMessageDetails      : 
Exception             : System.Management.Automation.RuntimeException: Unable to find type [Pester.OutputTypes].
                           at System.Management.Automation.TypeOps.ResolveTypeName(ITypeName typeName, IScriptExtent errorPos)
                           at System.Management.Automation.Language.Compiler.GetAttribute(TypeConstraintAst typeConstraintAst)
                           at System.Management.Automation.Language.TypeConstraintAst.GetAttribute()
                           at System.Management.Automation.Language.Compiler.GetRuntimeDefinedParameter(ParameterAst parameterAst, Boolean& customParameterSet, Boolean& 
                        usesCmdletBinding)
                           at System.Management.Automation.Language.Compiler.GetParameterMetaData(ReadOnlyCollection`1 parameters, Boolean automaticPositions, Boolean& 
                        usesCmdletBinding)
                           at 
                        System.Management.Automation.Language.FunctionDefinitionAst.System.Management.Automation.Language.IParameterMetadataProvider.GetParameterMetadata(Boolean 
                        automaticPositions, Boolean& usesCmdletBinding)
                           at System.Management.Automation.CompiledScriptBlockData.InitializeMetadata()
                           at System.Management.Automation.CompiledScriptBlockData.GetAttributes()
                           at System.Management.Automation.ScriptBlock.GetAttributes()
                           at System.Management.Automation.ScriptBlock.get_Attributes()
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.UseShouldProcessCorrectly.SupportsShouldProcess(String cmdName)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.UseShouldProcessCorrectly.CheckForSupportShouldProcess()
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.UseShouldProcessCorrectly.AnalyzeScript(Ast ast, String fileName)+MoveNext()
                           at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
                           at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.ScriptAnalyzer.<>c__DisplayClass82_1.<AnalyzeSyntaxTree>b__2()
TargetObject          : /home/jborean/dev/test_script.ps1
CategoryInfo          : InvalidOperation: (/home/jborean/dev/test_script.ps1:String) [Invoke-ScriptAnalyzer], RuntimeException
FullyQualifiedErrorId : RULE_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeScriptAnalyzerCommand
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {0, 1}

I've found that if I do pwsh -Command 'Import-Module Pester; Invoke-PSScriptAnalyzer -Path ... I cannot replicate the issue and the Pester types have been fully loaded. What is weird is that this only sometimes occurs which tells me the UseShouldProcessCorrectly rule is parsing module commands but there's a race condition where the type is not fully loaded by the time it tries to scan the attributes.

Some things I've tried

• Always importing Pester using Import-Module -Name Pester before calling PSSA, no more error but I don't want to do this for all modules in the code
• Reinstalled both Pester and PSScriptAnalyzer, still occassionally fails
• Use an older version of PSSA (1.17.1), works fine

My guess is that because this is an error only in the new version, the changes in #1074 or #1166 could be the cause.

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.2.0
PSEdition                      Core
GitCommitId                    6.2.0
OS                             Linux 5.0.7-arch1-1-ARCH #1 SMP PREEMPT Mon Apr 8 10:37:08 UTC 2019
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0


> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.18.0

> (Get-Module -ListAvailable Pester).Version | ForEach-Object { $_.ToString() }
4.7.3

[bergmeister]

Interesting, thanks for reporting it. I have seen similar problems before but in much more complex scenarios. Can you repro as well when you are already in the PowerShell shell and then call Invoke-ScriptAnalyzer?

The PRs that you mentioned brought back caching and improved it, which is something we'd not revert. I have talked with members of the PS team about similar exceptions that we sometimes see with PSSA and we concluded that this is probably not a race condition but a different multi-threading problem:
PSSA makes calls to Get-Command that populate the cache that might happen in different threads and it creates an instance of PowerShell for it. The returned objects have lazily initialised properties where PowerShell would evaluate them only when needed. Therefore it could be that we need to keep the PS runspace (which might be needed for evaluation) or that after the call to get-command we need to evaluate the properties to keep the property synced across threads or it could even be a PS bug.
PR #1178 improves PSSA to re-use a runspace pool, therefore it'd be interesting to see if this still happens with this PR.

[jborean93]

@bergmeister thanks for the reply, here are my replies to your questions;

Can you repro as well when you are already in the PowerShell shell and then call Invoke-ScriptAnalyzer?

Yes I can, it just appears to be more frequent if I do it outside of an interactive interpreter but that honestly could just be random luck.

The PRs that you mentioned brought back caching and improved it, which is something we'd not rever

Totally fine, didn't mean to say that they should be reverted, was just pointing them out as the potential changes that introduced the problem to help with investigating what may need to be done.

PR #1178 improves PSSA to re-use a runspace pool, therefore it'd be interesting to see if this still happens with this PR.

Unfortunately it doesn't help, I still get the same error with this branched checkout of PSScriptAnalyzer.

[bergmeister]

Thanks for the detailed feedback, I think in this we should try calling the properties on the CommandInfo object already at the point when we receive it from Get-Command because when I was chatting with @daxian-dbw he said the properties would then be set and we might not run into those issues of trying to determine the value at a later point in time in a different where PowerShell seems to not be able to do that any more.

[ztrhgf]

I have exactly the same problem with script, that contains some ActiveDirectory commands. Strange is, that it causes error only in VSC editor (not in PS console). And super strange is, that if I run Invoke-ScriptAnalyzer with -verbose it stills output the error, but not as terminating one. (I call the command in pre-commit git hook, PS script)

Error is:
Invoke-ScriptAnalyzer : Unable to find type [Microsoft.ActiveDirectory.Management.ADGroup]

[rjmholt]

I just hit this today. The problem occurs here:

https://github.com/PowerShell/PSScriptAnalyzer/blob/master/Rules/UseShouldProcessCorrectly.cs#L291-L345

This is really a PowerShell bug, since it should be able to provide us attributes on a CommandInfo object without having to load everything. If I can repro it in 7, we might be able to fix it in 7.1.

We should do the following:

• Wrap the current call in a try/catch
• See if we can resolve the attribute from the AST, if it's a function
• Give up gracefully and assume the command does not support -WhatIf

[bergmeister]

I think it's one of the many thread safety issues that PowerShell has, similar to PowerShell/PowerShell#9889

[rjmholt]

I think it's one of the many thread safety issues that PowerShell has

In this case I suspect it's a case of having a command info for a function without having loaded the module and therefore the types it depends on, but I could be wrong.

In any case, we can't rely on PowerShell to solve it and this is probably somewhere where we should do everything we can not to depend on the CommandInfo object, especially since that relies on local machine state (so PSSA won't give accurate information for this in CI for example).