ADObjectPermissionEntry: Occasional error creating PSDrive

[oWretch]

Details of the scenario you tried and the problem that is occurring

As part of a full SQL Server AlwaysOn configuration, I am attempting to set the permissions on the AlwaysOn Listener computer object. Occasionally this will fail with the following error:

PowerShell DSC resource MSFT_ADObjectPermissionEntry  failed to execute Set-TargetResource functionality with error
message: System.InvalidOperationException: Error creating AD: PS Drive. (ADCOMMON0033) --->
System.Management.Automation.ProviderNotFoundException: Cannot find a provider with the name 'ActiveDirectory'.
   at System.Management.Automation.SessionStateInternal.GetProvider(PSSnapinQualifiedName providerName)
   at System.Management.Automation.SessionStateInternal.GetSingleProvider(String name)
   at Microsoft.PowerShell.Commands.NewPSDriveCommand.ProcessRecord()
   --- End of inner exception stack trace ---
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure
    + PSComputerName        : Cluster-Node-1

I am running the following modules:
ActiveDirectoryDsc: 4.2.0.0
ActiveDirectory: 1.0.0.0 (as ships with Server 2016 build 14393.2551)

This error only occurs with the ADObjectPermissionEntry resource, and not everytime I deploy and run the DSC configuration. But once the error occurs, it will always occur for every ADObjectPermissionEntry resource, but other resources I am using (ADManagedServiceAccount, ADServicePrincipalName, and ADComputer) never fail with this issue.

Verbose logs showing the problem

VERBOSE: [Cluster-Node-1]: LCM:  [ Start  Resource ]  [[ADComputer]AgListenerAccount]
VERBOSE: [Cluster-Node-1]: LCM:  [ Start  Test     ]  [[ADComputer]AgListenerAccount]
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] Importing the module MSFT_ADComputer in force mode.
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] Importing the module MSFT_ADComputer in force mode.
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] Determining the current state of the computer account 'Listener-Account'. (ADC0006)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] Retrieving the information about the computer account 'Listener-Account' from Active Directory. (ADC0002)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] The computer account 'Listener-Account' is present in Active Directory. (ADC0003)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] Evaluating the state of the property 'Description'. (ADCOMMON0003)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] The parameter 'Description' is in desired state. (ADCOMMON0004)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] Evaluating the state of the property 'Path'. (ADCOMMON0003)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] The parameter 'Path' is in desired state. (ADCOMMON0004)
VERBOSE: [Cluster-Node-1]:                            [[ADComputer]AgListenerAccount] The computer account 'Listener-Account' is in the desired state. (ADC0011)
VERBOSE: [Cluster-Node-1]: LCM:  [ End    Test     ]  [[ADComputer]AgListenerAccount]  in 0.4290 seconds.
VERBOSE: [Cluster-Node-1]: LCM:  [ Skip   Set      ]  [[ADComputer]AgListenerAccount]
VERBOSE: [Cluster-Node-1]: LCM:  [ End    Resource ]  [[ADComputer]AgListenerAccount]
VERBOSE: [Cluster-Node-1]: LCM:  [ Start  Resource ]  [[ADObjectPermissionEntry]ClusterFullControlToAgListener]
VERBOSE: [Cluster-Node-1]: LCM:  [ Start  Test     ]  [[ADObjectPermissionEntry]ClusterFullControlToAgListener]
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Importing the module MSFT_ADObjectPermissionEntry in force mode.
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Importing the module MSFT_ADObjectPermissionEntry in force mode.
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Object permission entry not found on object 'CN=Listener-Account,OU=Servers,DC=example,DC=com'. (OPE0002)
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Object permission entry on object 'CN=Listener-Account,OU=Servers,DC=example,DC=com' is not in the desired state. (OPE0006)
VERBOSE: [Cluster-Node-1]: LCM:  [ End    Test     ]  [[ADObjectPermissionEntry]ClusterFullControlToAgListener]  in 0.4270 seconds.
VERBOSE: [Cluster-Node-1]: LCM:  [ Start  Set      ]  [[ADObjectPermissionEntry]ClusterFullControlToAgListener]
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Importing the module MSFT_ADObjectPermissionEntry in force mode.
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Importing the module MSFT_ADObjectPermissionEntry in force mode.
VERBOSE: [Cluster-Node-1]:                            [[ADObjectPermissionEntry]ClusterFullControlToAgListener] Creating new AD: PSDrive. (ADCOMMON0032)
VERBOSE: [Cluster-Node-1]: LCM:  [ End    Set      ]  [[ADObjectPermissionEntry]ClusterFullControlToAgListener]  in 0.3820 seconds.
PowerShell DSC resource MSFT_ADObjectPermissionEntry  failed to execute Set-TargetResource functionality with error message: System.InvalidOperationException: Error creating AD: PS Drive. (ADCOMMON0033) --->
System.Management.Automation.ProviderNotFoundException: Cannot find a provider with the name 'ActiveDirectory'.
   at System.Management.Automation.SessionStateInternal.GetProvider(PSSnapinQualifiedName providerName)
   at System.Management.Automation.SessionStateInternal.GetSingleProvider(String name)
   at Microsoft.PowerShell.Commands.NewPSDriveCommand.ProcessRecord()
   --- End of inner exception stack trace ---
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure
    + PSComputerName        : Cluster-Node-1

Suggested solution to the issue

The DSC configuration that is used to reproduce the issue (as detailed as possible)

Relevant section. The full DSC is much greater than this component.

$ConfigData = @{
    AllNodes = @(
        @{
            NodeName      = 'Cluster-Node-1'
            DNSDomainName = 'example.com'
            OuPath        = 'OU=Servers,DC=example,DC=com'
            ClusterName   = 'Cluster-Account'
            ListenerName  = 'Listener-Account'
        }
    )
}

Configuration ClusterPermissions {

    Import-DscResource -ModuleName ActiveDirectoryDsc

    WindowsFeature ActiveDirectoryPowerShell {
        Name   = 'RSAT-AD-PowerShell'
        Ensure = 'Present'
    }

    ADComputer AgListenerAccount {
        Ensure               = 'Present'
        DependsOn            = '[WindowsFeature]ActiveDirectoryPowerShell'
        PsDscRunAsCredential = $DomainAdminCredential

        ComputerName      = $Node.ListenerName
        Path              = $Node.OuPath
        Description       = 'SQL Server AlwaysOn listener name account'
    }

    ADObjectPermissionEntry ClusterFullControlToAgListener {
        Ensure                             = 'Present'
        DependsOn                          = '[ADComputer]AgListenerAccount'
        PsDscRunAsCredential               = $DomainAdminCredential

        Path                               = "CN=$($Node.ListenerName),$($Node.OuPath)"
        IdentityReference                  = "$($Node.DNSDomainName)\$($Node.ClusterName)$"
        AccessControlType                  = 'Allow'
        ActiveDirectoryRights              = 'GenericAll'
        ObjectType                         = '00000000-0000-0000-0000-000000000000'
        ActiveDirectorySecurityInheritance = 'None'
        InheritedObjectType                = '00000000-0000-0000-0000-000000000000'
    }
}

The operating system the target node is running

OsName               : Microsoft Windows Server 2016 Datacenter
OsOperatingSystemSKU : DatacenterServerEdition
OsArchitecture       : 64-bit
WindowsBuildLabEx    : 14393.2551.amd64fre.rs1_release.181004-1309
OsLanguage           : en-US
OsMuiLanguages       : {en-US}

Version and build of PowerShell the target node is running

Name                           Value
----                           -----
PSVersion                      5.1.14393.2515
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.2515
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Version of the DSC module that was used ('dev' if using current dev branch)

4.2.0.0

[X-Guardian]

Hi @oWretch, the AdObjectPermissionEntry is the only ActiveDirectoryDsc resource that uses the AD PSDrive, so that is why the other resources aren't affected.

The problem is that the AD PSDrive is created when the ActiveDirectory PowerShell module is loaded into the session. In your case, that will be when the ADComputer resource is executed. I have seen before that sometimes the AD PSdrive doesn't get created for some reason, but I have never been able to reproduce it often enough to do any debugging and implement any remediation code. How reliably can you recreate it?

[oWretch]

@X-Guardian It is pretty consistent at present. Let me know what other logs you want me to gather etc to help track it down.

I don’t have an issue with the AD drive loading if I manually load the module.

[X-Guardian]

Excellent, that should make life easier. Are you running your PowerShell session using the same credentials that you are specifying in the PsDscRunAsCredential when you are successfully getting the AD drive mounted?

[X-Guardian]

Assuming you are, lets try force-loading the PowerShell module when the AdObjectPermissionEntry resource starts. This can be done by modifying the ActiveDirectoryDsc Assert-Module function in Modules/ActiveDirectoryDsc.Common/ActiveDirectoryDsc.Common.psm1 and adding a -Force parameter onto the Import-Module -Name $ModuleName command on line 524.

[oWretch]

Excellent, that should make life easier. Are you running your PowerShell session using the same credentials that you are specifying in the PsDscRunAsCredential when you are successfully getting the AD drive mounted?

Yes.

[oWretch]

Assuming you are, lets try force-loading the PowerShell module when the AdObjectPermissionEntry resource starts. This can be done by modifying the ActiveDirectoryDsc Assert-Module function in Modules/ActiveDirectoryDsc.Common/ActiveDirectoryDsc.Common.psm1 and adding a -Force parameter onto the Import-Module -Name $ModuleName command on line 524.

When I add the -Force parameter on line 524 the DSC application succeeds.

[oWretch]

I suspect the issue is that the AD module is still loaded, and it fails trying to load again. The modules I have loaded just before it tries to load the AD module are:

Get-Module | Select-Object Name,Version
Name                            Version
----                            -------
ActiveDirectory                 1.0.0.0
Microsoft.PowerShell.Management 3.1.0.0
Microsoft.PowerShell.Security   3.0.0.0
Microsoft.PowerShell.Utility    3.1.0.0
Microsoft.WSMan.Management      3.0.0.0
MSFT_ADComputer                 0.0    
MSFT_ADObjectPermissionEntry    0.0    

PSProviders:

Get-PsProvider
Name        Capabilities                        Drives                                           
----        ------------                        ------                                           
Registry    ShouldProcess, Transactions         {HKLM, HKCU}                                     
Alias       ShouldProcess                       {Alias}                                          
Environment ShouldProcess                       {Env}                                            
FileSystem  Filter, ShouldProcess, Credentials  {C, D, E, F}                                     
Function    ShouldProcess                       {Function}                                       
Variable    ShouldProcess                       {Variable}     

PSDrives:

Get-PSDrive
Name     Provider   
----     --------   
Alias    Alias      
C        FileSystem 
Cert     Certificate
D        FileSystem 
E        FileSystem 
Env      Environment
F        FileSystem 
Function Function   
HKCU     Registry   
HKLM     Registry   
Variable Variable   
WSMan    WSMan      

Interestingly, the AD Module is loaded but the AD drive is not available.

[X-Guardian]

Don't forget that the Dsc code is not running under your own user context, so modules you have loaded within your own PowerShell session have no bearing on Dsc, but what you are seeing there is likely to be the same scenario as what is happening when the Dsc runs. Interestingly, you don't even have the ActiveDirectory PSProvider loaded, so that it is the stage that is failing.
The only way to add a PSProvider to your PowerShell session is through importing the module that owns it, so I think that is the route we should take. I can add code to the Assert-ADPSDrive function to check if the ActiveDirectory PSProvider exists, and if not, force load the module.

[oWretch]

Don't forget that the Dsc code is not running under your own user context, so modules you have loaded within your own PowerShell session have no bearing on Dsc, but what you are seeing there is likely to be the same scenario as what is happening when the Dsc runs.

I meant to include that those dumps are taken when DSC is running. I added the export commands to the Assert-Module function to grab what was loaded in DSC.

[X-Guardian]

OK, excellent. That proves it then.

I've raised PR #528 to force load the ActiveDirectory module if the AD PS Provider isn't installed. Do you want to test it?

[oWretch]

Thanks @X-Guardian. That has solved the problem here.

[X-Guardian]

Excellent news. I'll get the PR review finished and then we can look at getting a new release published including this fix.

[shurick81]

When do you think it will be time to release this improvement?

[gaelcolas]

To be released, this module will have to be updated with the new release Pipeline automation. So hopefully before Xmas, depending on contributions.