DSC on Azure Pipelines Ubuntu

[Gijsreyn]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest version
• Search the existing issues.

Summary

I am experiencing issues when installing dsc.exe on ubuntu-latest agent pool in Azure Pipelines. I'm running the following script:

jobs: 
  - job: test
    displayName: 'test'
    pool: 
      vmImage: ubuntu-latest
    steps: 
      - checkout: self 
        persistCredentials: true 
      - task: PowerShell@2
        displayName: "Run"
        inputs:
          pwsh: true
          targetType: "inline"
          script: |
            $uri = 'https://github.com/PowerShell/DSC/releases/download/v3.0.0-preview.11/DSC-3.0.0-preview.11-x86_64-unknown-linux-gnu.tar.gz'
            $filePath = '/tmp/DSC-' + '3.0.0-preview.11' + '-x86_64-unknown-linux-gnu.tar.gz'

            curl -L -o $filePath $uri
            # Create the target folder where powershell will be placed
            sudo mkdir -p /opt/microsoft/dsc

            # Expand dsc to the target folder
            sudo tar zxf $filePath -C /opt/microsoft/dsc

            # Set execute permissions
            sudo chmod +x /opt/microsoft/dsc

            # Create the symbolic link that points to dsc
            sudo ln -s /opt/microsoft/dsc /usr/bin/dsc

            # Add to path
            $env:PATH += [System.IO.Path]::PathSeparator + "/usr/bin/dsc"

            # Call version
            dsc --version 

            # Call list of resources
            dsc -l trace resource list
          ignoreLASTEXITCODE: true

The dsc --version is properly called, but when doing the resource list, I can observe:

This hangs for an infinity time.

Steps to reproduce

See description

Expected behavior

Able to run `dsc resource list` on ubuntu-latest agent

Actual behavior

Hangs

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.5
PSEdition                      Core
GitCommitId                    7.4.5
OS                             Microsoft Windows 10.0.26100
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Version

dsc 3.0.0-preview.11

Visuals

No response

[Gijsreyn]

The same is on macos-latest

[SteveL-MSFT]

@Gijsreyn can you confirm this is still an issue with latest release?

[Gijsreyn]

@SteveL-MSFT if your talking about preview.11, it was tested on that one. I can see preview.11 being the latest.

[SteveL-MSFT]

Thanks, I'll try to investigate this later this week

[SteveL-MSFT]

I suspect the problem is that ADO Pipelines always has STDIN, STDOUT, STDERR pipes redirected to send input and read output. dsc detects that STDIN is redirected and expecting input, but no input is actually ever sent hence the hang. So dsc needs to check if there's actually any input somehow.

[Gijsreyn]

That's not later Steve because your suspicion is right :)

When changing the line and adding input to it as such: {} | dsc -l trace resource list, it worked.

[SteveL-MSFT]

Use that as a workaround for now, working on a fix.

[SteveL-MSFT]

So I wrote a test that repros the problem by simply redirecting stdin and never sending input. The problem is that I don't see a way to not block on attempting to read stdin. Let me think about this more, but I think what I might have to do is add an explicit --stdin switch to inform dsc to read from stdin rather than just trying.

[Gijsreyn]

Thanks Steve for the follow up and diving right into it. I guess there are some rough edge cases still to be uncovered as dsc gets updates. If I find more, I'll definently raise an issue.

[SteveL-MSFT]

I don't like the explicit --stdin option, but I tried a few things and the problem is Rust follows a cross-platform model which is blocking reads of STDIN. This by itself is not a big problem as I can do the reading on another thread, the problem is detecting if any input is coming in and finishing the read and managing that with a timeout is too complex right now. This is only a problem in automation environments (like Azure DevOps Pipelines) which redirects all the pipes whether or not any input is sent to STDIN.

The good news is that even with the --stdin switch, if we eventually resolve the detection problem with STDIN, we can have it just work and keep the switch simply for compatibility.

[SteveL-MSFT]

After doing a bit of research on this, it appears --file - is the convention for indicating reading from STDIN. So I'll make that change and also change some of the arg names to align with conventions.