Wildcards in NO_PROXY environment variable break tasks relying on typed-rest-client

[maciejporebski]

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: DownloadPackageV1 (and likely other tasks using typed-rest-client)

Working version: 1.179.0
Broken version: 1.182.2

list here (V# not needed):
https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/DownloadPackageV1

Environment

• Server - Azure Pipelines or TFS on-premises? Azure Pipelines
• Agent - Hosted or Private: Private
  • Version: 2.164.8
  • OS: Windows Server 2016

Issue Description

When using a task which utilized typed-rest-client on a machine with no_proxy environment variable containing wildcards the task fails (in case of DownloadPackageV1 task it fails silently).

The failure is caused by the typed-rest-client constructing a RegExp for each value in the no_proxy list without escaping the wildcard: https://github.com/microsoft/typed-rest-client/blob/master/lib/HttpClient.ts#L146

Issue for root cause in typed-rest-client repo: microsoft/typed-rest-client#215
PR for root cause in typed-rest-client repo: microsoft/typed-rest-client#216

This issue mentions the same problem in NuGetAuthenticateTask: #14015

Task logs

2021-02-05T16:28:24.1614930Z ==============================================================================
2021-02-05T16:28:24.1614930Z Task         : Download package
2021-02-05T16:28:24.1614930Z Description  : Download a package from a package management feed in Azure Artifacts
2021-02-05T16:28:24.1614930Z Version      : 1.182.2
2021-02-05T16:28:24.1614930Z Author       : ms-vscs-rm
2021-02-05T16:28:24.1614930Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/download-package
...
2021-02-05T16:28:25.3946163Z (node:13168) UnhandledPromiseRejectionWarning: SyntaxError: Invalid regular expression: /*.<HOSTNAME REDACTED>/: Nothing to repeat
2021-02-05T16:28:25.3946163Z ##[debug]Found resource area with locationUrl: https://feeds.dev.azure.com/<ORG NAME REDACTED>/
2021-02-05T16:28:25.3956164Z     at new RegExp (<anonymous>)
2021-02-05T16:28:25.3956164Z ##[debug]Agent.ProxyUrl=<PROXY URL REDACTED>
2021-02-05T16:28:25.3956164Z     at HttpClient.no_proxy.split.forEach.bypass (D:\agent\_work\_tasks\DownloadPackage_8d6e8f7e-267d-442d-8c92-1f586864c62f\1.182.2\node_modules\azure-devops-node-api\node_modules\typed-rest-client\HttpClient.js:114:49)
2021-02-05T16:28:25.3956164Z ##[debug]Agent.ProxyUsername=undefined
2021-02-05T16:28:25.3956164Z     at Array.forEach (<anonymous>)
2021-02-05T16:28:25.3956164Z ##[debug]Agent.ProxyPassword=undefined
2021-02-05T16:28:25.3956164Z     at new HttpClient (D:\agent\_work\_tasks\DownloadPackage_8d6e8f7e-267d-442d-8c92-1f586864c62f\1.182.2\node_modules\azure-devops-node-api\node_modules\typed-rest-client\HttpClient.js:113:33)
2021-02-05T16:28:25.3956164Z ##[debug]Agent.ProxyBypassList=["<URL REDACTED>"]
2021-02-05T16:28:25.3956164Z     at new RestClient (D:\agent\_work\_tasks\DownloadPackage_8d6e8f7e-267d-442d-8c92-1f586864c62f\1.182.2\node_modules\azure-devops-node-api\node_modules\typed-rest-client\RestClient.js:25:23)
2021-02-05T16:28:25.3956164Z ##[debug]isTriggeringArtifact=undefined
2021-02-05T16:28:25.3956164Z     at new WebApi (D:\agent\_work\_tasks\DownloadPackage_8d6e8f7e-267d-442d-8c92-1f586864c62f\1.182.2\node_modules\azure-devops-node-api\WebApi.js:162:21)
2021-02-05T16:28:25.3956164Z ##[debug]Agent.Version=2.164.8
2021-02-05T16:28:25.3956164Z     at locationUtility.getServiceUriFromAreaId.then.url (D:\agent\_work\_tasks\DownloadPackage_8d6e8f7e-267d-442d-8c92-1f586864c62f\1.182.2\connections.js:23:16)
2021-02-05T16:28:25.3956164Z ##[debug]SYSTEM_TASKINSTANCEID=<ID REDACTED>
2021-02-05T16:28:25.3956164Z     at process._tickCallback (internal/process/next_tick.js:68:7)
2021-02-05T16:28:25.3956164Z ##[debug]SYSTEM_JOBID=<ID REDACTED>
2021-02-05T16:28:25.3956164Z (node:13168) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
2021-02-05T16:28:25.3956164Z ##[debug]SYSTEM_PLANID=<ID REDACTED>
2021-02-05T16:28:25.3956164Z (node:13168) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

[Enable debug logging and please provide the zip file containing all the logs for a speedy resolution]

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

[Insert error from the logs here for a quick overview]

[github-actions]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[johnsharland77]

Is there any news on getting this fixed? We are running Azure DevOps Server and have just upgraded to the latest server (2020.1.1 RC1) and the latest agent (2.189.0) and are getting the same error. This is preventing us from being able to use the Azure App Service Deploy task to deploy into Azure from our private agents.

[anatolybolshakov]

Hi @maciejporebski @johnsharland77 this issue seems to be already fixed for DownloadPackageV1 task (v1.185.0) and rolled out for Azure DevOps Services. To receive last task update on on-prem at the moment - you need to manually download it using tfx-cli tool.

[anatolybolshakov]

@maciejporebski @johnsharland77 let me close this one since it seems to be already resolved - please let us know if you still have any issues.

[maciejporebski]

Hey @anatolybolshakov. We now get this error when the Release pipeline attempts to download artifacts.

Task: Download build artifacts
Version: 0.190.0
Agent Version: 2.187.2

The version of 'typed-rest-client' in the node_modules of that task is 1.2.0

2021-08-11T15:34:29.0006293Z SyntaxError: Invalid regular expression: /*.example.com/: Nothing to repeat
2021-08-11T15:34:29.0006293Z     at new RegExp (<anonymous>)
2021-08-11T15:34:29.0006293Z     at HttpClient.requestOptions.proxy.proxyBypassHosts.forEach.bypass (<agent directory>\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.190.0\node_modules\typed-rest-client\HttpClient.js:100:53)
2021-08-11T15:34:29.0006293Z     at Array.forEach (<anonymous>)
2021-08-11T15:34:29.0006293Z     at new HttpClient (<agent directory>\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.190.0\node_modules\typed-rest-client\HttpClient.js:99:55)
2021-08-11T15:34:29.0006293Z     at new RestClient (<agent directory>\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.190.0\node_modules\typed-rest-client\RestClient.js:25:23)
2021-08-11T15:34:29.0006293Z     at new WebApi (<agent directory>\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.190.0\node_modules\azure-devops-node-api\WebApi.js:147:21)
2021-08-11T15:34:29.0006293Z     at <agent directory>\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.190.0\main.js:98:26
2021-08-11T15:34:29.0006293Z     at Generator.next (<anonymous>)
2021-08-11T15:34:29.0016291Z     at <agent directory>\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.190.0\main.js:8:71
2021-08-11T15:34:29.0016291Z     at new Promise (<anonymous>)

[anatolybolshakov]

Hi @maciejporebski reopening this one - let us take a look.

[anatolybolshakov]

Looks like issue is still relevant for tasks which use typed-rest-client less than v1.8.3 (fixed in PR) - for azure-devops-node-api it was picked up in v10.2.1. We need to update all tasks which use these dependencies - to resolve issue for other in-the-box tasks.

[anatolybolshakov]

Hi @maciejporebski we've rolled out related fix - closing this issue at the moment. Please let us know if you have any other questions.