[New Feature]: User-Agent

[OfficialEsco]

Description of the new feature/enhancement

When you try to download https://dl.genymotion.com/releases/genymotion-3.2.1/genymotion-3.2.1.exe via YamlCreate and WinGetCreate you'll get
[Error] Invalid URL Response - The URL did not return a successful response from the server
or
HTTP response was unsuccessful. Status code: Forbidden

---

A User-Agent like Microsoft-Delivery-Optimization/10.1 is needed to visit this URL, otherwise it will return a Cloudflare captcha page.
Both WinGetCreate and YamlCreate should put a User-Agent into the session.

Originally posted by @SpecterShell in microsoft#34770 (comment)

Proposed technical implementation details (optional)

No response

[OfficialEsco]

---

Should be done in this way:

$_WebClient.Headers.Add("User-Agent", "Microsoft-Delivery-Optimization/10.1")

And do not forget Test-Url:

$HTTP_Request.Headers.Add("User-Agent", "Microsoft-Delivery-Optimization/10.1")

Originally posted by @SpecterShell in microsoft#34770 (comment)

---

Hmm, this might resolve multiple issues?

Looks like they changed how they distribute URLs ;

I found the files on their site, but it looks like it requires the request to have the headers set. As long as the header has the
referrer, then it works. Probably nothing we can do about it for now though

wget -Headers @{'Referer'='https://www.irfanview.info/files/iview459_setup.exe'} https://www.irfanview.info/files>/iview459_setup.exe -OutFile D:\Users\Trenly\Downloads\Temp\iview_wget_test.exe

Originally posted by @Trenly in microsoft#38172 (comment)

edit: nvm i just remembered i did a manual test before and winget-cli failed 🤦‍♂️ but i think they are looking for a fix too

---

And for example TechPowerUp.NVCleanstall

Include "/files/" in the URL, remove the hash and use winget's user-agent, i.e.:

curl --head --user-agent "winget/xx" https://us2-dl.techpowerup.com/files/NVCleanstall_1.10.0.exe

Originally posted by W1zzardTPU in microsoft#32777 (comment)

[Trenly]

PS C:\WINDOWS\system32> Function Test-Url {
>>     Param
>>     (
>>         [Parameter(Mandatory = $true, Position = 0)]
>>         [string] $URL
>>     )
>>     try {
>>         $HTTP_Request = [System.Net.WebRequest]::Create($URL)
>>         $HTTP_Request.Headers.Add("User-Agent", "Microsoft-Delivery-Optimization/10.1")
>>         $HTTP_Response = $HTTP_Request.GetResponse()
>>         $HTTP_Status = [int]$HTTP_Response.StatusCode
>>     } catch {
>>         # Take no action here; If there is an exception, we will treat it like a 404
>>         $HTTP_Status = 404
>>     }
>>     If ($null -eq $HTTP_Response) { $HTTP_Status = 404 }
>>     Else { $HTTP_Response.Close() }
>>
>>     return $HTTP_Status
>> }
>>
PS C:\WINDOWS\system32> Test-Url 'https://dl.genymotion.com/releases/genymotion-3.2.1/genymotion-3.2.1.exe'
404

@SpecterShell - It's throwing an error when trying to set the user agent with your method, resulting in a 404 still

---

Also, when trying to run manually -

PS C:\WINDOWS\system32> $HTTP_Request = [System.Net.WebRequest]::Create($URL)
PS C:\WINDOWS\system32> $HTTP_Request.Headers.Add("User-Agent", "Microsoft-Delivery-Optimization/10.1")
Exception calling "Add" with "2" argument(s): "The 'User-Agent' header must be modified using the appropriate property
or method.
Parameter name: name"
At line:1 char:1
+ $HTTP_Request.Headers.Add("User-Agent", "Microsoft-Delivery-Optimizat ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException

[Trenly]

Figured it out; It's $HTTP_Request.UserAgent = 'Microsoft-Delivery-Optimization/10.1'

[OfficialEsco]

Looks like this only solved the Genymotion download issue :(

[vedantmgoyal9]

I guess we should add it as a parameter. If default UA doesn't work, then suggest user to enter a UA using a parameter.

[Trenly]

I guess we should add it as a parameter. If default UA doesn't work, then suggest user to enter a UA using a parameter.

No.
If it doesn’t work using a default agent, then it won’t pass the pipelines anyways.

Also, it isn’t good to have everything as a parameter. If you have more than 5-10 parameters for a single command then there is a good chance that sonething needs to be redesigned.