Support URL specific encodings (query component)

[jeff-woehler]

Hello Support,

When clicking a link in VSCode, the URL is incorrectly unescaped prior to handing the request to the browser.

Example URL pasted into VSCode:
http://www.test.com/path/service?authId=CN%3DQ10

Result - the '%3D' is unescaped to '=' when you Ctrl-Click:
GET /path/service?authId=CN=Q10 HTTP/1.1

Please review and correct.

Regards,
Jeff Woehler

• VSCode Version: Code 1.11.2 (6eaebe3, 2017-04-13T08:03:11.395Z)
• OS Version: Windows_NT ia32 6.1.7601
• Extensions:

Extension	Author	Version
xml	DotJoshJohnson	1.7.0
beautify	HookyQR	1.0.2
csharp	ms-vscode	1.8.1
debugger-for-chrome	msjsdiag	3.0.1

---

Steps to Reproduce:

[humphd]

If you decode %2F to / at this point, it's now part of the path and not data in the path, and there's nothing you can do later with toString() that will allow you to recover it.

Anyway, just trying to help. Sounds like that's not wanted, so I'll leave this to you.

[kaleidawave]

This issue is still present in 1.31.0 and have had this issue many times with the '%2F' string being changed to '/' when forwarded to Google Chrome. This is a big issue for firebase storage as it cannot locate the resource if there is a forward slash. In the mean time is there a way of circumventing this by changing the contents of the logging function.

[StephenWeatherford]

Johannes - We see you merged a change related to this yesterday. Unfortunately, it does not fix the issue for us. This is an actual URL generated by our extension. It should open a new issue in github:

ISSUE 1:
https://github.com/Microsoft/vscode-azurearmtools/issues/new?body=%0A%3C!--%20IMPORTANT%3A%20Please%20be%20sure%20to%20remove%20any%20private%20information%20before%20submitting.%20--%3E%0A%0ARepro%20steps%3A%0A%3C!--%20TODO%3A%20Enter%20steps%20to%20reproduce%20issue%20--%3E%0A%0A1.%0A2.%0A%0AAction%3A%20startArmLanguageClient%0AError%20type%3A%20Error%0AError%20Message%3A%20hi%0A%0AVersion%3A%200.5.1-alpha%0AOS%3A%20win32%0AProduct%3A%20Visual%20Studio%20Code%20-%20Insiders%0AProduct%20Version%3A%201.34.0-insider%0ALanguage%3A%20en%0A%0A%3Cdetails%3E%0A%3Csummary%3ECall%20Stack%3C%2Fsummary%3E%0A%0A%60%60%60%0AObject.vscode_azureextensionui_1.callWithTelemetryAndErrorHandlingSync%20(startArmLanguageServer.js%3A17%3A15)%0AObject.callWithTelemetryAndErrorHandlingSync%20(node_modules%2Fvscode-azureextensionui%2Fout%2Fsrc%2FcallWithTelemetryAndErrorHandling.js%3A46%3A25)%0AObject.startArmLanguageServer%20(startArmLanguageServer.js%3A16%3A31)%0AAzureRMTools.ensureDeploymentTemplateEventsHookedUp%20(out%2Fsrc%2FAzureRMTools.js%3A226%3A34)%0AObject.%5Banonymous%20(out%2Fsrc%2FAzureRMTools.js%3A128%3A24)%0AObject.callWithTelemetryAndErrorHandlingSync%20(node_modules%2Fvscode-azureextensionui%2Fout%2Fsrc%2FcallWithTelemetryAndErrorHandling.js%3A46%3A25)%0AAzureRMTools.updateDeploymentTemplate%20(out%2Fsrc%2FAzureRMTools.js%3A110%3A35)%0Anew%20AzureRMTools%20(out%2Fsrc%2FAzureRMTools.js%3A88%3A18)%0AObject.%5Banonymous%5D%20(out%2Fsrc%2FAzureRMTools.js%3A45%3A44)%0AGenerator.next%20(%5Banonymous%5D)%0A__awaiter%20(out%2Fsrc%2FAzureRMTools.js%3A10%3A71)%0Anew%20Promise%20(%5Banonymous%5D)%0A__awaiter%20(out%2Fsrc%2FAzureRMTools.js%3A6%3A12)%0AObject.%5Banonymous%5D%20(out%2Fsrc%2FAzureRMTools.js%3A41%3A20)%0AObject.%5Banonymous%5D%20(node_modules%2Fvscode-azureextensionui%2Fout%2Fsrc%2FcallWithTelemetryAndErrorHandling.js%3A61%3A51)%0AGenerator.next%20(%5Banonymous%5D)%0A__awaiter%20(node_modules%2Fvscode-azureextensionui%2Fout%2Fsrc%2FcallWithTelemetryAndErrorHandling.js%3A11%3A71)%0Anew%20Promise%20(%5Banonymous%5D)%0A__awaiter%20(node_modules%2Fvscode-azureextensionui%2Fout%2Fsrc%2FcallWithTelemetryAndErrorHandling.js%3A7%3A12)%0AObject.callWithTelemetryAndErrorHandling%20(node_modules%2Fvscode-azureextensionui%2Fout%2Fsrc%2FcallWithTelemetryAndErrorHandling.js%3A58%3A12)%0AObject.%5Banonymous%5D%20(out%2Fsrc%2FAzureRMTools.js%3A40%3A41)%0AGenerator.next%20(%5Banonymous%5D)%0A__awaiter%20(out%2Fsrc%2FAzureRMTools.js%3A10%3A71)%0Anew%20Promise%20(%5Banonymous%5D)%0A__awaiter%20(out%2Fsrc%2FAzureRMTools.js%3A6%3A12)%0AObject.activateInternal%20(out%2Fsrc%2FAzureRMTools.js%3A34%3A12)%0Aactivate%20(main.js%3A24%3A28)%0AFunction._callActivateOptional%20(AppData%2FLocal%2FPrograms%2FMicrosoft%20VS%20Code%20extensionHostProcess.js%3A725%3A689)%0AFunction._callActivate%20(AppData%2FLocal%2FPrograms%2FMicrosoft%20VS%20Code%20extensionHostProcess.js%3A725%3A378)%0Adefine._doActivateExtension.Promise.all.then.e%20(AppData%2FLocal%2FPrograms%2FMicrosoft%20VS%20Code%20extensionHostProcess.js%3A724%3A569)%0A%60%60%60%0A%0A%3C%2Fdetails%3E

Expected result: Working New Issue page
Actual result: Blank page or error 500

Opening this through the "opn" npm module works fine. But if you send this through vscode.open, or if you just enter it into a vscode editor and do CTRL+CLICK, it brings up a blank page in Edge and shows a 500 error in Chrome.

ISSUE 2:
The following shortened URL (without the stack) does bring up the New Issues page, but the newlines are missing:

https://github.com/Microsoft/vscode-azurearmtools/issues/new?body=%0A%3C!--%20IMPORTANT%3A%20Please%20be%20sure%20to%20remove%20any%20private%20information%20before%20submitting.%20--%3E%0A%0ARepro%20steps%3A%0A%3C!--%20TODO%3A%20Enter%20steps%20to%20reproduce%20issue%20--%3E%0A%0A1.%0A2.%0A%0AAction%3A%20startArmLanguageClient%0AError%20type%3A%20Error%0AError%20Message%3A%20hi%0A%0AVersion%3A%200.5.1-alpha%0AOS%3A%20win32%0AProduct%3A%20Visual%20Studio%20Code%20-%20Insiders%0AProduct%20Version%3A%201.34.0-insider%0ALanguage%3A%20en%0A%0A%3Cdetails%3E%0A%3Csummary%3ECall%20

Expected result:

Actual result:

[jrieken]

@StephenWeatherford Both links work for me with latest insiders. With what version did you try?

[jrieken]

URL query string reference: https://en.wikipedia.org/wiki/Query_string

[jrieken]

done via #73849 - schemes that use query strings (http, https, and ftp) now won't encode &, =, and ; when occurring inside the query-component.

[jrieken]

This needed to re-opened because of regressions caused by this. This issue actually talks about two problems

1. Decoding things that shouldn't be decoded, e.g. %3D inside the query component, tracked in Uri#parse can break path-component #45515
2. Encoding things that shouldn't be encoded because scheme-specific rules exists, e.g. the generic URI syntax doesn't define query strings but in http/ftp it's a thing. Even tho our URI implementation ignores most of these scheme specific encodings we do a much better job now of picking the right encoding when URIs "leave" VS Code, e.g when opening an external browser.

[generalov]

There is a similar issue with a fragment component. Browser navigates to a wrong URL if the fragment contains = character:

const url = "https://tfs.server/tfs/collection/_versionControl#path=%24%2Fproject%2Findex.js"
const oUri = vscode.Uri.parse(uri)
//oUri._formatted = url  // uncomment to apply the workaround
vscode.commands.executeCommand("vscode.open", oUri)

Actual: https://tfs.server/tfs/collection/_versionControl#path%3D%24%2Fproject%2Findex.js
Expected: https://tfs.server/tfs/collection/_versionControl#path=%24%2Fproject%2Findex.js

oUri.toString() produces the actual result. It seems the escaping of = character is intended as the unit test expects:

vscode/src/vs/base/test/common/uri.test.ts

Line 66 in 2ea7d60

assert.equal(URI.from({ scheme: 'http', authority: 'a-test-site.com', path: '/', query: 'test=true' }).toString(), 'http://a-test-site.com/?test%3Dtrue');

However, such behaviour unexpectedly changes a valid URL and does not allow a TFS server to show the path.

I found a workaround for the issue (and I believe this fixes the escaping issues for all URL components) but it looks a bit awful. :)

[928PJY]

Hi @jrieken, is that any update about this issue? is that any workaround about this issue? I notice that you have removed the Bug tag, so just want to know will you take this as a high priority problem or not?

[jrieken]

Good timing @928PJY - the issue is very complex but I happen to be looking at this. My current work and progress is in #83060. No promises but the problem is acknowledged and understood but also far from easy to fix.

[jrieken]

Implemented via #83060

[jrieken]

Verifier: Have links like https://github.com/Microsoft/vscode/issues?q=is%3Aopen+assignee%3Ajrieken+milestone%3A%22October+2019%22 and make sure they can be opened as-is, e.g no double encoding etc

[jrieken]

re-open via 58479e8

[jrieken]

/duplicate of #83645

Closing this as a duplicate of #83645 which is all about finding a way out of the URI nightmare.

[vscodebot]

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for existing issues here. See also our issue reporting guidelines.

Happy Coding!