Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Absolute Windows paths are handled as invalid URL path in ESM import() #34765

Closed
StarpTech opened this issue Aug 13, 2020 · 15 comments
Closed

Absolute Windows paths are handled as invalid URL path in ESM import() #34765

StarpTech opened this issue Aug 13, 2020 · 15 comments
Labels
esm Issues and PRs related to the ECMAScript Modules implementation. feature request Issues that request new features to be added to Node.js.

Comments

@StarpTech
Copy link

  • Version: v14.8v0
  • Platform: Windows
  • Subsystem:

Related: #31710

What steps will reproduce the bug?

Windows paths are interpreted as invalid file URL. In Linux and Mac, the issue can't be reproduced.

Fail

await import("D:\repositories\fastify-autoload\routes\hello.mjs")

Works

await import(url.pathToFileURL("D:\repositories\fastify-autoload\routes\hello.mjs").href)

Root cause

#31710 (comment)

What is the expected behavior?

Interpret windows paths as valid URL paths so we can provide consistent cross-platform behaviour.

What do you see instead?

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:698:11)
    at Loader.resolve (internal/modules/esm/loader.js:82:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:226:28)
    at Loader.import (internal/modules/esm/loader.js:161:28)
    at importModuleDynamically (internal/modules/cjs/loader.js:1144:27)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:27:14)
    at loadPlugin (D:\repositories\fastify-autoload\index.js:118:5)
    at D:\repositories\fastify-autoload\index.js:28:12
    at Array.map (<anonymous>)
    at fastifyAutoload (D:\repositories\fastify-autoload\index.js:27:29) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

@mcollina

@jasnell jasnell added esm Issues and PRs related to the ECMAScript Modules implementation. feature request Issues that request new features to be added to Node.js. labels Aug 13, 2020
@mcollina
Copy link
Member

cc @nodejs/modules

@jkrems
Copy link
Contributor

jkrems commented Aug 13, 2020

This behavior isn't restricted to windows paths, the same is true for absolute Unix paths as well. The module import specifiers only support URLs, not file paths. To import file paths, they need to be converted to URLs first on all platforms (like you mentioned).

The general problem with supporting both URLs and file paths is that some URLs are also valid (but different!) file paths and vice versa. We decided to go with URLs over file paths to be consistent with other runtimes (e.g. browsers) that are also using URL-based import specifiers.

@StarpTech
Copy link
Author

StarpTech commented Aug 14, 2020

Do you mean that /mnt/c/Users/starptech/test.mjs is a valid URL and file path in Linux and MAC? or what's the reason that it doesn't fail on all OS?

@jkrems
Copy link
Contributor

jkrems commented Aug 14, 2020

@StarpTech Yes, /mnt/c/Users/starptech/test.mjs is both a valid absolute POSIX file path and a valid host-relative file: URL. If resolved from file:///any/other/path, the host-relative URL happens to resolve to the same file. But this is coincidence, not guaranteed. Other examples:

  • /mnt/c/Users/starptech/my%20test.mjs:

    • POSIX file path: Valid! It's a file that has a literal percentage sign followed by a "20" in its path.
    • Host-relative URL: Valid! But the filename has a space in it, not a percentage sign. It's a different file than the posix file path interpretation.
  • /mnt/c/Users/starptech/%A.mjs:

    • POSIX file path: Valid! It's a file that has a literal percentage sign followed by an "A" in its path.
    • Host-relative URL: Not valid! Or rather, will fail while trying to convert it to a filename.

Not to mention that host-relative URLs are host-relative. Which means that even the "valid" cases that only use alphanumerical characters with slashes (the characters that happen to be interpreted the same in posix paths and file: URLs) will still only work when imported from specific modules. Example:

// Works!
import 'file:///mnt/c/Users/starptech/test.mjs';

// Fails, sometimes!
import '/mnt/c/Users/starptech/test.mjs';

The second import would fail if the module above was loaded from a data: URL. Or really: From any URL that isn't a local file URL. Unless the author really knows what they're doing, I would highly recommend including the protocol for file URLs and not flipping that coin.

@StarpTech
Copy link
Author

If it can't be guaranteed, from a design perspective the API is prone to errors. Why we don't enforce the file: protocol for absolute paths?

@bmeck
Copy link
Member

bmeck commented Aug 14, 2020

@StarpTech i don't understand what you want to enforce, can you clarify? right now https://url.spec.whatwg.org/#relative-url-string is what is being followed to https://url.spec.whatwg.org/#path-absolute-url-string .

@MylesBorins
Copy link
Contributor

One other option we can consider here would be to improve the error message in the case of an absolute windows path.

@mcollina
Copy link
Member

The problem here is that I had absolutely not clue on why some code I wrote was not working on windows as the code was working smoothly on Linux and Mac. The developer experience for this was far from great, because the only way to debug this would be to have a Windows machine

One other option we can consider here would be to improve the error message in the case of an absolute windows path.

I think this would be the bare minimum change that improve the developer experience.

@bmeck
Copy link
Member

bmeck commented Aug 14, 2020

We could also just make the paths not work on Linux or Mac. That seems odd though and would remove some URL support.

@MylesBorins
Copy link
Contributor

I think improving the error message so it is clear is a good first step. If I am mistaken it seems like the issue is manually creating a URL... is the exact same code working on OSX and failing on windows?

@StarpTech
Copy link
Author

StarpTech commented Aug 14, 2020

@bmeck forget it. I think logging a warning message in case of an absolute URL without a special scheme e.g file:// would be a good hint.

If I am mistaken it seems like the issue is manually creating a URL... is the exact same code working on OSX and failing on windows?

@MylesBorins Yes, it works by the use of url.pathToFileURL.

@devsnek
Copy link
Member

devsnek commented Aug 14, 2020

What I think would be helpful is adding a check in the ERR_UNSUPPORTED_ESM_URL_SCHEME constructor which checks if the url's scheme is a single letter and the system is on windows, and then offering that you probably want to prefix it with file://

@DerekNonGeneric
Copy link
Contributor

/cc @weswigham

Trott pushed a commit that referenced this issue Aug 19, 2020
Refs: #34765

PR-URL: #34795
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
BethGriggs pushed a commit that referenced this issue Aug 20, 2020
Refs: #34765

PR-URL: #34795
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
BethGriggs pushed a commit that referenced this issue Aug 20, 2020
Refs: #34765

PR-URL: #34795
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
guybedford pushed a commit to guybedford/node that referenced this issue Sep 28, 2020
Refs: nodejs#34765

PR-URL: nodejs#34795
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
codebytere pushed a commit that referenced this issue Oct 1, 2020
Refs: #34765

PR-URL: #34795
Backport-PR-URL: #35385
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
@jasnell
Copy link
Member

jasnell commented Jan 5, 2021

I believe this is resolved since #34795 landed

@tinwaninja
Copy link

@bmeck forget it. I think logging a warning message in case of an absolute URL without a special scheme e.g file:// would be a good hint.

If I am mistaken it seems like the issue is manually creating a URL... is the exact same code working on OSX and failing on windows?

@MylesBorins Yes, it works by the use of url.pathToFileURL.

thank you so much

iiroj added a commit to lint-staged/lint-staged that referenced this issue Nov 21, 2021
iiroj added a commit to lint-staged/lint-staged that referenced this issue Nov 22, 2021
saibotsivad added a commit to tobiaslabs/hunch that referenced this issue Jun 29, 2023
luwes added a commit to luwes/next-video that referenced this issue Feb 1, 2024
luwes added a commit to luwes/next-video that referenced this issue Feb 1, 2024
luwes added a commit to muxinc/next-video that referenced this issue Feb 2, 2024
* chore: add windows ci test

* fix: dynamic import file path in Windows

* fix: try forward slashes

* chore: log out file path

* fix: pathToFileURL().href
related issue nodejs/node#34765

* chore: log out error

* fix: try adding cwd()

* fix: cross platform write file script
related shelljs/shx#192

* fix: try next-video/process import

* fix: cross platform glob expansion

* chore: use codecov v4

* chore: move next.config.js mock test file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
esm Issues and PRs related to the ECMAScript Modules implementation. feature request Issues that request new features to be added to Node.js.
Projects
None yet
Development

No branches or pull requests

9 participants