fix(gatsby): ensure that writing node manifests to disk does not break on Windows

[veryspry]

Description

Writing node manifest files is failing on Windows and this provides a fix for that.

Related Issues

Resolves #33743

[KyleAMathews]

could we use joinPath here? https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-core-utils/src/path.ts

[TylerBarnes]

It doesn't look like it handles :'s

[veryspry]

I'm down but I'm curious, what does this quadruple backslash do for us?

[veryspry]

Oh yeah, also it won't solve this issue on its own (doesn't handle : like @TylerBarnes said). But if it gives us some extra guardrails for other scenarios using it makes sense

[TylerBarnes]

Ah, got it, @KyleAMathews I thought you meant can we use it to solve this problem. I see you mean for extra edge-case guards like @veryspry is saying. Makes sense!

[KyleAMathews]

sorry I should have been more direct — this is the utility we use all over core for creating cross-os paths (https://github.com/gatsbyjs/gatsby/search?q=joinPath) — I forget its whole history but it should be pretty battle-tested now.

I see now better what you're solving e.g. that some manifest IDs can have reserved characters in them. Wouldn't we always need to replace those? There's some others we'd need to handle https://superuser.com/questions/358855/what-characters-are-safe-in-cross-platform-file-names-for-linux-windows-and-os

[veryspry]

Gotcha! I'll make that change

Yes, we probably do need to handle more characters other than the colon. : is probably the most pressing since a node manifest file name will likely be created using a timestamp but I'll look into handling other cases as well,

[veryspry]

I made the change to use joinPath like you suggested.

Also added some more chars to the replace regex to account for other special chars on Windows

[KyleAMathews]

[TylerBarnes]

Heck yes! 💪

[TylerBarnes]

Looks like the processNodeManifest tests need to be updated to pass on windows

[TylerBarnes]

I wonder if we shouldn't use joinPath here 🤔 the result in the tests seems like too many backslashes

Expected: "C:\\Users\\circleci\\project\\public\\__node-manifests\\test\\1.json", ObjectContaining {"page": {"path": "/1"}}
Received
->     1: "C:\\\\Users\\\\circleci\\\\project\\\\public\\\\__node-manifests\\\\test\\\\1.json"

I might be missing something though

[veryspry]

I tested on a Windows machine with this canary and node manifests were written correctly to disk

[email protected]+cd4571cfd3

[TylerBarnes]

Sweet 😄 I guess we just need to use joinPath in our unit tests then

[veryspry]

I tested on a Windows machine with this canary and node manifests were written correctly to disk

[email protected]+cd4571cfd3

Oh NVM, I did not confirm that conclusively. Looks like there was a webpack error during build time that must have prevented the files from getting written to disk? The log saying that the files were successfully written running though 🤔

I need to head out for the night but I will dig in some more in the morning

[veryspry]

I wonder if we shouldn't use joinPath here thinking the result in the tests seems like too many backslashes

Expected: "C:\\Users\\circleci\\project\\public\\__node-manifests\\test\\1.json", ObjectContaining {"page": {"path": "/1"}}
Received
->     1: "C:\\\\Users\\\\circleci\\\\project\\\\public\\\\__node-manifests\\\\test\\\\1.json"

I might be missing something though

I'm pretty sure that this is okay since the extra backslashes should be ignored as redundant escape chars. I'm guessing the use of four \ in a row in joinPath is to ensure that the Windows path delimiter is escaped properly inside interpolated strings

[wardpeet]

joinPath is only necessary when you use it in a string, here it's actually useless. I wonder why we don't use @sindresorhus/slugify to remove all special characters.

[veryspry]

joinPath is only necessary when you use it in a string, here it's actually useless. I wonder why we don't use @sindresorhus/slugify to remove all special characters.

@wardpeet Aha, that makes sense. I'm going to revert it then in favor of path.join. I suppose that @sindresorhus/slugify would make sense since it handles characters other than then ones we included in this check. The only question I have is, does this handle backslashes and other characters that are specifically reserved on Windows? It is unclear from the package README if it does

On another note, we really should only be running the replace function on the filename, not the whole path. That way we can also replace any instances of \ before building the the full path

[wardpeet]

Oh yes only on the filename not full path 👍. This more or less removes os specific code