Parallel Queries and Temporary Query Result Error

[ovflowd]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

Our Gatsby application suffers from crashes (100% 10/10) when running gatsby build due to "Parallel Queries" crashing without saying which query is crashing.

And during development runtimes, randomly, when opening different pages, it just crashes most of the time with Error: Couldn't find temp query result for "/en/blog/releases/v16.6.2". (Or similar, with the difference of other page titles).

Note that by removing the newly added content/blog/releases the crash from gatsby build disappears, but the random temp query result issue persists. I have no idea which file is causing it to crash. All those files abide by MDXv2 standards. (Even tho the App uses MDXv1)

Reproduction Link

https://github.com/ovflowd/nodejs.dev/tree/test/gatsby-crashes

Steps to Reproduce

1. Clone, checkout
2. Run gatsby build
3. Crashes

OR

1. Clone, checkout
2. Run gatsby develop
3. Navigate to different blog posts
4. Crashes

Expected Result

1. To not crash?

Actual Result

1. It crashes.

Environment

  System:
    OS: macOS 12.5.1
    CPU: (10) arm64 Apple M1 Max
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.9.0 - /var/folders/kr/dtpb07wj0cl0vp56klz_xc440000gq/T/yarn--1663025110592-0.9226765595115247/node
    Yarn: 1.22.19 - /var/folders/kr/dtpb07wj0cl0vp56klz_xc440000gq/T/yarn--1663025110592-0.9226765595115247/yarn
    npm: 8.19.1 - /opt/homebrew/bin/npm
  Browsers:
    Chrome: 105.0.5195.102
    Firefox: 103.0.2
    Safari: 15.6.1
  npmPackages:
    gatsby: ^4.22.1 => 4.22.1 
    gatsby-link: ^4.22.0 => 4.22.0 
    gatsby-plugin-canonical-urls: ^4.22.0 => 4.22.0 
    gatsby-plugin-catch-links: ^4.22.0 => 4.22.0 
    gatsby-plugin-manifest: ^4.22.0 => 4.22.0 
    gatsby-plugin-mdx: ^3.20.0 => 3.20.0 
    gatsby-plugin-meta-redirect: ^1.1.1 => 1.1.1 
    gatsby-plugin-offline-next: ^5.2.3 => 5.2.3 
    gatsby-plugin-sass: ^5.22.0 => 5.22.0 
    gatsby-plugin-sitemap: ^5.22.0 => 5.22.0 
    gatsby-plugin-svgr: ^3.0.0-beta.0 => 3.0.0-beta.0 
    gatsby-plugin-typescript: ^4.22.0 => 4.22.0 
    gatsby-plugin-webfonts: ^2.2.2 => 2.2.2 
    gatsby-remark-autolink-headers: ^5.22.0 => 5.22.0 
    gatsby-remark-prismjs: ^6.22.0 => 6.22.0 
    gatsby-source-filesystem: ^4.22.0 => 4.22.0 
    gatsby-theme-i18n: ^3.0.0 => 3.0.0 
    gatsby-transformer-yaml: ^4.22.0 => 4.22.0 

Config Flags

FAST_DEV: true

[ovflowd]

Hey @LekoArts, sorry for pinging you, but this is causing disruption for the Node.js org. Could you maybe give an 👀 here on your earlier convenience? Thanks 🙇

[LekoArts]

Please provide a minimal reproduction and not the whole project. This has too many variables and things that might cause this (outside of Gatsby's reach). Only if you can't reproduce it at all in a more minimal project we'll deal with the whole project.

[ovflowd]

Will try to do, ty!

[ovflowd]

Note.: It only happens on build, when just running dev server, all pages work nicely.

[graysonhicks]

Hi @ovflowd ! I gave this a run locally and believe the MDX v2 syntax in the content/blog/releases folder is causing the issue. I ran the posts through an MDX validator and found the following syntax errors:

• <br> tag wasn't closed (replaced in all files with <br />
• the use of <autolink> tags, which aren't supported in v1 is found here and here and here and here and here and here and here and here and here
• several to bots in this one
• unescaped ( here and here
• unescaped { here and here and here and here and here and here
• remove escape and add closing /> for td here
• replace escaped angle brackets with < and &gt for Typescript type (or wrap in string completely) here and here and here and here

After fixing those locally, my build succeeds. Long-term, you'll need to ensure that your MD is outputted as compliant with MDX V1, or, upgrade your Gatsby MDX setup to V2 (but that has backwards breaking changes for the MD files not in the release dir). I've got the fixed files in my fork here.

[ovflowd]

Hey, @graysonhicks thank you so much for actually spending time on this and doing such a deep analysis, this is spectacular! Regarding making all the Markdown files MDX v1 compliant. At least on the main branch, (not sure if in the linked branch too) we already use MDX v1 validation here that comes from here (Ignore the comment there it is outdated).

So in theory all Markdown files should already be MDX v1 compliant. Is there any way we could automatically fix these issues? As we're planning into migrating all the blog posts from nodejs.org to there, this sounds like a massive work 😅

Not to mention, more API docs are going to be moved, as the script right now only generates v18 docs, but if I modify the script to also add the v16 docs, the build also crashes. It is kinda tricky to have MDX just right-away crashing when it finds non-compliant stuff. Other Markdown parsers don't do that... I do understand we should be compliant, and if there's an easier way of doing so, I'm down, and I realise now this is not a Gatsby specific issue but MDX one...

I'm definitely open for tips/advice :)

[graysonhicks]

No problem @ovflowd . I only tested on the test/gatsby-crash branch. How are the MD files in releases being generated? That's likely where a change needs to happen, maybe something changed to start exporting the files in MDX v2 format.

[ovflowd]

How are the MD files in releases being generated?

They're not being generated. Basically "copy/pasted" from here. Then we run a script that modifies the frontmatter to match the ones from our blog. Also, we run our lint command to check they respect all rules; otherwise, we run --fix, but in my experience, it doesn't necessarily fix all things.

What is generated are our API docs through this script. And those are heavily modified through a few regexe's to be MDX compliant, or at least as much as possible. The lint command also doesn't give any error to them.

[ovflowd]

maybe something changed to start exporting the files in MDX v2 format.

They're not being exported on MDXv2. We never attempted to go through the MDXv2 format. (I tried, and checked with linting to MDXv2 how much change would be needed, and ehhh, 1k+ errors straight of the bat, the TL;DR is here: #36519 (comment))

[ovflowd]

@graysonhicks is there any way I can debug which file threw a parallel query error?

[graysonhicks]

@ovflowd Thanks for pointing to that larger discussion, I think I found and fixed a lot of the errors you had already seen 😅 . Interestingly, you note in that thread that you know there is syntax breaking MDXv2, but it appears to be breaking in MDXv1 based on my testing today as well. Ideally you would have all the content be fixed, as Gatsby expects valid MD(X) files. I agree there could be better error handling on the particular file. I ran them through a compiler here. Did anything change with how release MD was written? You may want to look at an MDX linter since you are using gatsby-plugin-mdx on MD files and maybe that's why I saw some errors in the compiler that you weren't seeing with your linter?

[graysonhicks]

Confirmed several of the issues I found were syntax errors introduced in MDXv2, so I'm not sure why those would be failing on MDXv1:

https://mdxjs.com/docs/troubleshooting-mdx/#unexpected-end-of-file-in-expression-expected-a-corresponding-closing-brace-for-

[ovflowd]

. Did anything change with how release MD was written?

Nope. As I mentioned, the mdx parser for eslint gives 0 errors about any of those files... (Or if they did, I think I fixed them). But it is honestly a bummer that Gatsby itself crashes (Worker exits unsuccessfully) without even pointing out the error or where. It is tough to debug like that. Because this means any new MDX file could potentially break the whole App.

And sadly, we cannot migrate to MDXv2 as stated in the thread, or at least gatsby-plugin-mdx@4 because it has to break changes that are just entirely incompatible with, for example, i18n.

[ovflowd]

Note.: The way how we do i18n right now is every page makes the query of the same MDX document in the selected language and the current language. If a file with the current language is not found, we fall back to the default language file. With the MDX files being assigned directly on gatsby-node I find it really hard to actually create the pages in all languages. It would require me to make even more patches on the source of gatsby-theme-i18n. Or just abandon that plugin at all and make our own.

[LekoArts]

If I understand it correctly then, we can close this issue and move further discussion to "Discussions" as the root of the issue was invalid MDX? Our workers should throw more descriptive errors, if we can have a reproduction of a swallowed error (showing the above error, not the original one) we can discuss that in a new issue.

For your other questions you can use GitHub Discussions or Discord, please note though that we're a small team and there's no guarantee to always get a promptly answer as such we partially rely on community members helping each other.

[ovflowd]

as the root of the issue was invalid MDX

I don't think so, again, MDX parser and linter doesn't complain about the contents I added.

Our workers should throw more descriptive errors,

But they don't. They gave nothing, and I'm not exaggerating. Here's an example:

success Building HTML renderer - 19.096s
success Execute page configs - 0.044s
success Caching Webpack compilations - 0.001s

 ERROR #85928 

An error occurred during parallel query running.
Go here for troubleshooting tips: https://gatsby.dev/pqr-feedback



  Error: Worker exited before finishing task
  
  - index.js:117 ChildProcess.<anonymous>
    [nodejs.dev]/[gatsby-worker]/dist/index.js:117:45
  
  - node:events:513 ChildProcess.emit
    node:events:513:28
  
  - child_process:291 Process.ChildProcess._handle.onexit
    node:internal/child_process:291:12
  

not finished run queries in workers - 0.177s

please note though that we're a small team and there's no guarantee to always get a promptly answer as such we partially rely on community members helping each other.

I understand that, and I appreciate the help. I'm just honestly out of options 😅 (As I honestly have no idea what to do with this error)

[graysonhicks]

So I was still unable to find a local linting solution that showed the same errors I was able to find by running through the MDX compiler. But, I reintroduced all of the syntax errors and bumped these dependencies:

    "eslint-mdx": "^2.0.0"
    "eslint-plugin-mdx": "^2.0.0"

It then builds successfully for me. Can you try that?

[ovflowd]

@graysonhicks I don't thibk that using eslint-mdx and eslint-plugin-mdx v2 would make sense for a project using mdx v1

[ovflowd]

It then builds successfully for me.

What builds successfully? (It is important to mention that the test-branch I created is quite outdated already, I would need to create another reproduction branch)

[graysonhicks]

The test/gatsby-crashes branch builds for me successfully with that change (or with the syntax changes).

[ovflowd]

How could an eslint-plugin change make the builds pass? Am I missing something? 😅

[graysonhicks]

I don't know, I'm just letting you know what can unblock you.

[ovflowd]

Gotcha, sorry for the late reply. I'm going to do what you did then, and see what happens. 🙏

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[ovflowd]

Let's close this for now, as there are no actual paths for fixing any of these issues.