Gatsby, React & Hydration

[eyalroth]

Seems that hydration-related issues are extremely common when developing in GatsbyJS, but the issue has not been fully and formally addressed, so I would like to do so here.

The problem

Due to many Gatsby sites having some dynamic components -- despite being mostly statically generated -- and due to the SSR model behind Gatsby, many Gatsby sites fails to properly update HTML attributes; most notably class, but also href and so forth. This happens when the client and server DOMs differ in their attributes, as the React hydration functionality does not (re)consolidate them as well as the render method:

React expects that the rendered content is identical between the server and the client. It can patch up differences in text content, but you should treat mismatches as bugs and fix them. In development mode, React warns about mismatches during hydration. There are no guarantees that attribute differences will be patched up in case of mismatches. This is important for performance reasons because in most apps, mismatches are rare, and so validating all markup would be prohibitively expensive.

https://reactjs.org/docs/react-dom.html#hydrate

To give a few instances of this problem from this repository, here is a short list: #3741 #5100 #8560 #9911 #10370 #12360 #14601 #17676 .

It is not surprising that the problem is so common. It is often hard to detect and resolve, and for two main reasons:

1. There is a difference between production and development modes in regards to how Gatsby uses hydration (addressed by [feat]: Converge behaviour between develop and build #10706).
2. The problem has vague and indirect side-effects, and it not necessarily easy to debug and reach to the conclusion that it's the root of the situation.

Reproduction

I also created a minimal Gatsby site repository to reproduce the problem: https://github.com/eyalroth/gatsby-hydrate-bug

The site has only 2 pages and a layout. The layout is automatically added to the 2 pages via wrapPageElement (same code as in gatsby-plugin-layout). The layout wraps the page content with a div that has a class attribute set to the current time, while also appending the time as text beneath the page content.

Upon building (and serving) the site, navigating to the index and inspecting it in the developer tools, one can see that the time displayed in the page is not the same as in the div class attribute:

Navigating to the second page will correct this behavior, and you'll see that the time will be the same between the page content and the class attribute:

This will remain so as long as you keep navigating between the pages in the same window. If you happen to refresh the page or open it in a window, the inconsistency will creep back; you'll actually notice that the time in the class attribute will remain the same every time you refresh (hinting that it is cached). "Hard refresh" (CTRL + F5) will load the page properly.

Workarounds

Replacing hydration with rendering

One possible solution to (some instances of) the problem is to entirely replace the hydration with rendering:

const ReactDOM = require('react-dom')

export function replaceHydrateFunction() {
    return (element, container, callback) => {
        ReactDOM.render(element, container, callback)
    }
}

However, there are two major drawbacks to this workaround:

1. It may very well reduce performance, as the whole idea behind the hydration functionality is to improve performance.
2. Directly rendering from the client, instead of hydrating, is deprecated in React 16, and is said to be completely removed in React 17.

Two pass rendering

The other workaround is to re-render a component which is expected to change between the client and the server when it first mounts, and attach a different key attribute to the component the second time around:

class Layout extends React.Component {
  constructor(props) {
    super(props)
    this.state = { isClient: false }
  }
  render() {
    // can be `div` or anything else, I tried to keep this generic
    return <React.Fragment key={this.state.isClient}> 
      {/*...*/}
    </React.Fragment/>
  }
  componentDidMount() {
    this.setState({ isClient: true })
  }
}

This workaround is also documented in React's website:

If you intentionally need to render something different on the server and the client, you can do a two-pass rendering. Components that render something different on the client can read a state variable like this.state.isClient, which you can set to true in componentDidMount(). This way the initial render pass will render the same content as the server, avoiding mismatches, but an additional pass will happen synchronously right after hydration. Note that this approach will make your components slower because they have to render twice, so use it with caution.

The drawback of this approach is that the user may very well visually notice the re-rendering, which wouldn't be the best user experience.

What's next?

I believe this problem should be better highlighted and documented in the Gatsby site. As I said earlier, it is hard to detect and be aware of, especially when (new) developers -are not always aware of the entire design behind the framework, and may very well not be able to tell what's the differences between server-side or client-side rendering, and when the components are expected to differ between the two.

This is most notable with gatsby-plugin-offline, where (I believe) the problem is happening since the default shell page is first generated with all the components in wrapPageElement (but without any page content), so only these components are actually affected by the problem; for instance, a layout component "injected" by gatsby-plugin-layout.

#17676 is reporting that the problem occurs even when the offline plugin is disabled, so I'm not sure whether a builtin workaround in that plugin -- for instance, generating the default shell app without wrapPageElement -- will be enough to solve the problem for all the various use-cases.

Thoughts?

[sidharthachatterjee]

Thank you for the detailed issue, @eyalroth

I think we should do two things here

• Document this better, maybe in https://www.gatsbyjs.org/docs/debugging-html-builds/

Would you like to do this considering you've done all this great work in investigating this? 🤗

for instance, generating the default shell app without wrapPageElement -- will be enough to solve the problem for all the various use-cases

• We could try this, cc @davidbailey00 (who maintains the offline plugin)

[eyalroth]

@sidharthachatterjee

Document this better, maybe in https://www.gatsbyjs.org/docs/debugging-html-builds/

I feel like this should be mentioned in multiple locations, not just in the debugging section, which I believe people tend to get to very late in their development and/or debugging process (and some might not even notice the problems at first).

Of the locations I thought would be appropriate to mention this:

1. Anywhere where hydration is addressed:
   1.a https://www.gatsbyjs.org/docs/glossary/#hydration
   1.a https://www.gatsbyjs.org/docs/client-data-fetching/
   1.a https://www.gatsbyjs.org/docs/production-app/#dom-hydration
2. Layout-related pages:
   2.a https://www.gatsbyjs.org/docs/layout-components/#how-to-prevent-layout-components-from-unmounting
   2.b https://www.gatsbyjs.org/packages/gatsby-plugin-layout/
3. The "wrap" APIs:
   3.a https://www.gatsbyjs.org/docs/browser-apis/#wrapPageElement
   3.b https://www.gatsbyjs.org/docs/browser-apis/#wrapRootElement
   3.c https://www.gatsbyjs.org/docs/ssr-apis/#wrapPageElement
   3.d https://www.gatsbyjs.org/docs/ssr-apis/#wrapRootElement
4. The offline plugin, which seem to be involved in many of the problem's instances: https://www.gatsbyjs.org/packages/gatsby-plugin-offline/

Perhaps this problem deserves a dedicated page / blog post?

Would you like to do this considering you've done all this great work in investigating this? 🤗

I actually mentioned the problem in part of my guide on how to implement a layout in a Gatsby site. It's in my blog, which is not published yet, and will likely go through several more edits. The post is currently online here, where's the source for it is here. I'm not sure it follows the official style guide though 😆

[SMerdzhanov]

I'm quite confused by the last few replies around this so broadly mentioned issue.

Thank you for suggesting these workarounds @eyalroth! I'm actually already testing the hydration-one. However, as you mentioned yourself, this might have a performance impact and in general, is still a "workaround". However, I really appreciate your efforts and will get back to you when I have more feedback from my experiments.

@sidharthachatterjee wouldn't it be better if we come up with a reasonable and grounded explanation of why this issue is happening in the first place? Instead of thinking about how to document a workaround which is still unverified? Did anyone confirm that these proposed solutions fixed the initial issue? What about gatsby-plugin-offline and the assumption it might play a role too?

Does this mean we're accepting a potential performance hit? Is this the solution we're accepting? Just trying to understand what's next here?

[pitkes22]

I have been struggling with this bug for a very long time. Happy to see that it is finally getting some attention. 👍

I have recently run in it when I was using gatsby-plugin-layout with multiple layouts and I was conditional rendering them one at the time but in a production version of the site, there was part of one wrapped in component from the other one which doesn't make any sense.

Also, I have noticed that this behavior varies when I load the page for the first time. Probably because of the gatsby-plugin-offline mentioned above.

One more issue that may be related to this bug: #15993

[eyalroth]

I have recently run in it when I was using gatsby-plugin-layout with multiple layouts and I was conditional rendering them one at the time but in a production version of the site, there was part of one wrapped in component from the other one which doesn't make any sense.

Multiple layouts are actually quite a complicated feature, and may not work hand in hand with the layout plugin. I've written about it here (note that this is still unpublished and may be subjected to edits).

[gillkyle]

This is great research and an awesome writeup! Thanks a bunch @eyalroth 🙂

I'm interested in other's thoughts on this too, since I'm not entirely sure what the nature of a guide like this would look like, maybe a write up similar to what you've done in this issue.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[davidde]

It seems to me that at least a subset of these 'mysterious' issues where gatsby build styles differ from gatsby develop are caused by preprocessor imports.

I've had a similar issue that was caused by variables imported from an .scss stylesheet:

import vars from './layout.scss'

The imported variables are undefined on gatsby build (but work fine on gatsby develop).
This results in obscure bugs that only manifest themselves on first page load in production.

@eyalroth @sidharthachatterjee @SMerdzhanov @LekoArts Can anyone confirm if this is a gatsby issue or a configuration issue?

[davidde]

@eyalroth @sidharthachatterjee @SMerdzhanov @LekoArts

For your convenience, I filed #19563 to track this preprocessor issue.
Since this probably also happens to each theme or plugin that uses preprocessor imports, it might actually be causing more problems than apparent at first glance.

It includes a minimal reproduction that clearly demonstrates this problem:
https://github.com/DavidDeprost/gatsby-build--sass-bug

[eyalroth]

@DavidDeprost Thank you for filing that issue. I think it's more related to #10706, and not so much to this ticket. The problem addressed here seems to be rooted in a behavior which affects both the production and development builds.

[davidde]

You're totally right, it is more related to that issue. I only came to this issue because it was mentioned in some of the other issues that they were closed in favor of this one. I guess I had the mistaken impression this was also the place to gather other possible causes.

Thanks for the issue number, the preprocessor issue will be of more use in there.

[SMerdzhanov]

Thank you for keeping this thread up 👍

Both issues linked #19563 and #10706 seems related, but not quite actionable/active. I'm curious how there aren't more people experiencing the same issue.

In any case, for the moment I've managed to apply a workaround that seems to be a fix for now. However it's a workaround and has some negative performance impact. I'm talking about replacing the hydrate() function as suggested above ^^

So far I haven't found any better solution.

[RuudBurger]

Just wanted to pitch in with a reproducable example.

import React, { useCallback, useEffect, useState } from 'react';

export default () => <div>
    <DoesntWorkComponent />
    <WorksComponent />
</div>

const hasWindow = typeof window !== 'undefined';

const DoesntWorkComponent = () => {
    const [dimensions, setDimensions] = useState({
        innerWidth: hasWindow ? window.innerWidth : 0,
        innerHeight: hasWindow ? window.innerHeight : 0,
    });

    const handleResize = useCallback(() => {
        setDimensions({
            innerWidth: hasWindow ? window.innerWidth : 0,
            innerHeight: hasWindow ? window.innerHeight : 0,
        });
    }, []);

    useEffect(() => {
        handleResize();
        hasWindow && window.addEventListener('resize', handleResize);
        return () => {
            hasWindow && window.removeEventListener('resize', handleResize);
        };
    }, []);

    return <div width={dimensions.innerWidth}
                height={dimensions.innerHeight}
                style={{
                    width: dimensions.innerWidth,
                    backgroundColor: 'gray',
                }}>{dimensions.innerWidth} x {dimensions.innerHeight}</div>;
};

const WorksComponent = () => {
    const [dimensions, setDimensions] = useState({
        innerWidth: 0, // THIS IS THE ONLY DIFFERENCE
        innerHeight: 0, // THIS IS THE ONLY DIFFERENCE
    });

    const handleResize = useCallback(() => {
        setDimensions({
            innerWidth: hasWindow ? window.innerWidth : 0,
            innerHeight: hasWindow ? window.innerHeight : 0,
        });
    }, []);

    useEffect(() => {
        handleResize();
        hasWindow && window.addEventListener('resize', handleResize);
        return () => {
            hasWindow && window.removeEventListener('resize', handleResize);
        };
    }, []);

    return <div width={dimensions.innerWidth}
                height={dimensions.innerHeight}
                style={{
                    width: dimensions.innerWidth,
                    backgroundColor: 'gray',
                }}>{dimensions.innerWidth} x {dimensions.innerHeight}</div>;
};

Note the "THIS IS THE ONLY DIFFERENCE" comment on the 2 lines.
This will render the width x height inside the components correctly. But if you look at the following gif, you will see that the dom (and the styles) don't update correctly.
If I resize the window (height only) it will update the one prop, but not the other. If I resize the window (width only) it will snap to the state I expected in the first place.
Adding hasWindow to the deps array in useEffect or useCallback doesn't change anything.

(sorry for the slow gif, converting it from mov to gif changed the framerate..)

[chandlervdw]

I've run into this issue and was about to pull my hair out. The only solution (janky) I've found was here. This is obviously less than ideal but I'm not sure how to help debug this further.

[wahlforss]

This is a constant issue in my experience. It constantly pops up and is a total pain. This happened while using Formik

[eyalroth]

@blainekasten This is not a develop-only problem, it's affecting production as well.

[SMerdzhanov]

Hey @blainekasten, I'm a bit confused maybe.

Issue #25729 is about running true SSR in development mode.

If I understand correctly, that's an attempt to be able to see and eventually prevent the issue from happening by pointing it out while developing.

But this isn't actually going to solve it, right?

So then #25729 would be helpful but it seems not like a solution but patch? Don't we want to keep this thread open and actually resolve the issue?

[verekia]

If I'm not mistaken, the fact that SSR & client mismatches mess up the DOM is not a Gatsby problem it's a React problem, and it's expected to happen. It doesn't matter if we are in development or production mode, these issues in your code must be fixed before shipping to production. If we have SSR in development, we will be able to fix those issues and there shouldn't be mismatches in production. In my opinion I don't think there is anything to "fix" in Gatsby, except being able to see those React warnings when developing. It's the same as missing key attributes, you are supposed to catch those when developing, and it's expected to break your app if you don't. But maybe there is an other scenario I'm not seeing.

[eyalroth]

@verekia There are no fixes, only partial workarounds.

This might be a problem stemming from React, but dismissing it as a non-Gatsby problem for this reason is akin to dismissing a food poisoning in your restaurant due to bad handling by the food supplier. At the end of the day, no one will come eat at your restaurant.

This issue might have been dismissed if it didn't affect anyone, or affected only a niche use-case, but this is one of the highest voted issues in this repository. This is no surprising, because it is a big problem with one of the most fundamental features / characteristics that Gatsby is aiming to provide -- fast (static) yet flexible (dynamic) web pages.

[polarathene]

While it won't affect many users, those who use art direction feature of gatsby-image have experienced issues from hydration since the SSR can only bake in the state and CSS for a particular breakpoint.

React hydrates to the SSR state assuming it's the same instead of re-rendering if not, but due to the nature of that problem, you'd also have breakage prior to JS being available, so that needed to be addressed to. I created a PR that resolves all that, the hydration issue is a pretty simple fix toggling a bool state as the official React docs advise.

---

I'd prefer the benefits of gatsby develop and wouldn't want to lose those for serving SSR in develop mode personally, you're just trading one set of issues for another. Instead it's best just to provide solid documentation on the "phenomena", so that confused users can understand the issue and what the common avenues for resolving it are, just providing SSR during develop doesn't magically solve the problem, only make it visible sooner while potentially hiding other information worth knowing.

Developers can get an SSR update via gatsby build followed by gatsby serve and visiting localhost:9000. Might seem like a hassle, but it's probably better than some fast iteration developer mode that rebuilds/refreshes SSR output with file changes, since that'd be leveraging cache which can hide/cause other problems. Better practice to do a fresh build & serve before deploying to production/staging.

[verekia]

Well if they do implement SSR in development mode, it's pretty easy to put it behind a flag.
gatsby develop for fast builds without SSR
gatsby develop --ssr for slower builds with SSR

Developers can get an SSR update via gatsby build followed by gatsby serve and visiting localhost:9000

@polarathene That's the point, this is currently not possible because warnings are not shown in production mode. The app is just completely messed up and there is no information about what's happening. "Why does my header look like a button?! Why is my title sticking at the bottom of the screen?!" It is extremely confusing. And even if you are aware of hydration problems, it's hard to debug.

It would be a bit better if there was a way to show warnings in production mode, but implementing SSR in dev provides a much better workflow and lets developers try their app with full debugging capabilities.

[eyalroth]

@verekia I agree that the development mode should resemble production as much as possible, but the development mode has other workflow features in mind, such as reloading changes on the fly. If those are incompatible (I can't say as I'm not proficient enough in this), then the choice is between two separate workflows.

This is outside of the scope of this issue, and in fact, this is addressed by this issue, which is also linked in the description of this one.

Regardless, even if development mode was identical to production, I believe this issue would've plagued many projects. It is hard to detect for other reasons as well, and in general it is not documented enough -- in fact, some official guides of certain plugins contain instructions that will raise this problem.

IMO gatsby should aim to provide plugins and other solutions that make it easy to avoid this problem, and that it should only be encountered in niche use cases, which doesn't seem to be current state of affairs.

[bogdibota]

If you are just looking to copy-paste the two-pass rendering solution, here's the HOC version:

export const withTwoPassRendering = (WrappedComponent) => ({
  children,
  ...rest
}) => {
  const [isClient, setIsClient] = useState(false);

  useEffect(() => {
    setIsClient(true);
  }, [setIsClient]);

  return (
    <WrappedComponent {...rest} key={isClient}>
      {children}
    </WrappedComponent>
  );
};

And the usage example:

Before:

<MyComponent prop={value}>
   ...children
</MyComponent>

After:

const MyComponentWithTwoPass = withTwoPassRendering(MyComponent);

// ...

<MyComponentWithTwoPass prop={value}>
   ...children
</MyComponentWithTwoPass>

[monsieurBoutte]

Man the fact that this has been a known issue for over a year without any substantial update on how to resolve this, scares the shit out of me. I'm having to rely on the workaround of skipping rehydration and its murdering the performance of my production app. At this point I'm about to move my code base from Gatsby to Next.

Am I over reacting here? Is the gatsby team working on a fix and not a patch? Has anyone moved their code base to NextJs for this reason already?

[monsieurBoutte]

Follow up: I don't think the current "workaround" of replacing the hydration function is much of a workaround at all.

Why?
Because I've realized, in production yet again, that this only "works" within session context. If the user leaves the browser session (i.e. changes from the mobile browser to another app, leaving their current tab open) when they come back and refresh the page... white screen, and the only way to rectify this is by having the user log out and back in.

There no way I can rely on this as an acceptable solution to tell my users to just log out and back in.

For this reason, I'm absolutely moving my codebase to nextjs.

I really hope the gatsby team is able to prioritize this issue much higher because this going to cause a lot of pain for production apps built with gatsby

[verekia]

Is the two-pass rendering not an option for you? Hydration problems are not a Next vs Gatsby thing, it's React. You will have to fix hydration issues with both.

The problem here is mainly about informing users of mismatches, there is no bug. They need to show the warning, or at the very least document the problem well.

But if you are aware of scenarios that will cause hydration issues, you can handle them correctly already. With a two-pass rendering (if you need SEO for that component), or if you want to avoid content flashing by rendering your content client-side only (like a Sign Up / Log Out button depending on if the user is logged in or not).

[eyalroth]

@verekia Perhaps this is not a bug, but I would say it's a big problem nonetheless that needs a better solution than just proper documentation and warnings.

Any website that wants to have a common layout between its pages -- something I believe is fairly common -- would have to deal with this problem, and it's not all that trivial. See my post about gatsby layouts (which I already linked in this issue).

[monsieurBoutte]

My apologies, I should follow up here, I realized my white screen on session expiration wasn't related to Gatsby's hydration issue. The issue was on my part which was simply that I wasn't properly redirected my users to the login screen when their auth session cookies expired.

However, I do still think the hydration workaround is an issue and the withTwoPassRendering doesn't circumvent the performance losses. Surely there's a better way to handle this common error than passing off the responsibility to the dev implementing a workaround for the framework

[bzmillerboy]

I found this article very helpful in working around this problem: https://www.joshwcomeau.com/react/the-perils-of-rehydration/

[javiertury]

After countless hours, I discovered why styles were broken after SSR hydration on production builds. The problem was that babel-plugin-styled-components doesn't work correctly if styled-components is re-exported (styled-components/babel-plugin-styled-components#351).

import styled from '../lib/styled-components'; //  Does not work :(
import styled from 'styled-components'; //  Works

If you were re-exporting the module to add theme type definitions, please use module augmentation instead.

// src/types/styled-components.ts
import type { Theme } from '@material-ui/core';

declare module 'styled-components' {
  export interface DefaultTheme extends Theme {
  }
}

[NoahDavidATL]

Has this been fixed? Tired of using work-arounds.

[MarcelLouwrens]

Any update in this? Currently using https://www.joshwcomeau.com/react/the-perils-of-rehydration/, but even using simple dynamic class template strings causes my whole page to be rendered twice? Very strange and difficult to debug.

[traviswimer]

This sounds like you might just be using the workaround too high up.

The solution you linked to works by forcing a re-render, so if you're using it for your page component, it should be expected that your page will render twice.

This workaround isn't something you can just wrap your whole app with, it needs to be used in the specific locations where hydration mismatches are a problem. Basically it should be at the lowest level, where it is acceptable for the user to see a re-render.

[amalitsky]

Ok, writing this to future myself: removing the "native" image ![title](file_path) from MDX content resolves this issue (for my very specific case I guess).

Even though @polarathene got some presumably related fix merged in, it doesn't work for me on the latest 4.24.7 gatsby and all plugins up to date.

Stumbled upon this more than once, unfortunately.
And every time through playwright tests which means extensive debugging.

[nugoo1]

Why is this still an issue ?? Jesus..

[breart]

Why is this still an issue ?? Jesus..

Already started migrating to NextJS. @gatsbyjs team 🫡

[traviswimer]

Already started migrating to NextJS. @gatsbyjs team 🫡

Did NextJS really figure out a decent way to "solve" this?

I would think it would be impossible to create a universal solution to this, since it's an inherent problem with two machines producing different output rather than a bug.