Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactored wp-env Config Parsing #50071

Merged
merged 19 commits into from
Apr 28, 2023
Merged

Refactored wp-env Config Parsing #50071

merged 19 commits into from
Apr 28, 2023

Conversation

ObliviousHarmony
Copy link
Contributor

What?

This pull request refactors the existing .wp-env.json config file parsing in order to more cleanly support root properties. It aimed to increase composition and testability while maintaining full compatibility.

Why?

While working on #49996 we decided that it would be best if the postInstall script existed at the configuration root. When I looked into implementing that, however, it proved to be troublesome due to the way config parsing was designed. It was explicitly created to cascade the config file almost immediately, and as a result, it was not easy to introduce root-level options and validation in a way that would not cascade.

How?

We delay the cascade of options from the root to the environments until after validation and parsing have completed. This lets us interact with the config object while the root and environments are both present but separate. As a result we will be able to support root-level configuration options without any additional changes.

I was worried about breaking compatibility, so, this pull request also features an exhaustive test suite update. All new functionality is more or less completely covered for both happy path and failure states. This was a great exercise and caught a number of bugs in the process.

Testing Instructions

  1. Run npm run env -- start --update and make sure that both the development and tests environments work.
  2. Run npm run test:unit packages/env and make sure that they pass.
  3. Create a .wp-env.override.json file and make sure that it overrides as expected. Config and mappings should be merged while plugins and themes are overridden.

This commit changes the way that configuration parsing works in order
to better facilitate the creation of root-only configuration options.
This does, however, leave `wp-env` in a non-functional state. Now
that the config parsing itself is done, I wanted to commit it. I almost
accidentally deleted this once today and that was too many times!
This commit adds basically all of the other tests from
`tests/config.js` in their respective test files. This covers the
last of the tests and the config parsing should now have the
same or better level of test coverage.
In order to create the `WPConfig` object we will switch to
composing the output from config parsing into a new class.
This lets us avoid having to rewrite all of the tests to get
a different output and continue with the trend of
composing together the parsing logic.
This commit finishes the refactor and hooks up the loading
of config files using the new functionality.
@skorasaurus skorasaurus added the [Tool] Env /packages/env label Apr 26, 2023
@noahtallen noahtallen added the [Type] Enhancement A suggestion for improvement. label Apr 27, 2023
Copy link
Member

@noahtallen noahtallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, made it through the non test files 😅

Overall it definitely looks more clear and less clever. :P I did leave a note in one or two places where I think there could be more indirection than there needs to be

packages/env/lib/config/load-config.js Show resolved Hide resolved
packages/env/lib/config/parse-config.js Show resolved Hide resolved
packages/env/lib/config/parse-config.js Outdated Show resolved Hide resolved
packages/env/lib/config/parse-config.js Show resolved Hide resolved
packages/env/lib/config/post-process-config.js Outdated Show resolved Hide resolved
packages/env/lib/config/read-raw-config-file.js Outdated Show resolved Hide resolved
Since we're already handling this in the new parser, there's
no need to perform this translation here.
Copy link
Member

@noahtallen noahtallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for follow-up: we should probably add some documentation for the config parsing at some point

packages/env/test/parse-config.js Outdated Show resolved Hide resolved
packages/env/test/parse-config.js Outdated Show resolved Hide resolved
@noahtallen
Copy link
Member

Also, you make a lot of good points above :) My subjective take is that some of this makes the code more verbose than it really needs to be. Which isn't a huge problem, but I think there are one or two layers of indirection that are a bit difficult to grok.

That said, I think when we get some documentation in place (like "how to add a config key" or "how to add an env var override"), it really won't be a problem

@ObliviousHarmony
Copy link
Contributor Author

Thanks for the review @noahtallen!

I'm actually pretty happy about the above test failure because it shows that the test suite is really robust. When I removed the testsPort handling from readRawConfigFile() it caused the option to be silently discarded. This bug was then caught by the test suite.

I also aded an integration test for the environment variable overrides so that the tests cover all four configuration states.

@noahtallen
Copy link
Member

The Puppeteer 3 suite seems to be failing across all commits right now, unfortunately.

Copy link
Member

@noahtallen noahtallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I'm happy with this! Thanks for the improvement.

@noahtallen noahtallen enabled auto-merge (squash) April 28, 2023 19:47
@noahtallen noahtallen disabled auto-merge April 28, 2023 19:48
@noahtallen noahtallen merged commit b554ae4 into WordPress:trunk Apr 28, 2023
@github-actions github-actions bot added this to the Gutenberg 15.8 milestone Apr 28, 2023
@ObliviousHarmony ObliviousHarmony deleted the refactor/env-config-parsing branch April 28, 2023 19:54
expect( parsed ).toEqual( {
...DEFAULT_CONFIG,
coreSource: {
basename: 'gutenberg',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ObliviousHarmony @noahtallen One side-effect of hard-coding the basename (and also testPath) like this is is that if someone checks out the gutenberg repo into a differently named folder, these tests will fail.

This is currently causing a few issues for CI jobs that run on security patches, as those are created on a differently named private repo.

I was wondering what your thoughts on a good fix would be?

A naive option could be expect.any( String ), but I think maybe this should all be mocked rather than being based off of the real file system.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an interesting case @talldan. It seems like the best source would be to just generally avoid any relative paths and use fake absolute paths in our tests. I've gone ahead and created an issue for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Tool] Env /packages/env [Type] Enhancement A suggestion for improvement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants