Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow usage of turbo without turbo.json #9149

Conversation

chris-olszewski
Copy link
Member

Description

This PR adds the ability to use turbo in a monorepo that doesn't have a turbo.json. This feature is currently gated behind --experimental-allow-no-turbo-json/TURBO_ALLOW_NO_TURBO_JSON.

A majority of the PR is refactoring the EngineBuilder so that it no longer directly loads TurboJsons, but delegates to a TurboJsonLoader. This allows us to use different strategies for resolving TurboJson loads depending on runtime options e.g. single package mode or task access.

Reviewing this PR is best done by viewing each commit individually.

Testing Instructions

Unit testing for turbo.json loading changes.

Integration test for verifying the new loader is activated with the new env var/flag.

Copy link

vercel bot commented Sep 16, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-nonmonorepo ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 17, 2024 2:19pm
8 Skipped Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-designsystem-docs ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-gatsby-web ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-kitchensink-blog ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-native-web ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-svelte-web ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-tailwind-web ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm
examples-vite-web ⬜️ Ignored (Inspect) Visit Preview Sep 17, 2024 2:19pm

Copy link
Member Author

@chris-olszewski chris-olszewski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer's Guide

@@ -650,41 +626,6 @@ mod test {
.unwrap()
}

#[test]
fn test_turbo_json_loading() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is moved to TurboJsonLoader unit tests

/// Create a loader that will load turbo.json files throughout the workspace
pub fn workspace(
repo_root: AbsoluteSystemPathBuf,
packages: HashMap<PackageName, AbsoluteSystemPathBuf>,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be made more ergonomic by just taking &PackageGraph and keeping the extraction logic in this module. I avoided this to allow for testing loaders without constructing an entire package graph.

@chris-olszewski chris-olszewski marked this pull request as ready for review September 16, 2024 14:41
@chris-olszewski chris-olszewski requested a review from a team as a code owner September 16, 2024 14:41
Copy link
Contributor

@NicholasLYang NicholasLYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, just a couple small comments

crates/turborepo-lib/src/turbo_json/mod.rs Outdated Show resolved Hide resolved
crates/turborepo-lib/src/engine/builder.rs Outdated Show resolved Hide resolved
crates/turborepo-lib/src/engine/builder.rs Outdated Show resolved Hide resolved
/// Load a turbo.json for a given package
pub fn load(&self, package: &PackageName) -> Result<TurboJson, Error> {
pub fn load<'a>(&'a mut self, package: &PackageName) -> Result<&'a TurboJson, Error> {
if !self.cache.contains_key(package) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing the Entry API doesn't work here cause of the partial borrows?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct

@@ -114,6 +144,7 @@ impl TurboJsonLoader {
)
}
}
Strategy::Noop => Err(Error::NoTurboJSON),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we should make the API into a Result<Option<TurboJson>, Error> to more explicitly force callers to handle the non-existent turbo.json case

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only when building the engine do we view a non-existent turbo.json as a non-error. I'm not sure requiring the other callers to construct the NoTurboJSON error type manually is worth the better ergonomics in the engine builder.

let task_name = TaskName::from(script.as_str()).into_root_task();
turbo_json.tasks.insert(
task_name,
Spanned::new(RawTaskDefinition {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be worth keeping some span info here. We can take the script span from package.json and use it as the task definition location

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having trouble getting spans for the script names as BTreeMap requires impl Ord for Spanned<T> which has issues since it contains Range which doesn't implement Ord.

I don't think this is worth the effort to plumb through. The only place this span would get used would be for validating no package task syntax and having that error point to a package.json isn't helpful when we're the ones generating turbo.json.

crates/turborepo-lib/src/run/builder.rs Outdated Show resolved Hide resolved
### Description

In response to
#9149 (comment)
this PR avoids mutating `opts` when building `Run` we now set env mode
at the task level.

A majority of this PR is adding the ability to set env mode at the task
level.

### Testing Instructions

Updated integration test (update needed as task hash changed since env
mode is now set at the task level instead of the global level).
@chris-olszewski chris-olszewski enabled auto-merge (squash) September 17, 2024 14:33
@chris-olszewski chris-olszewski merged commit 8856b3b into main Sep 17, 2024
40 checks passed
@chris-olszewski chris-olszewski deleted the chrisolszewski/fdrtn-177-allow-running-turbo-without-turbojson branch September 17, 2024 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants