feat(ci): run integration tests on windows #6279

mehulkar · 2023-10-25T19:06:24Z

This PR enables starts running our integration tests on Windows and fixes some of the
issues that come out of that. Mostly the issues are around finding binaries with .exe
extension (the test runner and turbo itself), file path separators, and line endings.

This PR adds a new Github Action "job" for both the Go and Rust variants, because
there are still a few remaining tests that are not passing that will need a little more
investigation. Once this diff is merged, we can individually look at each test (e.g. #6477)

Closes TURBO-1533

Some Prysk references

vercel · 2023-10-25T19:06:26Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
examples-basic-web	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 20, 2023 7:33pm
examples-gatsby-web	🔄 Building (Inspect)	Visit Preview	💬 Add feedback	Nov 20, 2023 7:33pm
examples-nonmonorepo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 20, 2023 7:33pm
examples-tailwind-web	🔄 Building (Inspect)	Visit Preview	💬 Add feedback	Nov 20, 2023 7:33pm

7 Ignored Deployments

Name	Status	Preview	Updated (UTC)
examples-cra-web	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm
examples-designsystem-docs	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm
examples-kitchensink-blog	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm
examples-native-web	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm
examples-svelte-web	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm
examples-vite-web	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm
turbo-site	⬜️ Ignored (Inspect)	Visit Preview	Nov 20, 2023 7:33pm

github-actions · 2023-10-25T19:07:14Z

🟢 Turbopack Benchmark CI successful 🟢

Thanks

mehulkar · 2023-11-17T22:36:21Z

turborepo-tests/integration/tests/run-summary/error.t

@@ -80,7 +80,7 @@ Validate that we got a full task summary for the failed task with an error in .e
    "execution": {
      "startTime": [0-9]+, (re)
      "endTime": [0-9]+, (re)
-      "error": "command .* npm run maybefails exited \(1\)", (re)
+      "error": "command (.*)npm run maybefails exited \(1\)", (re)


This came from Rust tests, I thiink it's unrelated to this PR, but I was cross cehcking that test suite to make sure anything didn't break

chris-olszewski

I'd like to see the Windows integration tests get to running the integration test. Currently they're failing on building the binary.

I also don't love distributing our own compiled version of echo $@. I can accept that we might need an executable due to lack of Windows shebang support, but I think at minimum we should include the source in repo. In the ideal world we'd just add the compilation step as a task-dependency to the integration tests and avoid checking in binaries, but if that's not easily doable I think it's also okay to have a script that refreshes checked in binaries.

.github/workflows/test.yml

turborepo-tests/integration/tests/task-dependencies/topological.t

… non blocking

- Add turbo.exe stub source into this repo - Run it as a task dependency of integration tests - Place the turbo.exe in the appropriate folders in the fixtures

mehulkar · 2023-11-20T07:49:34Z

I'd like to see the Windows integration tests get to running the integration test

It builds and runs the tests for the Go codepath, which is where I started a few weeks ago now. I've disabled the Rust codepath, which is failing to build. I'd like to debug that separately so we can get something merged (it might take me some time to look into this.

I also don't love distributing our own compiled version of echo $@. I can accept that we might need an executable due to lack of Windows shebang support, but I think at minimum we should include the source in repo. In the ideal world we'd just add the compilation step as a task-dependency to the integration tests and avoid checking in binaries, but if that's not easily doable I think it's also okay to have a script that refreshes checked in binaries.

Done, see #6508 or e010acb for details.

due to lack of Windows shebang support

For what it's worth I don't think this is the problem. Executing these echo $@ files manually from Bash works just fine, it's just through Rust that it seems to fail.

github-actions · 2023-11-20T08:03:01Z

Linux Benchmark for 5267d8e

Test	Base	PR	%	Significant %
bench_startup/Turbopack CSR/1000 modules	1116.01ms ± 2.87ms	1080.03ms ± 6.42ms	-3.22%	-1.57%

Click to view full benchmark

Test	Base	PR	%	Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules	21.58ms ± 0.86ms	21.52ms ± 0.84ms	-0.30%
bench_hmr_to_eval/Turbopack CSR/1000 modules	21.23ms ± 0.84ms	21.15ms ± 0.84ms	-0.39%
bench_startup/Turbopack CSR/1000 modules	1116.01ms ± 2.87ms	1080.03ms ± 6.42ms	-3.22%	-1.57%

github-actions · 2023-11-20T09:10:58Z

Linux Benchmark for `9ea5945`

Click to view benchmark

Test	Base	PR	%
bench_hmr_to_commit/Turbopack CSR/1000 modules	22.99ms ± 0.95ms	23.01ms ± 0.96ms	+0.09%
bench_hmr_to_eval/Turbopack CSR/1000 modules	22.49ms ± 0.96ms	22.77ms ± 1.02ms	+1.23%
bench_startup/Turbopack CSR/1000 modules	1104.14ms ± 7.12ms	1091.56ms ± 8.64ms	-1.14%

github-actions · 2023-11-20T18:06:13Z

Linux Benchmark for `bc8911f`

Click to view benchmark

Test	Base	PR	%
bench_hmr_to_commit/Turbopack CSR/1000 modules	22.69ms ± 0.93ms	22.67ms ± 0.96ms	-0.05%
bench_hmr_to_eval/Turbopack CSR/1000 modules	22.44ms ± 0.80ms	22.29ms ± 1.02ms	-0.68%
bench_startup/Turbopack CSR/1000 modules	1102.86ms ± 9.12ms	1087.61ms ± 14.96ms	-1.38%

github-actions · 2023-11-20T18:38:20Z

Linux Benchmark for `b98b9ae`

Click to view benchmark

Test	Base	PR	%
bench_hmr_to_commit/Turbopack CSR/1000 modules	21.44ms ± 0.84ms	21.54ms ± 0.89ms	+0.47%
bench_hmr_to_eval/Turbopack CSR/1000 modules	20.98ms ± 0.81ms	20.96ms ± 0.81ms	-0.11%
bench_startup/Turbopack CSR/1000 modules	1102.80ms ± 6.07ms	1120.68ms ± 14.38ms	+1.62%

github-actions · 2023-11-20T19:48:03Z

Linux Benchmark for `2cc6b6e`

Click to view benchmark

Test	Base	PR	%
bench_hmr_to_commit/Turbopack CSR/1000 modules	23.17ms ± 0.97ms	23.24ms ± 1.01ms	+0.32%
bench_hmr_to_eval/Turbopack CSR/1000 modules	23.35ms ± 0.67ms	23.02ms ± 0.82ms	-1.40%
bench_startup/Turbopack CSR/1000 modules	1114.34ms ± 11.59ms	1107.81ms ± 17.32ms	-0.59%

mehulkar · 2023-11-20T21:28:29Z

The Turborepo Rust testing on macOS is failing but is fixed in #6517

turbo-orchestrator bot added area: ci created-by: turborepo labels Oct 25, 2023

vercel bot deployed to Preview – turbo-site October 26, 2023 03:55 View deployment

vercel bot deployed to Preview – turbo-site October 26, 2023 18:09 View deployment

mehulkar mentioned this pull request Oct 27, 2023

chore(test): fix gitignore in fixture #6300

Merged

mehulkar changed the title ~~windows only~~ feat(ci): run integration tests on windows Oct 30, 2023

vercel bot deployed to Preview – turbo-site October 30, 2023 19:20 View deployment

turbo-orchestrator bot removed the owned-by: web-tooling label Oct 30, 2023

vercel bot deployed to Preview – examples-vite-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-tailwind-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-svelte-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-cra-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-designsystem-docs October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-gatsby-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-nonmonorepo October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-native-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-basic-web October 31, 2023 19:47 View deployment

vercel bot deployed to Preview – examples-kitchensink-blog October 31, 2023 19:48 View deployment

vercel bot deployed to Preview – turbo-site October 31, 2023 19:48 View deployment

vercel bot deployed to Preview – turbo-site November 1, 2023 08:43 View deployment

vercel bot had a problem deploying to Preview – examples-nonmonorepo November 1, 2023 17:22 Failure

vercel bot deployed to Preview – turbo-site November 1, 2023 17:23 View deployment

mehulkar added 4 commits November 2, 2023 12:03

set autocrlf settings before checkout also

a574f9c

set local config after creating it

eb39302

double escape

3c0f4db

use bash for git setting too

be7bea4

vercel bot deployed to Preview – turbo-site November 17, 2023 22:20 View deployment

vercel bot deployed to Preview – examples-nonmonorepo November 17, 2023 22:21 View deployment

Merge branch 'main' into mk/windows

7f99358

mehulkar commented Nov 17, 2023

View reviewed changes

chris-olszewski requested changes Nov 18, 2023

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

turborepo-tests/integration/tests/task-dependencies/topological.t Outdated Show resolved Hide resolved

mehulkar added 2 commits November 17, 2023 18:27

Enable windows tests for rust code path in matrix, since it's already…

32482b5

… non blocking

revert

76ee8cc

mehulkar requested a review from chris-olszewski November 18, 2023 00:28

mehulkar added 2 commits November 20, 2023 01:45

Build turbo.exe stub on Windows (#6508)

e010acb

- Add turbo.exe stub source into this repo - Run it as a task dependency of integration tests - Place the turbo.exe in the appropriate folders in the fixtures

Disable windows from rust codepath

54a18e5

Merge branch 'main' into mk/windows

09d40a2

vercel bot deployed to Preview – turbo-site November 20, 2023 08:54 View deployment

mehulkar added 2 commits November 20, 2023 11:48

fixup

243c24c

Merge branch 'main' into mk/windows

60e150f

mehulkar requested a review from NicholasLYang November 20, 2023 18:04

Merge branch 'main' into mk/windows

51f6b79

mehulkar mentioned this pull request Nov 20, 2023

chore(tests): delete e2e tests for turbo prune #6515

Merged

chris-olszewski approved these changes Nov 20, 2023

View reviewed changes

Merge branch 'main' into mk/windows

06a3e1f

vercel bot deployed to Preview – examples-nonmonorepo November 20, 2023 19:32 View deployment

vercel bot deployed to Preview – examples-basic-web November 20, 2023 19:33 View deployment

mehulkar merged commit 1431566 into main Nov 20, 2023
52 of 66 checks passed

mehulkar deleted the mk/windows branch November 20, 2023 21:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ci): run integration tests on windows #6279

feat(ci): run integration tests on windows #6279

mehulkar commented Oct 25, 2023 •

edited

Loading

vercel bot commented Oct 25, 2023 •

edited

Loading

github-actions bot commented Oct 25, 2023 •

edited

Loading

mehulkar Nov 17, 2023

chris-olszewski left a comment

mehulkar commented Nov 20, 2023 •

edited

Loading

github-actions bot commented Nov 20, 2023

github-actions bot commented Nov 20, 2023

github-actions bot commented Nov 20, 2023

github-actions bot commented Nov 20, 2023

github-actions bot commented Nov 20, 2023

mehulkar commented Nov 20, 2023

feat(ci): run integration tests on windows #6279

feat(ci): run integration tests on windows #6279

Conversation

mehulkar commented Oct 25, 2023 • edited Loading

vercel bot commented Oct 25, 2023 • edited Loading

github-actions bot commented Oct 25, 2023 • edited Loading

🟢 Turbopack Benchmark CI successful 🟢

mehulkar Nov 17, 2023

Choose a reason for hiding this comment

chris-olszewski left a comment

Choose a reason for hiding this comment

mehulkar commented Nov 20, 2023 • edited Loading

github-actions bot commented Nov 20, 2023

Linux Benchmark for 5267d8e

github-actions bot commented Nov 20, 2023

Linux Benchmark for 9ea5945

github-actions bot commented Nov 20, 2023

Linux Benchmark for bc8911f

github-actions bot commented Nov 20, 2023

Linux Benchmark for b98b9ae

github-actions bot commented Nov 20, 2023

Linux Benchmark for 2cc6b6e

mehulkar commented Nov 20, 2023

mehulkar commented Oct 25, 2023 •

edited

Loading

vercel bot commented Oct 25, 2023 •

edited

Loading

github-actions bot commented Oct 25, 2023 •

edited

Loading

mehulkar commented Nov 20, 2023 •

edited

Loading

Linux Benchmark for `9ea5945`

Linux Benchmark for `bc8911f`

Linux Benchmark for `b98b9ae`

Linux Benchmark for `2cc6b6e`