Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace cli flag --prebuilt-platform with --build-host #6859

Merged
merged 121 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
dc0902b
WIP
lukewilliamboswell Jul 2, 2024
753cfb4
fix: building valgrind surgical host
lukewilliamboswell Jul 9, 2024
85447ec
replace stray strings with impl on Target
lukewilliamboswell Jul 9, 2024
96c559e
minor fixes
lukewilliamboswell Jul 9, 2024
4ccbc41
replace gen-stub-lib in nix CI tests
lukewilliamboswell Jul 9, 2024
287d00d
remove gen_stub_lib
lukewilliamboswell Jul 9, 2024
f515311
clippy fix
lukewilliamboswell Jul 9, 2024
bd68787
add --build-host for benchmark tests
lukewilliamboswell Jul 9, 2024
61a5a0a
checkpoint: refactor prebuilt host artifacts
r-bar Jul 9, 2024
d747cc9
Merge remote-tracking branch 'remotes/luke/rebuild-platform' into reb…
r-bar Jul 9, 2024
dfa3254
checkpoint: it compiles
r-bar Jul 11, 2024
bc29782
checkpoint: Run struct refactor for helpers
r-bar Jul 22, 2024
89d63f2
Fix ownership issues with runners
smores56 Aug 14, 2024
3662c90
WIP get things compiling... hacks everywhere beware
lukewilliamboswell Aug 15, 2024
4ec213a
Get tests working
smores56 Aug 15, 2024
550bade
Merge branch 'main' into rebuild-platform
smores56 Aug 15, 2024
49f5ea8
Merge branch 'main' into rebuild-platform
smores56 Aug 15, 2024
28ceb0e
Fix formatting and clippy issues
smores56 Aug 15, 2024
2928969
restore one test
lukewilliamboswell Aug 15, 2024
6276cc7
WIP more cli_run tests passing
lukewilliamboswell Aug 15, 2024
4863234
Merge branch 'rebuild-platform' of github.com:roc-lang/roc into rebui…
lukewilliamboswell Aug 15, 2024
014514a
WIP more passing roc_cli tests
lukewilliamboswell Aug 15, 2024
64cc816
WIP all the roc_cli tests passing, woo
lukewilliamboswell Aug 15, 2024
9ecb209
add suppress warning flag, remove test test_roc_app_slim
lukewilliamboswell Aug 16, 2024
0767115
WIP refactor to build test platform once per test run
lukewilliamboswell Aug 16, 2024
804cb6e
WIP upgrade cli test packages fixture tests
lukewilliamboswell Aug 16, 2024
c8522c2
WIP upgrade cli tests transitive-deps
lukewilliamboswell Aug 16, 2024
d2fbf86
format test files, upgrade some tests
lukewilliamboswell Aug 16, 2024
7ce9ecd
WIP improve cli tests
lukewilliamboswell Aug 16, 2024
894e4d1
make clippy happy again
lukewilliamboswell Aug 16, 2024
5c8eb4f
rename cli to basic-cli
lukewilliamboswell Aug 16, 2024
6ab0de1
fix benchmark tests
lukewilliamboswell Aug 16, 2024
0256c4e
move false interpreter into cli tests
lukewilliamboswell Aug 16, 2024
8ad8198
move effects cli test
lukewilliamboswell Aug 16, 2024
fa40166
move tui example into cli tests
lukewilliamboswell Aug 16, 2024
c3785a2
fix more tests
lukewilliamboswell Aug 16, 2024
767ec7a
cli run tests passing on macos
lukewilliamboswell Aug 16, 2024
ac78d39
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Aug 17, 2024
8612220
fix plumbing for roc_glue
lukewilliamboswell Aug 18, 2024
b0c1d88
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Aug 18, 2024
e7a90f1
fix clippy
lukewilliamboswell Aug 18, 2024
cd955fd
remove check_compile_error, fix cli tests
lukewilliamboswell Aug 18, 2024
93a4c58
fix cli tests - mainly wasm and i386
lukewilliamboswell Aug 18, 2024
0e917be
add LEGACY LINKER conditional for test runs
lukewilliamboswell Aug 18, 2024
4e6178c
move valgrind configuration into Run builder
lukewilliamboswell Aug 18, 2024
f92974c
use legacy linker in tests
lukewilliamboswell Aug 18, 2024
012a2fc
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Sep 3, 2024
a11f51a
fixup
lukewilliamboswell Sep 3, 2024
ec8f6e8
clippy fix
Anton-4 Sep 3, 2024
251e1ed
misc improvements
Anton-4 Sep 3, 2024
ef87653
migrate cli test combine-tasks.roc off basic-cli
lukewilliamboswell Sep 4, 2024
c6a67c9
remove basic-cli tests from roc_cli
lukewilliamboswell Sep 4, 2024
5dfcdae
migrate inspect-logging.roc into cli tests and use effects platform
lukewilliamboswell Sep 4, 2024
73846ed
add test for module_params pass_task.roc
lukewilliamboswell Sep 4, 2024
beb2dcb
remove unneeded basic-cli reference in formatting test
lukewilliamboswell Sep 4, 2024
bb57c4a
remove unneded basic-cli reference in test_reporting
lukewilliamboswell Sep 4, 2024
dc138b8
remove last usage of basic-cli platform from the tests
lukewilliamboswell Sep 4, 2024
03a2d36
fix surgical linking
Anton-4 Oct 8, 2024
e3afeaa
WIP Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Oct 8, 2024
81668e6
move roc_cli tests to use insta snapshots
lukewilliamboswell Oct 8, 2024
8b8db29
restore module_params cli tests
lukewilliamboswell Oct 8, 2024
de9491e
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Oct 8, 2024
61d4ceb
fix input 'rust-overlay' has an override for a non-existent input 'fl…
lukewilliamboswell Oct 8, 2024
57c0f0d
fix can test tip message
lukewilliamboswell Oct 9, 2024
3bad18d
refactoring progress, use valgrind on exe
Anton-4 Oct 9, 2024
0659abd
cli tests refactoring progress
Anton-4 Oct 11, 2024
7dfa469
stuck on Only 64bit little endian elf currently...
Anton-4 Oct 12, 2024
a12d281
fix 'Only 64bit little endian...' with prebuilt platform
Anton-4 Oct 15, 2024
b346adf
fixes and refactoring progress
Anton-4 Oct 16, 2024
5b451f9
fixed run_multi_dep_str
Anton-4 Oct 16, 2024
a18e410
refactoring cli_tests progress
Anton-4 Oct 16, 2024
54ae11b
finished refactoring simple zig platform tests
Anton-4 Oct 18, 2024
9f4c8b1
refactoring progress
Anton-4 Oct 18, 2024
37f1c98
finished refactoring benchmarks
Anton-4 Oct 19, 2024
5953155
done with cli_tests
Anton-4 Oct 19, 2024
c64160a
valgrind refactoring progress
Anton-4 Oct 19, 2024
0e28199
fixed valgrind crate
Anton-4 Oct 21, 2024
1b6d83d
typo
Anton-4 Oct 22, 2024
63b8e50
refactoring PrebuiltHost, fixed glue tests
Anton-4 Oct 23, 2024
78438df
Merge branch 'rebuild-platform' of github.com:roc-lang/roc into rebui…
Anton-4 Oct 23, 2024
9b7c6c9
cleanup warnings
Anton-4 Oct 23, 2024
9260145
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Oct 23, 2024
d0ee040
make clippy happy
lukewilliamboswell Oct 24, 2024
157c0d2
clippy happy in --release too
lukewilliamboswell Oct 24, 2024
bc60d62
cleaup platform-switching Rust platform
lukewilliamboswell Oct 24, 2024
745c99f
fix rust platform switching build script
lukewilliamboswell Oct 24, 2024
82beb35
updated benchmarks to new ExecCli
Anton-4 Oct 25, 2024
afd2636
removed unused dependency, wasm benchmark debugging progress
Anton-4 Oct 26, 2024
c00db6d
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Nov 4, 2024
81f466f
cargo fmt
lukewilliamboswell Nov 4, 2024
1b0cd14
add script to prebuild platform-switching rust
lukewilliamboswell Nov 4, 2024
f20465c
use no_main on static lib
lukewilliamboswell Nov 4, 2024
293fedb
add build script to allow undefined, make workspace cargo happy
lukewilliamboswell Nov 4, 2024
d525105
fix typo
lukewilliamboswell Nov 4, 2024
743030f
ignore platform-switching rust cli test
lukewilliamboswell Nov 4, 2024
8a566dc
WIP
lukewilliamboswell Nov 10, 2024
aa98706
complete merge remote/main into rebuild-platform
lukewilliamboswell Nov 11, 2024
e4d0115
Merge remote-tracking branch 'remote/main' into rebuild-platform
lukewilliamboswell Nov 11, 2024
f84aad1
move effectful test platform into rust crate
lukewilliamboswell Nov 11, 2024
e4fd9ce
use crate for test-platform-simple-zig
lukewilliamboswell Nov 11, 2024
fc0f938
fix test_reporting
lukewilliamboswell Nov 11, 2024
a04c173
use older syntax for rerun-if-changed
lukewilliamboswell Nov 11, 2024
fc315f7
revert separate test platform commits
lukewilliamboswell Nov 11, 2024
9465b29
fix test reporting
lukewilliamboswell Nov 11, 2024
09bd145
update snapshot for newline_in_packages syntax test
lukewilliamboswell Nov 11, 2024
def5372
fix combine_tasks_with_record_builder cli test on linux
lukewilliamboswell Nov 11, 2024
bc5e3f4
fix stray dbg limiting single Thread, causes stack overflow Windows
lukewilliamboswell Nov 11, 2024
76942e4
ignore windows tests that aren't implemented yet
lukewilliamboswell Nov 11, 2024
af6a7db
ensure Windows git keeps line endings consitent with Unix
lukewilliamboswell Nov 11, 2024
24a9000
use only unix style slashes in problem reports
lukewilliamboswell Nov 11, 2024
fb795da
update snapshots to use unix slashes
lukewilliamboswell Nov 11, 2024
0a4dc0e
cargo fmt
lukewilliamboswell Nov 11, 2024
699bea8
try dont use both linkers on basic-cli tests
lukewilliamboswell Nov 11, 2024
8833589
Merge branch 'rebuild-platform' of github.com:roc-lang/roc into rebui…
lukewilliamboswell Nov 11, 2024
04ab86f
clippy fix
lukewilliamboswell Nov 11, 2024
aad1df5
ignore basic-cli tests on linux, broken on nix CI
lukewilliamboswell Nov 11, 2024
74848bb
ignore expects dev test on linux, needs further discussion
lukewilliamboswell Nov 11, 2024
7295402
try unix instead of linux for ignore_attr
lukewilliamboswell Nov 11, 2024
98bb910
try unix instead of linux for ignore_attr
lukewilliamboswell Nov 11, 2024
9442a32
use legacy linker for basic-ssg website build script
lukewilliamboswell Nov 11, 2024
9a3495c
minor cleanup
Anton-4 Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Require roc files to be checked out with Unix line endings, even on windows
*.roc text eol=lf
* text=auto eol=lf

crates/compiler/test_mono/generated/* linguist-generated=true
crates/compiler/test_mono/generated/* linguist-generated=true
19 changes: 10 additions & 9 deletions .github/workflows/nightly_macos_apple_silicon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ on:
#pull_request:
workflow_dispatch:
schedule:
- cron: '0 9 * * *'
- cron: "0 9 * * *"

name: Nightly Release macOS Apple Silicon

Expand Down Expand Up @@ -30,15 +30,15 @@ jobs:
run: cargo test --locked --release

- name: get commit SHA
run: echo "SHA=$(git rev-parse --short "$GITHUB_SHA")" >> $GITHUB_ENV
run: echo "SHA=$(git rev-parse --short "$GITHUB_SHA")" >> $GITHUB_ENV

- name: get date
run: echo "DATE=$(date "+%Y-%m-%d")" >> $GITHUB_ENV

- name: build file name
env:
DATE: ${{ env.DATE }}
SHA: ${{ env.SHA }}
DATE: ${{ env.DATE }}
SHA: ${{ env.SHA }}
run: echo "RELEASE_FOLDER_NAME=roc_nightly-macos_apple_silicon-$DATE-$SHA" >> $GITHUB_ENV

- name: write version to file
Expand All @@ -56,8 +56,9 @@ jobs:
- name: extract tar for a quick test
run: ls | grep tar | xargs tar -xf

- name: test with rust platform
run: cd ${{ env.RELEASE_FOLDER_NAME }} && ./roc examples/platform-switching/rocLovesRust.roc
- name: test with zig platform
run: |
cd ${{ env.RELEASE_FOLDER_NAME }} && ./roc --build-host --suppress-build-host-warning crates/cli/tests/test-projects/test-platform-simple-zig/app.roc

- name: print short commit SHA
run: git rev-parse --short "$GITHUB_SHA"
Expand All @@ -66,6 +67,6 @@ jobs:
- name: Upload artifact Actually uploading to github releases has to be done manually
uses: actions/upload-artifact@v4
with:
name: ${{ env.RELEASE_FOLDER_NAME }}.tar.gz
path: ${{ env.RELEASE_FOLDER_NAME }}.tar.gz
retention-days: 4
name: ${{ env.RELEASE_FOLDER_NAME }}.tar.gz
path: ${{ env.RELEASE_FOLDER_NAME }}.tar.gz
retention-days: 4
2 changes: 1 addition & 1 deletion .github/workflows/nix_linux_x86_64.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: roc test all builtins
run: nix develop -c ./ci/roc_test_builtins.sh

- name: test wasm32 cli_run
- name: test wasm32 cli_tests
run: nix develop -c cargo test --locked --release --features="wasm32-cli-run"

- name: test the dev backend # these tests require an explicit feature flag
Expand Down
8 changes: 1 addition & 7 deletions .github/workflows/nix_macos_apple_silicon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,17 +34,11 @@ jobs:

# for skipped tests: see issue 6274
- name: execute tests with --release
run: nix develop -c cargo test --locked --release -- --skip cli_run::inspect_gui --skip cli_run::hello_gui
run: nix develop -c cargo test --locked --release -- --skip cli_tests::inspect_gui --skip cli_tests::hello_gui

- name: roc test all builtins
run: nix develop -c ./ci/roc_test_builtins.sh

- name: make a libapp.so for the next step
run: nix develop -c cargo run -- gen-stub-lib examples/platform-switching/rocLovesRust.roc

- name: check that the platform`s produced dylib is loadable
run: cd examples/platform-switching/rust-platform && nix develop -c cargo test --release --locked

- name: test aarch64 dev backend
run: nix develop -c cargo nextest-gen-dev --locked --release --no-fail-fast

Expand Down
8 changes: 1 addition & 7 deletions .github/workflows/nix_macos_x86_64.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,9 @@ jobs:

- uses: cachix/install-nix-action@v22

- name: execute cli_run tests only, the full tests take too long but are run nightly
- name: execute cli_tests tests only, the full tests take too long but are run nightly
run: nix develop -c cargo test --locked --release -p roc_cli -- --skip hello_gui
# see 5932 for hello_gui

- name: roc test all builtins
run: nix develop -c ./ci/roc_test_builtins.sh

- name: make a libapp.so for the next step
run: nix develop -c cargo run -- gen-stub-lib examples/platform-switching/rocLovesRust.roc

- name: check that the platform`s produced dylib is loadable
run: cd examples/platform-switching/rust-platform && nix develop -c cargo test --release --locked
3 changes: 0 additions & 3 deletions .github/workflows/test_nightly_macos_apple_silicon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@ jobs:
- name: test roc hello world
run: cd roc_nightly && ./roc examples/helloWorld.roc

- name: test platform switching rust
run: cd roc_nightly && ./roc examples/platform-switching/rocLovesRust.roc

- name: test platform switching zig
run: cd roc_nightly && ./roc examples/platform-switching/rocLovesZig.roc

Expand Down
7 changes: 2 additions & 5 deletions .github/workflows/ubuntu_x86_64.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,10 @@ jobs:

- name: regular rust tests
# see #5904 for skipped test
run: cargo test --locked --release -- --skip cli_run::expects_dev_and_test
run: cargo test --locked --release -- --skip cli_tests::expects_dev_and_test

- name: tests examples in docs
run: cargo test --doc --release

- name: check that the platform`s produced dylib is loadable
run: cd examples/platform-switching/rust-platform && LD_LIBRARY_PATH=. cargo test --release --locked
run: cargo test --doc --release

- name: test the dev backend # these tests require an explicit feature flag
run: cargo test --locked --release --package test_gen --no-default-features --features gen-dev
Expand Down
33 changes: 8 additions & 25 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ members = [
"crates/fs",
"crates/glue",
"crates/cli",
"crates/cli_utils",
"crates/cli_test_utils",
"crates/highlight",
"crates/error_macros",
"crates/reporting",
Expand All @@ -19,7 +19,7 @@ members = [
"crates/roc_std",
"crates/test_utils",
"crates/test_utils_dir",
"crates/valgrind",
"crates/valgrind_tests",
"crates/tracing",
"crates/utils/*",
"crates/soa",
Expand Down Expand Up @@ -173,7 +173,6 @@ schemars = "0.8.12"
serde = { version = "1.0.153", features = [
"derive",
] } # update roc_std/Cargo.toml on change
serde-xml-rs = "0.6.0"
serde_json = "1.0.94" # update roc_std/Cargo.toml on change
serial_test = "1.0.0"
signal-hook = "0.3.15"
Expand Down
7 changes: 4 additions & 3 deletions ci/basic_nightly_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ set -euxo pipefail

# if to prevent unset vars errror
if [ -n "$(ls | grep -v "roc_nightly.*tar\.gz" | grep -v "^ci$")" ]; then

# Remove everything in this dir except the tar and ci folder.
# We want to test like a user who would have downloaded the release, so we clean up all files from the repo checkout.
to_delete=$(ls | grep -v "roc_nightly.*tar\.gz" | grep -v "^ci$")

for file_or_dir in $to_delete
do
echo "Removing: $file_or_dir"
echo "Removing: $file_or_dir"
rm -rf "$file_or_dir"
done
fi
Expand All @@ -31,7 +31,8 @@ cd roc_nightly
# test roc hello world
./roc examples/helloWorld.roc

# test rust platform
# test rust platform (first prebuild the host)
examples/platform-switching/rust-platform/build.sh
./roc examples/platform-switching/rocLovesRust.roc

# test zig platform
Expand Down
4 changes: 2 additions & 2 deletions crates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ cargo doc --package roc_ast --open

The `roc` binary that brings together all functionality in the Roc toolset.

## `cli_utils/` - `cli_utils`
## `cli_test_utils/` - `cli_test_utils`

Provides shared code for cli tests and benchmarks.
Provides shared code for cli tests, cli benchmarks, glue tests, valgrind crate.

## `compiler/`

Expand Down
15 changes: 7 additions & 8 deletions crates/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@ target-x86 = ["roc_build/target-x86", "roc_repl_cli/target-x86"]
target-x86_64 = ["roc_build/target-x86_64", "roc_repl_cli/target-x86_64"]

target-all = [
"target-aarch64",
"target-arm",
"target-x86",
"target-x86_64",
"target-wasm32",
"target-aarch64",
"target-arm",
"target-x86",
"target-x86_64",
"target-wasm32",
]

sanitizers = ["roc_build/sanitizers"]
Expand Down Expand Up @@ -90,15 +90,14 @@ roc_repl_expect = { path = "../repl_expect" }


[dev-dependencies]
cli_utils = { path = "../cli_utils" }
roc_test_utils = { path = "../test_utils" }
cli_test_utils = { path = "../cli_test_utils" }
roc_command_utils = { path = "../utils/command" }

criterion.workspace = true
indoc.workspace = true
parking_lot.workspace = true
pretty_assertions.workspace = true
serial_test.workspace = true
insta.workspace = true

[build-dependencies]
chrono.workspace = true
Expand Down
2 changes: 1 addition & 1 deletion crates/cli/benches/time_bench.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use std::time::Duration;

use cli_utils::bench_utils::{
use cli_test_utils::bench_utils::{
bench_cfold, bench_deriv, bench_nqueens, bench_quicksort, bench_rbtree_ck,
};
use criterion::{measurement::WallTime, BenchmarkGroup, Criterion, SamplingMode};
Expand Down
8 changes: 2 additions & 6 deletions crates/cli/src/format.rs
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ mod tests {
use std::io::Write;
use tempfile::{tempdir, TempDir};

const FORMATTED_ROC: &str = r#"app [main] { pf: platform "https://github.com/roc-lang/basic-cli/releases/download/0.16.0/O00IPk-Krg_diNS2dVWlI0ZQP794Vctxzv0ha96mK0E.tar.br" }
const FORMATTED_ROC: &str = r#"app [main] { pf: platform "platform/main.roc" }

import pf.Stdout
import pf.Stdin
Expand All @@ -273,11 +273,7 @@ main =
name = Stdin.line!
Stdout.line! "Hi $(name)!""#;

const UNFORMATTED_ROC: &str = r#"app [main] { pf: platform "https://github.com/roc-lang/basic-cli/releases/download/0.16.0/O00IPk-Krg_diNS2dVWlI0ZQP794Vctxzv0ha96mK0E.tar.br" }


import pf.Stdout
import pf.Stdin
const UNFORMATTED_ROC: &str = r#"app [main] { pf: platform "platform/main.roc" }

main =
Stdout.line! "What's your name?"
Expand Down
Loading