Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reorganize tests #809

Merged
merged 24 commits into from
Aug 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
534fee0
WIP: reorganize tests
ranocha Aug 20, 2021
1a546d8
remove many save_restart callbacks from examples
ranocha Aug 20, 2021
6270f73
move some tree tests around
ranocha Aug 20, 2021
95cb88e
enable a restart callback used in the tests
ranocha Aug 20, 2021
d9424cc
split p4est tests into two jobs
ranocha Aug 20, 2021
2ed749a
fix typo
ranocha Aug 20, 2021
d90e806
re-enable the restart callback when used in tests
ranocha Aug 20, 2021
0be48d5
re-enable the restart callback when used in tests
ranocha Aug 20, 2021
021b9a8
move some TreeMesh3D tests
ranocha Aug 20, 2021
1ec7adc
new job tree_part6
ranocha Aug 23, 2021
572c6db
remove a lot of dummy ICs/BCs not really used anymore
ranocha Aug 23, 2021
a4e3176
add test file missing from 1ec7adcd5a4538f920c8f09ad6a2c0436b8e1c16
ranocha Aug 23, 2021
4f8c7a3
Merge branch 'main' into hr/tests
ranocha Aug 23, 2021
0a07738
mention changes of ICs/BCs/sources in NEWS.md
ranocha Aug 23, 2021
e165f0c
organize MPI and threaded tests
ranocha Aug 24, 2021
fa188de
more timed test sets
ranocha Aug 24, 2021
3ae2f3a
Merge branch 'main' into hr/tests
ranocha Aug 24, 2021
3af6d88
split misc tests into two parts and reduce amount of tests a bit
ranocha Aug 24, 2021
a91942f
fix tests
ranocha Aug 25, 2021
dd3622a
try to distribute some tests more uniformly
ranocha Aug 25, 2021
92a6438
fix typo
ranocha Aug 25, 2021
5eafb14
Revert "remove a lot of dummy ICs/BCs not really used anymore"
ranocha Aug 25, 2021
7479776
update docs on tests
ranocha Aug 25, 2021
6158da9
update codecov settings
ranocha Aug 25, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# https://docs.codecov.io/docs/codecovyml-reference

# We have
# 11 * [number of basic OS] + 5 * [number of additional OS]
# 15 * [number of basic OS] + 2 * [number of additional OS]
sloede marked this conversation as resolved.
Show resolved Hide resolved
# with
# [number of basic OS] = 1 (Linux)
# [number of additional OS] = 2 (Windows, MacOS)
# parallel jobs in ci.yml
codecov:
branch: main
notify:
after_n_builds: 21
after_n_builds: 19
comment:
after_n_builds: 21
after_n_builds: 19

coverage:
range: 80..95 # set 95% and above as solid green, everything below 80% as red
Expand Down
58 changes: 19 additions & 39 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,65 +59,45 @@ jobs:
arch:
- x64
trixi_test:
- 1d
- 2d_part1
- 2d_part2
- 2d_part3
- dgmulti
- 2d_mpi
- 2d_threaded
- 3d_part1
- 3d_part2
- tree_part1
- tree_part2
- tree_part3
- tree_part4
- tree_part5
- tree_part6
- structured
- p4est_part1
- p4est_part2
- unstructured_dgmulti
- paper_self_gravitating_gas_dynamics
- misc
- misc_part1
- misc_part2
- mpi
- threaded
include:
- version: '1.6'
os: macOS-latest
arch: x64
trixi_test: 2d_part1
trixi_test: mpi
- version: '1.6'
os: macOS-latest
arch: x64
trixi_test: 2d_part2
- version: '1.6'
os: macOS-latest
arch: x64
trixi_test: 2d_part3
- version: '1.6'
os: macOS-latest
arch: x64
trixi_test: 2d_mpi
- version: '1.6'
os: macOS-latest
arch: x64
trixi_test: 2d_threaded
- version: '1.6'
os: windows-latest
arch: x64
trixi_test: 2d_part1
- version: '1.6'
os: windows-latest
arch: x64
trixi_test: 2d_part2
- version: '1.6'
os: windows-latest
arch: x64
trixi_test: 2d_part3
trixi_test: threaded
- version: '1.6'
os: windows-latest
arch: x64
trixi_test: 2d_mpi
trixi_test: mpi
- version: '1.6'
os: windows-latest
arch: x64
trixi_test: 2d_threaded
trixi_test: threaded
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@v1
with:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
show-versioninfo: true
- run: julia -e 'using InteractiveUtils; versioninfo(verbose=true)'
- uses: actions/cache@v1
env:
cache-name: cache-artifacts
Expand Down
19 changes: 19 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,25 @@ used in the Julia ecosystem. Notable changes will be documented in this file
for human readability.


## Changes when updating to v0.4 from v0.3.x

#### Added

#### Changed

#### Deprecated

#### Removed

- Many initial/boundary conditions and source terms for typical setups were
moved from `Trixi/src` to the example elixirs `Trixi/examples`. Thus, they
are no longer available when `using Trixi`, e.g., the initial condition
for the Kelvin Helmholtz instability.
- Some initial/boundary conditions and source terms for academic verification
setups were removed, e.g., `initial_condition_linear_x` for the 2D linear
advection equation.


## Changes in the v0.3 lifecycle

#### Added
Expand Down
18 changes: 13 additions & 5 deletions docs/src/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,16 @@ executing
julia> using Pkg; Pkg.test("Trixi")
```
in the REPL. Since there already exist many tests, we have split them up into
multiple files in the `test` directory to allow for faster testing of individual parts of the code.
multiple files in the `test` directory to allow for faster testing of individual
parts of the code.
Thus in addition to performing all tests, you can also just `include` one of the
files named `test_xxx.jl` to run only a specific subset, e.g.,
```julia
julia> include(joinpath("test", "test_examples_2d.jl")) # Run all 2D tests
julia> # Run all 2D tests on the P4estMesh
include(joinpath("test", "test_p4est_2d.jl"))

julia> include(joinpath("test", "test_examples_2d_euler.jl")) # Run only 2D tests for the Euler equations
julia> # Run all 1D tests for the Euler equations on the TreeMesh
include(joinpath("test", "test_tree_1d_euler.jl"))
```
For the automated tests with GitHub Actions, we run multiple jobs in parallel to
reduce the waiting time until all tests are finished. You can see the different
Expand All @@ -52,8 +55,8 @@ to configure tests. In general, newly added code must be covered by at least one
test, and all new elixirs added to the `examples/` directory must be used at
least once during testing. New tests should be added to the corresponding
`test/test_xxx.jl` file, e.g., a test involving the 3D linear advection equation
would go into
[`test/test_examples_3d_advection.jl`](https://github.com/trixi-framework/Trixi.jl/blob/main/test/test_examples_3d_advection.jl).
on the `TreeMesh` would go into
[`test/test_tree_3d_advection.jl`](https://github.com/trixi-framework/Trixi.jl/blob/main/test/test_tree_3d_advection.jl).
Please study one of the existing tests and stay consistent to the current style
when creating new tests.

Expand All @@ -63,6 +66,11 @@ tests to pass with each novel feature added to Trixi. Therefore, new tests shoul
short as reasonably possible, i.e., without being too insensitive to pick up
changes or errors in the code.

When you add new tests, please check whether all CI jobs still take approximately
the same time. If the job where you added new tests takes much longer than
everything else, please consider moving some tests from one job to another
(or report this incident and ask the main developers for help).

!!! note "Test duration"
As a general rule, tests should last **no more than 10 seconds** when run
with a single thread and after compilation (i.e., excluding the first run).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -140,7 +137,7 @@ stepsize_callback = StepsizeCallback(cfl=1.6)

callbacks = CallbackSet(summary_callback,
analysis_callback, alive_callback,
save_restart, save_solution,
save_solution,
amr_callback, stepsize_callback);


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval)

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -65,7 +62,7 @@ stepsize_callback = StepsizeCallback(cfl=1.0)

callbacks = CallbackSet(summary_callback,
analysis_callback, alive_callback,
save_restart, save_solution,
save_solution,
stepsize_callback)
###############################################################################
# run the simulation
Expand Down
4 changes: 0 additions & 4 deletions examples/p4est_3d_dgsem/elixir_advection_amr.jl
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -58,7 +55,6 @@ stepsize_callback = StepsizeCallback(cfl=1.2)
callbacks = CallbackSet(summary_callback,
analysis_callback,
alive_callback,
save_restart,
save_solution,
amr_callback,
stepsize_callback)
Expand Down
6 changes: 1 addition & 5 deletions examples/p4est_3d_dgsem/elixir_advection_cubed_sphere.jl
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,6 @@ summary_callback = SummaryCallback()
# The AnalysisCallback allows to analyse the solution in regular intervals and prints the results
analysis_callback = AnalysisCallback(semi, interval=100)

# The SaveRestartCallback allows to save a file from which a Trixi simulation can be restarted
save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

# The SaveSolutionCallback allows to save the solution to a file in regular intervals
save_solution = SaveSolutionCallback(interval=100,
solution_variables=cons2prim)
Expand All @@ -51,7 +47,7 @@ save_solution = SaveSolutionCallback(interval=100,
stepsize_callback = StepsizeCallback(cfl=1.2)

# Create a CallbackSet to collect all callbacks such that they can be passed to the ODE solver
callbacks = CallbackSet(summary_callback, analysis_callback, save_restart, save_solution, stepsize_callback)
callbacks = CallbackSet(summary_callback, analysis_callback, save_solution, stepsize_callback)


###############################################################################
Expand Down
5 changes: 1 addition & 4 deletions examples/p4est_3d_dgsem/elixir_euler_free_stream.jl
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval)

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -99,7 +96,7 @@ stepsize_callback = StepsizeCallback(cfl=1.2)

callbacks = CallbackSet(summary_callback,
analysis_callback, alive_callback,
save_restart, save_solution,
save_solution,
stepsize_callback)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval)

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -61,7 +58,7 @@ stepsize_callback = StepsizeCallback(cfl=0.6)

callbacks = CallbackSet(summary_callback,
analysis_callback, alive_callback,
save_restart, save_solution,
save_solution,
stepsize_callback)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,12 @@ save_solution = SaveSolutionCallback(interval=100,
save_final_solution=true,
solution_variables=cons2prim)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

analysis_interval = 100
alive_callback = AliveCallback(analysis_interval=analysis_interval)
analysis_callback = AnalysisCallback(semi, interval=analysis_interval)

callbacks = CallbackSet(summary_callback, stepsize_callback,
save_restart, save_solution,
save_solution,
analysis_callback, alive_callback)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,17 +62,14 @@ save_solution = SaveSolutionCallback(interval=10,
save_final_solution=true,
solution_variables=cons2prim)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

analysis_interval = 100
alive_callback = AliveCallback(analysis_interval=analysis_interval)

analysis_callback = AnalysisCallback(semi_euler, interval=analysis_interval,
save_analysis=true)

callbacks = CallbackSet(summary_callback, stepsize_callback,
save_restart, save_solution,
save_solution,
analysis_callback, alive_callback)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,19 +87,15 @@ save_solution = SaveSolutionCallback(interval=100,
save_final_solution=true,
solution_variables=cons2prim)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

analysis_interval = 100
alive_callback = AliveCallback(analysis_interval=analysis_interval)

analysis_callback = AnalysisCallback(semi_euler, interval=analysis_interval,
save_analysis=true,
extra_analysis_integrals=(energy_total, energy_kinetic, energy_internal))

# TODO: Taal decide, first AMR or save solution etc.
callbacks = CallbackSet(summary_callback, amr_callback, stepsize_callback,
save_restart, save_solution,
save_solution,
analysis_callback, alive_callback)


Expand Down
5 changes: 1 addition & 4 deletions examples/structured_1d_dgsem/elixir_euler_source_terms.jl
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -49,7 +46,7 @@ stepsize_callback = StepsizeCallback(cfl=0.8)

callbacks = CallbackSet(summary_callback,
analysis_callback, alive_callback,
save_restart, save_solution,
save_solution,
stepsize_callback)


Expand Down
5 changes: 1 addition & 4 deletions examples/structured_2d_dgsem/elixir_advection_nonperiodic.jl
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,

alive_callback = AliveCallback(analysis_interval=analysis_interval)

save_restart = SaveRestartCallback(interval=100,
save_final_restart=true)

save_solution = SaveSolutionCallback(interval=100,
save_initial_solution=true,
save_final_solution=true,
Expand All @@ -51,7 +48,7 @@ stepsize_callback = StepsizeCallback(cfl=1.6)

callbacks = CallbackSet(summary_callback,
analysis_callback, alive_callback,
save_restart, save_solution,
save_solution,
stepsize_callback);
###############################################################################
# run the simulation
Expand Down
Loading