reorganize tests (#809)

* WIP: reorganize tests * remove many save_restart callbacks from examples * move some tree tests around * enable a restart callback used in the tests * split p4est tests into two jobs * fix typo * re-enable the restart callback when used in tests * re-enable the restart callback when used in tests * move some TreeMesh3D tests * new job tree_part6 * remove a lot of dummy ICs/BCs not really used anymore * add test file missing from 1ec7adc * mention changes of ICs/BCs/sources in NEWS.md * organize MPI and threaded tests * more timed test sets * split misc tests into two parts and reduce amount of tests a bit * fix tests * try to distribute some tests more uniformly * fix typo * Revert "remove a lot of dummy ICs/BCs not really used anymore" This reverts commit 572c6db. * update docs on tests * update codecov settings
trixi-framework · Aug 25, 2021 · 5f1324d · 5f1324d
1 parent 6491e0c
commit 5f1324d
Show file tree

Hide file tree

Showing 110 changed files with 707 additions and 708 deletions.
diff --git a/.codecov.yml b/.codecov.yml
@@ -1,17 +1,17 @@
 # https://docs.codecov.io/docs/codecovyml-reference
 
 # We have
-#   11 * [number of basic OS] + 5 * [number of additional OS]
+#   15 * [number of basic OS] + 2 * [number of additional OS]
 # with
 # [number of basic OS] = 1 (Linux)
 # [number of additional OS] = 2 (Windows, MacOS)
 # parallel jobs in ci.yml
 codecov:
   branch: main
   notify:
-    after_n_builds: 21
+    after_n_builds: 19
 comment:
-  after_n_builds: 21
+  after_n_builds: 19
 
 coverage:
   range: 80..95 # set 95% and above as solid green, everything below 80% as red

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -59,65 +59,45 @@ jobs:
         arch:
           - x64
         trixi_test:
-          - 1d
-          - 2d_part1
-          - 2d_part2
-          - 2d_part3
-          - dgmulti
-          - 2d_mpi
-          - 2d_threaded
-          - 3d_part1
-          - 3d_part2
+          - tree_part1
+          - tree_part2
+          - tree_part3
+          - tree_part4
+          - tree_part5
+          - tree_part6
+          - structured
+          - p4est_part1
+          - p4est_part2
+          - unstructured_dgmulti
           - paper_self_gravitating_gas_dynamics
-          - misc
+          - misc_part1
+          - misc_part2
+          - mpi
+          - threaded
         include:
           - version: '1.6'
             os: macOS-latest
             arch: x64
-            trixi_test: 2d_part1
+            trixi_test: mpi
           - version: '1.6'
             os: macOS-latest
             arch: x64
-            trixi_test: 2d_part2
-          - version: '1.6'
-            os: macOS-latest
-            arch: x64
-            trixi_test: 2d_part3
-          - version: '1.6'
-            os: macOS-latest
-            arch: x64
-            trixi_test: 2d_mpi
-          - version: '1.6'
-            os: macOS-latest
-            arch: x64
-            trixi_test: 2d_threaded
-          - version: '1.6'
-            os: windows-latest
-            arch: x64
-            trixi_test: 2d_part1
-          - version: '1.6'
-            os: windows-latest
-            arch: x64
-            trixi_test: 2d_part2
-          - version: '1.6'
-            os: windows-latest
-            arch: x64
-            trixi_test: 2d_part3
+            trixi_test: threaded
           - version: '1.6'
             os: windows-latest
             arch: x64
-            trixi_test: 2d_mpi
+            trixi_test: mpi
           - version: '1.6'
             os: windows-latest
             arch: x64
-            trixi_test: 2d_threaded
+            trixi_test: threaded
     steps:
       - uses: actions/checkout@v2
       - uses: julia-actions/setup-julia@v1
         with:
           version: ${{ matrix.version }}
           arch: ${{ matrix.arch }}
-          show-versioninfo: true
+      - run: julia -e 'using InteractiveUtils; versioninfo(verbose=true)'
       - uses: actions/cache@v1
         env:
           cache-name: cache-artifacts

diff --git a/NEWS.md b/NEWS.md
@@ -5,6 +5,25 @@ used in the Julia ecosystem. Notable changes will be documented in this file
 for human readability.
 
 
+## Changes when updating to v0.4 from v0.3.x
+
+#### Added
+
+#### Changed
+
+#### Deprecated
+
+#### Removed
+
+- Many initial/boundary conditions and source terms for typical setups were
+  moved from `Trixi/src` to the example elixirs `Trixi/examples`. Thus, they
+  are no longer available when `using Trixi`, e.g., the initial condition
+  for the Kelvin Helmholtz instability.
+- Some initial/boundary conditions and source terms for academic verification
+  setups were removed, e.g., `initial_condition_linear_x` for the 2D linear
+  advection equation.
+
+
 ## Changes in the v0.3 lifecycle
 
 #### Added

diff --git a/docs/src/testing.md b/docs/src/testing.md
@@ -32,13 +32,16 @@ executing
 julia> using Pkg; Pkg.test("Trixi")
 ```
 in the REPL. Since there already exist many tests, we have split them up into
-multiple files in the `test` directory to allow for faster testing of individual parts of the code.
+multiple files in the `test` directory to allow for faster testing of individual
+parts of the code.
 Thus in addition to performing all tests, you can also just `include` one of the
 files named `test_xxx.jl` to run only a specific subset, e.g.,
 ```julia
-julia> include(joinpath("test", "test_examples_2d.jl")) # Run all 2D tests
+julia> # Run all 2D tests on the P4estMesh
+       include(joinpath("test", "test_p4est_2d.jl"))
 
-julia> include(joinpath("test", "test_examples_2d_euler.jl")) # Run only 2D tests for the Euler equations
+julia> # Run all 1D tests for the Euler equations on the TreeMesh
+       include(joinpath("test", "test_tree_1d_euler.jl"))
 ```
 For the automated tests with GitHub Actions, we run multiple jobs in parallel to
 reduce the waiting time until all tests are finished. You can see the different
@@ -52,8 +55,8 @@ to configure tests. In general, newly added code must be covered by at least one
 test, and all new elixirs added to the `examples/` directory must be used at
 least once during testing. New tests should be added to the corresponding
 `test/test_xxx.jl` file, e.g., a test involving the 3D linear advection equation
-would go into
-[`test/test_examples_3d_advection.jl`](https://github.com/trixi-framework/Trixi.jl/blob/main/test/test_examples_3d_advection.jl).
+on the `TreeMesh` would go into
+[`test/test_tree_3d_advection.jl`](https://github.com/trixi-framework/Trixi.jl/blob/main/test/test_tree_3d_advection.jl).
 Please study one of the existing tests and stay consistent to the current style
 when creating new tests.
 
@@ -63,6 +66,11 @@ tests to pass with each novel feature added to Trixi. Therefore, new tests shoul
 short as reasonably possible, i.e., without being too insensitive to pick up
 changes or errors in the code.
 
+When you add new tests, please check whether all CI jobs still take approximately
+the same time. If the job where you added new tests takes much longer than
+everything else, please consider moving some tests from one job to another
+(or report this incident and ask the main developers for help).
+
 !!! note "Test duration"
     As a general rule, tests should last **no more than 10 seconds** when run
     with a single thread and after compilation (i.e., excluding the first run).

diff --git a/examples/p4est_2d_dgsem/elixir_advection_amr_solution_independent.jl b/examples/p4est_2d_dgsem/elixir_advection_amr_solution_independent.jl
@@ -118,9 +118,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -140,7 +137,7 @@ stepsize_callback = StepsizeCallback(cfl=1.6)
 
 callbacks = CallbackSet(summary_callback,
                         analysis_callback, alive_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         amr_callback, stepsize_callback);
 
 

diff --git a/examples/p4est_2d_dgsem/elixir_euler_source_terms_nonperiodic.jl b/examples/p4est_2d_dgsem/elixir_euler_source_terms_nonperiodic.jl
@@ -53,9 +53,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval)
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -65,7 +62,7 @@ stepsize_callback = StepsizeCallback(cfl=1.0)
 
 callbacks = CallbackSet(summary_callback,
                         analysis_callback, alive_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         stepsize_callback)
 ###############################################################################
 # run the simulation

diff --git a/examples/p4est_3d_dgsem/elixir_advection_amr.jl b/examples/p4est_3d_dgsem/elixir_advection_amr.jl
@@ -36,9 +36,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -58,7 +55,6 @@ stepsize_callback = StepsizeCallback(cfl=1.2)
 callbacks = CallbackSet(summary_callback,
                         analysis_callback,
                         alive_callback,
-                        save_restart,
                         save_solution,
                         amr_callback,
                         stepsize_callback)

diff --git a/examples/p4est_3d_dgsem/elixir_advection_cubed_sphere.jl b/examples/p4est_3d_dgsem/elixir_advection_cubed_sphere.jl
@@ -39,10 +39,6 @@ summary_callback = SummaryCallback()
 # The AnalysisCallback allows to analyse the solution in regular intervals and prints the results
 analysis_callback = AnalysisCallback(semi, interval=100)
 
-# The SaveRestartCallback allows to save a file from which a Trixi simulation can be restarted
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 # The SaveSolutionCallback allows to save the solution to a file in regular intervals
 save_solution = SaveSolutionCallback(interval=100,
                                      solution_variables=cons2prim)
@@ -51,7 +47,7 @@ save_solution = SaveSolutionCallback(interval=100,
 stepsize_callback = StepsizeCallback(cfl=1.2)
 
 # Create a CallbackSet to collect all callbacks such that they can be passed to the ODE solver
-callbacks = CallbackSet(summary_callback, analysis_callback, save_restart, save_solution, stepsize_callback)
+callbacks = CallbackSet(summary_callback, analysis_callback, save_solution, stepsize_callback)
 
 
 ###############################################################################

diff --git a/examples/p4est_3d_dgsem/elixir_euler_free_stream.jl b/examples/p4est_3d_dgsem/elixir_euler_free_stream.jl
@@ -87,9 +87,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval)
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -99,7 +96,7 @@ stepsize_callback = StepsizeCallback(cfl=1.2)
 
 callbacks = CallbackSet(summary_callback,
                         analysis_callback, alive_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         stepsize_callback)
 
 

diff --git a/examples/p4est_3d_dgsem/elixir_euler_source_terms_nonperiodic.jl b/examples/p4est_3d_dgsem/elixir_euler_source_terms_nonperiodic.jl
@@ -49,9 +49,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval)
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -61,7 +58,7 @@ stepsize_callback = StepsizeCallback(cfl=0.6)
 
 callbacks = CallbackSet(summary_callback,
                         analysis_callback, alive_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         stepsize_callback)
 
 

diff --git a/examples/paper_self_gravitating_gas_dynamics/elixir_euler_convergence.jl b/examples/paper_self_gravitating_gas_dynamics/elixir_euler_convergence.jl
@@ -36,15 +36,12 @@ save_solution = SaveSolutionCallback(interval=100,
                                      save_final_solution=true,
                                      solution_variables=cons2prim)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 analysis_interval = 100
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 analysis_callback = AnalysisCallback(semi, interval=analysis_interval)
 
 callbacks = CallbackSet(summary_callback, stepsize_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         analysis_callback, alive_callback)
 
 

diff --git a/examples/paper_self_gravitating_gas_dynamics/elixir_eulergravity_convergence.jl b/examples/paper_self_gravitating_gas_dynamics/elixir_eulergravity_convergence.jl
@@ -62,17 +62,14 @@ save_solution = SaveSolutionCallback(interval=10,
                                      save_final_solution=true,
                                      solution_variables=cons2prim)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 analysis_interval = 100
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
 analysis_callback = AnalysisCallback(semi_euler, interval=analysis_interval,
                                      save_analysis=true)
 
 callbacks = CallbackSet(summary_callback, stepsize_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         analysis_callback, alive_callback)
 
 

diff --git a/examples/paper_self_gravitating_gas_dynamics/elixir_eulergravity_sedov_blast_wave.jl b/examples/paper_self_gravitating_gas_dynamics/elixir_eulergravity_sedov_blast_wave.jl
@@ -87,19 +87,15 @@ save_solution = SaveSolutionCallback(interval=100,
                                      save_final_solution=true,
                                      solution_variables=cons2prim)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 analysis_interval = 100
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
 analysis_callback = AnalysisCallback(semi_euler, interval=analysis_interval,
                                      save_analysis=true,
                                      extra_analysis_integrals=(energy_total, energy_kinetic, energy_internal))
 
-# TODO: Taal decide, first AMR or save solution etc.
 callbacks = CallbackSet(summary_callback, amr_callback, stepsize_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         analysis_callback, alive_callback)
 
 

diff --git a/examples/structured_1d_dgsem/elixir_euler_source_terms.jl b/examples/structured_1d_dgsem/elixir_euler_source_terms.jl
@@ -37,9 +37,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -49,7 +46,7 @@ stepsize_callback = StepsizeCallback(cfl=0.8)
 
 callbacks = CallbackSet(summary_callback,
                         analysis_callback, alive_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         stepsize_callback)
 
 

diff --git a/examples/structured_2d_dgsem/elixir_advection_nonperiodic.jl b/examples/structured_2d_dgsem/elixir_advection_nonperiodic.jl
@@ -39,9 +39,6 @@ analysis_callback = AnalysisCallback(semi, interval=analysis_interval,
 
 alive_callback = AliveCallback(analysis_interval=analysis_interval)
 
-save_restart = SaveRestartCallback(interval=100,
-                                   save_final_restart=true)
-
 save_solution = SaveSolutionCallback(interval=100,
                                      save_initial_solution=true,
                                      save_final_solution=true,
@@ -51,7 +48,7 @@ stepsize_callback = StepsizeCallback(cfl=1.6)
 
 callbacks = CallbackSet(summary_callback,
                         analysis_callback, alive_callback,
-                        save_restart, save_solution,
+                        save_solution,
                         stepsize_callback);
 ###############################################################################
 # run the simulation