AbstractSchedules for scheduling output and diagnostics

.buildkite/pipeline.yml

@@ -15,8 +15,9 @@ steps:
       - "wget -N -P $SVERDRUP_HOME https://julialang-s3.julialang.org/bin/linux/x64/$JULIA_MINOR_VERSION/julia-$JULIA_VERSION-linux-x86_64.tar.gz"
       - "tar xf $SVERDRUP_HOME/julia-$JULIA_VERSION-linux-x86_64.tar.gz -C $SVERDRUP_HOME"
 
-      # Instantiate and precompile 
+      # Instantiate, build, and precompile 
       - "$SVERDRUP_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project -e 'using Pkg; Pkg.instantiate(; verbose=true)'"
+      - "$SVERDRUP_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project -e 'using Pkg; Pkg.build()'"
       - "$SVERDRUP_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project -e 'using Pkg; Pkg.precompile()'"
       - "$SVERDRUP_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project -e 'using Pkg; Pkg.status()'"
 

docs/src/library.md

@@ -168,6 +168,7 @@ Pages   = [
     "OutputWriters/output_writer_utils.jl",
     "OutputWriters/jld2_output_writer.jl",
     "OutputWriters/netcdf_output_writer.jl",
+    "OutputWriters/windowed_time_average.jl",
     "OutputWriters/checkpointer.jl"
 ]
 ```

docs/src/model_setup/checkpointing.md

@@ -14,12 +14,12 @@ end
 ```
 
 ```jldoctest
-julia> using Oceananigans.OutputWriters;
+julia> using Oceananigans.OutputWriters, Oceananigans.Utils
 
 julia> model = IncompressibleModel(grid=RegularCartesianGrid(size=(16, 16, 16), extent=(1, 1, 1)));
 
-julia> checkpointer = Checkpointer(model, time_interval=1000000, prefix="model_checkpoint")
-Checkpointer{Nothing,Int64,Array{Symbol,1}}(nothing, 1000000, 0.0, ".", "model_checkpoint", [:architecture, :grid, :clock, :coriolis, :buoyancy, :closure, :velocities, :tracers, :timestepper], false, false)
+julia> checkpointer = Checkpointer(model, schedule=TimeInterval(5years), prefix="model_checkpoint")
+Checkpointer{TimeInterval,Array{Symbol,1}}(TimeInterval(1.5768e8, 0.0), ".", "model_checkpoint", [:architecture, :grid, :clock, :coriolis, :buoyancy, :closure, :velocities, :tracers, :timestepper], false, false)
 ```
 
 The default options should provide checkpoint files that are easy to restore from in most cases. For more advanced

docs/src/model_setup/output_writers.md

@@ -1,20 +1,52 @@
 # Output writers
 
-Saving model data to disk can be done in a flexible manner using output writers. The two main output writers currently
-implemented are a NetCDF output writer (relying on [NCDatasets.jl](https://github.com/Alexander-Barth/NCDatasets.jl))
-and a JLD2 output writer (relying on [JLD2.jl](https://github.com/JuliaIO/JLD2.jl)).
+`AbstractOutputWriter`s save data to disk.
+`Oceananigans` provides three ways to write output:
 
-Output writers are stored as a list of output writers in `simulation.output_writers`. Output writers can be specified
-at model creation time or be specified at any later time and appended (or assigned with a key value pair) to
-`simulation.output_writers`.
+1. [`NetCDFOutputWriter`](@ref) for output of arrays and scalars that uses [NCDatasets.jl](https://github.com/Alexander-Barth/NCDatasets.jl)
+2. [`JLD2OutputWriter`](@ref) for arbitrary julia data structures that uses [JLD2.jl](https://github.com/JuliaIO/JLD2.jl)
+3. [`Checkpointer`](@ref) that automatically saves as much model data as possible, using [JLD2.jl](https://github.com/JuliaIO/JLD2.jl)
+
+The `Checkpointer` is discussed on a separate documentation page.
+
+## Basic usage
+
+[`NetCDFOutputWriter`](@ref) and [`JLD2OutputWriter`](@ref) require four inputs:
+
+1. The `model` from which output data is sourced (required to initialize the `OutputWriter`).
+2. A key-value pairing of output "names" and "output" objects. `JLD2OutputWriter` accepts `NamedTuple`s and `Dict`s;
+   `NetCDFOutputWriter` accepts `Dict`s with string-valued keys. Output objects are either `AbstractField`s or
+   functions that return data when called via `func(model)`.
+3. A `schedule` on which output is written. `TimeInterval`, `IterationInterval`, `WallTimeInterval` schedule
+   periodic output according to the simulation time, simulation interval, or "wall time" (the physical time
+   according to a clock on your wall). A fourth `schedule` called `AveragedTimeInterval` specifies
+   periodic output that is time-averaged over a `window` prior to being written.
+4. The filename and directory. Currently `NetCDFOutputWriter` accepts one `filepath` argument, while
+   `JLD2OutputWriter` accepts a filename `prefix` and `dir`ectory.
+
+Other important keyword arguments are
+
+* `field_slicer::FieldSlicer` for outputting subregions, two- and one-dimensional slices of fields.
+  By default a `FieldSlicer` is used to remove halo regions from fields so that only the physical
+  portion of model data is saved to disk.
+
+* `array_type` for specifying the type of the array that holds outputted field data. The default is
+  `Array{Float32}`, or arrays of single-precision floating point numbers.
+
+Once an `OutputWriter` is created, it can be used to write output by adding it the
+ordered dictionary `simulation.output_writers`. prior to calling `run!(simulation)`.
+
+More specific detail about the `NetCDFOutputWriter` and `JLD2OutputWriter` is given below.
 
 ## NetCDF output writer
 
 Model data can be saved to NetCDF files along with associated metadata. The NetCDF output writer is generally used by
 passing it a dictionary of (label, field) pairs and any indices for slicing if you don't want to save the full 3D field.
 
-The following example shows how to construct NetCDF output writers for two different kinds of outputs (3D fields and
-slices) along with output attributes
+### Examples
+
+Saving the u velocity field and temperature fields, the full 3D fields and surface 2D slices
+to separate NetCDF files:
 
 ```jldoctest netcdf1
 using Oceananigans, Oceananigans.OutputWriters
@@ -28,26 +60,48 @@ simulation = Simulation(model, Δt=12, stop_time=3600);
 fields = Dict("u" => model.velocities.u, "T" => model.tracers.T);
 
 simulation.output_writers[:field_writer] =
-    NetCDFOutputWriter(model, fields, filepath="output_fields.nc", time_interval=60)
+    NetCDFOutputWriter(model, fields, filepath="more_fields.nc", schedule=TimeInterval(60))
 
 # output
-NetCDFOutputWriter (time_interval=60): output_fields.nc
+NetCDFOutputWriter scheduled on TimeInterval(1 minute):
+├── filepath: more_fields.nc
 ├── dimensions: zC(16), zF(17), xC(16), yF(16), xF(16), yC(16), time(0)
-└── 2 outputs: ["T", "u"]
+├── 2 outputs: ["T", "u"]
+├── field slicer: FieldSlicer(:, :, :, with_halos=false)
+└── array type: Array{Float32}
 ```
-
 ```jldoctest netcdf1
 simulation.output_writers[:surface_slice_writer] =
-    NetCDFOutputWriter(model, fields, filepath="output_surface_xy_slice.nc",
-                       time_interval=60, field_slicer=FieldSlicer(k=grid.Nz))
+    NetCDFOutputWriter(model, fields, filepath="another_surface_xy_slice.nc",
+                       schedule=TimeInterval(60), field_slicer=FieldSlicer(k=grid.Nz))
 
 # output
-NetCDFOutputWriter (time_interval=60): output_surface_xy_slice.nc
+NetCDFOutputWriter scheduled on TimeInterval(1 minute):
+├── filepath: another_surface_xy_slice.nc
 ├── dimensions: zC(1), zF(1), xC(16), yF(16), xF(16), yC(16), time(0)
-└── 2 outputs: ["T", "u"]
+├── 2 outputs: ["T", "u"]
+├── field slicer: FieldSlicer(:, :, 16, with_halos=false)
+└── array type: Array{Float32}
 ```
 
-Writing a scalar, profile, and slice to NetCDF:
+```jldoctest netcdf1
+simulation.output_writers[:averaged_profile_writer] =
+    NetCDFOutputWriter(model, fields,
+                       filepath = "another_averaged_z_profile.nc",
+                       schedule = AveragedTimeInterval(60, window=20),
+                       field_slicer = FieldSlicer(i=1, j=1))
+
+# output
+NetCDFOutputWriter scheduled on TimeInterval(1 minute):
+├── filepath: another_averaged_z_profile.nc
+├── dimensions: zC(16), zF(17), xC(1), yF(1), xF(1), yC(1), time(0)
+├── 2 outputs: ["T", "u"] averaged on AveragedTimeInterval(window=20 seconds, stride=1, interval=1 minute)
+├── field slicer: FieldSlicer(1, 1, :, with_halos=false)
+└── array type: Array{Float32}
+```
+
+`NetCDFOutputWriter` also accepts output functions that write scalars and arrays to disk,
+provided that their `dimensions` are provided:
 
 ```jldoctest
 using Oceananigans, Oceananigans.OutputWriters
@@ -59,9 +113,7 @@ model = IncompressibleModel(grid=grid);
 simulation = Simulation(model, Δt=1.25, stop_iteration=3);
 
 f(model) = model.clock.time^2; # scalar output
-
 g(model) = model.clock.time .* exp.(znodes(Cell, grid)); # vector/profile output
-
 h(model) = model.clock.time .* (   sin.(xnodes(Cell, grid, reshape=true)[:, :, 1])
                             .*     cos.(ynodes(Face, grid, reshape=true)[:, :, 1])); # xy slice output
 
@@ -77,54 +129,139 @@ output_attributes = Dict(
 
 global_attributes = Dict("location" => "Bay of Fundy", "onions" => 7);
 
-simulation.output_writers[:stuff] =
+simulation.output_writers[:things] =
     NetCDFOutputWriter(model, outputs,
-                       iteration_interval=1, filepath="stuff.nc", dimensions=dims, verbose=true,
+                       schedule=IterationInterval(1), filepath="some_things.nc", dimensions=dims, verbose=true,
                        global_attributes=global_attributes, output_attributes=output_attributes)
 
 # output
-NetCDFOutputWriter (iteration_interval=1): stuff.nc
+NetCDFOutputWriter scheduled on IterationInterval(1):
+├── filepath: some_things.nc
 ├── dimensions: zC(16), zF(17), xC(16), yF(16), xF(16), yC(16), time(0)
-└── 3 outputs: ["profile", "slice", "scalar"]
+├── 3 outputs: ["profile", "slice", "scalar"]
+├── field slicer: FieldSlicer(:, :, :, with_halos=false)
+└── array type: Array{Float32}
 ```
 
-See [`NetCDFOutputWriter`](@ref) for more details and options.
+See [`NetCDFOutputWriter`](@ref) for more information.
 
 ## JLD2 output writer
 
-JLD2 is a an HDF5 compatible file format written in pure Julia and is generally pretty fast. JLD2 files can be opened in
-Python with the [h5py](https://www.h5py.org/) package.
+JLD2 is a fast HDF5 compatible file format written in pure Julia.
+JLD2 files can be opened in Julia with the [JLD2.jl](https://github.com/JuliaIO/JLD2.jl) package
+and in Python with the [h5py](https://www.h5py.org/) package.
 
-The JLD2 output writer is generally used by passing it a dictionary or named tuple of (label, function) pairs where the
-functions have a single input `model`. Whenever output needs to be written, the functions will be called and the output
-of the function will be saved to the JLD2 file. For example, to write out 3D fields for w and T and a horizontal average
-of T every 1 hour of simulation time to a file called `some_data.jld2`
+The `JLD2OutputWriter` receives either a `Dict`ionary or `NamedTuple` containing
+`name, output` pairs. The `name` can be a symbol or string. The `output` must either be
+an `AbstractField` or a function called with `func(model)` that returns arbitrary output.
+Whenever output needs to be written, the functions will be called and the output
+of the function will be saved to the JLD2 file.
 
-```julia
-using Oceananigans
-using Oceananigans.OutputWriters
+### Examples
+
+Write out 3D fields for w and T and a horizontal average:
+```jldoctest jld2_output_writer
+using Oceananigans, Oceananigans.OutputWriters, Oceananigans.Fields
 using Oceananigans.Utils: hour, minute
 
-model = IncompressibleModel(grid=RegularCartesianGrid(size=(16, 16, 16), extent=(1, 1, 1)))
+model = IncompressibleModel(grid=RegularCartesianGrid(size=(1, 1, 1), extent=(1, 1, 1)))
+
 simulation = Simulation(model, Δt=12, stop_time=1hour)
 
-function init_save_some_metadata(file, model)
+function init_save_some_metadata!(file, model)
     file["author"] = "Chim Riggles"
     file["parameters/coriolis_parameter"] = 1e-4
     file["parameters/density"] = 1027
+    return nothing
 end
 
 T_avg =  AveragedField(model.tracers.T, dims=(1, 2))
 
-outputs = Dict(
-    :w => model -> model.velocities.u,
-    :T => model -> model.tracers.T,
-    :T_avg => model -> T_avg(model)
-)
+# Note that model.velocities is NamedTuple
+simulation.output_writers[:velocities] = JLD2OutputWriter(model, model.velocities,
+                                                          prefix = "some_more_data",
+                                                          schedule = TimeInterval(20minute),
+                                                          init = init_save_some_metadata!)
+
+# output
+JLD2OutputWriter scheduled on TimeInterval(20 minutes):
+├── filepath: ./some_more_data.jld2
+├── 3 outputs: (:u, :v, :w)
+├── field slicer: FieldSlicer(:, :, :, with_halos=false)
+├── array type: Array{Float32}
+├── including: [:grid, :coriolis, :buoyancy, :closure]
+└── max filesize: Inf YiB
+```
+
+and a time- and horizontal-average of temperature `T` every 1 hour of simulation time
+to a file called `some_averaged_data.jld2`
+
+```jldoctest jld2_output_writer
+simulation.output_writers[:avg_T] = JLD2OutputWriter(model, (T=T_avg,),
+                                                     prefix = "some_more_averaged_data",
+                                                     schedule = AveragedTimeInterval(20minute, window=5minute))
+# output
+JLD2OutputWriter scheduled on TimeInterval(20 minutes):
+├── filepath: ./some_more_averaged_data.jld2
+├── 1 outputs: (:T,) averaged on AveragedTimeInterval(window=5 minutes, stride=1, interval=20 minutes)
+├── field slicer: FieldSlicer(:, :, :, with_halos=false)
+├── array type: Array{Float32}
+├── including: [:grid, :coriolis, :buoyancy, :closure]
+└── max filesize: Inf YiB
+```
+
+See [`JLD2OutputWriter`](@ref) for more information.
 
-jld2_writer = JLD2OutputWriter(model, outputs, init=init_save_some_metadata, interval=20minute, prefix="some_data")
+## Time-averaged output
 
-push!(simulation.output_writers, jld2_writer)
+Time-averaged output is specified by setting the `schedule` keyword argument for either `NetCDFOutputWriter` or
+`JLD2OutputWriter` to [`AveragedTimeInterval`](@ref).
+
+With `AveragedTimeInterval`, the time-average of ``a`` is taken as a left Riemann sum corresponding to
+
+```math
+\langle a \rangle = \frac{1}{T} \int_{t_i-T}^{t_i} a \, \mathrm{d} t \, ,
 ```
+where $\langle a \rangle$ is the time-average of $a$, $T$ is the time-`window` for averaging specified by
+the `window` keyword argument to `AveragedTimeInterval`, and the $t_i$ are discrete times separated by the
+time `interval`. The $t_i$ specify both the end of the averaging window and the time at which output is written.
+
+### Example
+
+Building an `AveragedTimeInterval` that averages over a 1 year window, every 4 years,
+
+```jldoctest averaged_time_interval
+using Oceananigans.OutputWriters: AveragedTimeInterval
+using Oceananigans.Utils: year, years
+
+schedule = AveragedTimeInterval(4years, window=1year)
 
-See [`JLD2OutputWriter`](@ref) for more details and options.
+# output
+AveragedTimeInterval(window=1 year, stride=1, interval=4 years)
+```
+
+An `AveragedTimeInterval` schedule directs an output writer
+to time-average its outputs before writing them to disk:
+
+```jldoctest averaged_time_interval
+using Oceananigans
+using Oceananigans.OutputWriters: JLD2OutputWriter
+using Oceananigans.Utils: minutes
+
+model = IncompressibleModel(grid=RegularCartesianGrid(size=(1, 1, 1), extent=(1, 1, 1)))
+
+simulation = Simulation(model, Δt=10minutes, stop_time=30years)
+
+simulation.output_writers[:velocities] = JLD2OutputWriter(model, model.velocities,
+                                                          prefix = "even_more_averaged_velocity_data",
+                                                          schedule = AveragedTimeInterval(4years, window=1year, stride=2))
+
+# output
+JLD2OutputWriter scheduled on TimeInterval(4 years):
+├── filepath: ./even_more_averaged_velocity_data.jld2
+├── 3 outputs: (:u, :v, :w) averaged on AveragedTimeInterval(window=1 year, stride=2, interval=4 years)
+├── field slicer: FieldSlicer(:, :, :, with_halos=false)
+├── array type: Array{Float32}
+├── including: [:grid, :coriolis, :buoyancy, :closure]
+└── max filesize: Inf YiB
+```

examples/eady_turbulence.jl

@@ -287,12 +287,12 @@ nothing # hide
 # we create a `JLD2OutputWriter` that saves `ζ` and `δ` and add it to 
 # `simulation`.
 
-using Oceananigans.OutputWriters: JLD2OutputWriter
+using Oceananigans.OutputWriters: JLD2OutputWriter, TimeInterval
 
 simulation.output_writers[:fields] = JLD2OutputWriter(model, (ζ=ζ, δ=δ),
-                                                      time_interval = 2hour,
-                                                             prefix = "eady_turbulence",
-                                                              force = true)
+                                                      schedule = TimeInterval(2hour),
+                                                        prefix = "eady_turbulence",
+                                                         force = true)
 nothing # hide
 
 # All that's left is to press the big red button:

examples/internal_wave.jl

@@ -118,12 +118,12 @@ simulation = Simulation(model, Δt = 0.02 * 2π/ω, stop_iteration = 100)
 
 # and add an output writer that saves the vertical velocity field every two iterations:
 
-using Oceananigans.OutputWriters: JLD2OutputWriter
+using Oceananigans.OutputWriters: JLD2OutputWriter, IterationInterval
 
 simulation.output_writers[:velocities] = JLD2OutputWriter(model, model.velocities,
-                                                          iteration_interval = 2,
-                                                                      prefix = "internal_wave",
-                                                                       force = true)
+                                                          schedule = IterationInterval(2),
+                                                            prefix = "internal_wave",
+                                                             force = true)
 
 # With initial conditions set and an output writer at the ready, we run the simulation
 

examples/langmuir_turbulence.jl

@@ -215,7 +215,7 @@ using Oceananigans.Utils: minute
 fields_to_output = merge(model.velocities, model.tracers, (νₑ=model.diffusivities.νₑ,))
 
 simulation.output_writers[:fields] =
-    JLD2OutputWriter(model, fields_to_output, time_interval = 2minute,
+    JLD2OutputWriter(model, fields_to_output, schedule=TimeInterval(2minute),
                      prefix = "langmuir_turbulence", force = true)
 nothing # hide
 

examples/ocean_wind_mixing_and_convection.jl

@@ -129,7 +129,7 @@ nothing # hide
 
 ## Instantiate a JLD2OutputWriter to write fields. We will add it to the simulation before
 ## running it.
-field_writer = JLD2OutputWriter(model, fields_to_output; time_interval=hour/4,
+field_writer = JLD2OutputWriter(model, fields_to_output; schedule=TimeInterval(hour/4),
                                 prefix="ocean_wind_mixing_and_convection", force=true)
 
 # ## Running the simulation

examples/one_dimensional_diffusion.jl

@@ -87,11 +87,11 @@ plot!(p, interior(T)[1, 1, :], z, linewidth=2, label=tracer_label(model.clock.ti
 # Interesting! Next, we add an output writer that saves the temperature field
 # in JLD2 files, and run the simulation for longer so that we can animate the results.
 
-using Oceananigans.OutputWriters: JLD2OutputWriter
+using Oceananigans.OutputWriters: JLD2OutputWriter, IterationInterval
 
 simulation.output_writers[:temperature] =
     JLD2OutputWriter(model, model.tracers, prefix = "one_dimensional_diffusion",
-                     iteration_interval = 100, force = true)
+                     schedule=IterationInterval(100), force = true)
 
 ## Run simulation for 10,000 more iterations
 simulation.stop_iteration += 10000