Merge docu to main branch

README.md

@@ -69,17 +69,17 @@ However, you still need to clone the GrainLearning repository to run the tutoria
 ## Tutorials
 
 1. Linear regression with
-   the [`run_sim`](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/linear_regression/python_linear_regression_solve.py#L14)
+   the [`run_sim`](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/simple_regression/linear_regression/python_linear_regression_solve.py#L14)
    callback function of the [`DynamicSystem`](https://github.com/GrainLearning/grainLearning/blob/main/grainlearning/dynamic_systems.py)
    class,
-   in [python_linear_regression_solve.py](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/linear_regression/python_linear_regression_solve.py)
+   in [python_linear_regression_solve.py](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/simple_regression/linear_regression/python_linear_regression_solve.py)
 
 2. Nonlinear, multivariate regression
 
 3. Interact with the numerical model of your choice
-   via [`run_sim`](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/linear_regression/linear_regression_solve.py#L11)
+   via [`run_sim`](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/simple_regression/linear_regression/linear_regression_solve.py#L11)
    ,
-   in [linear_regression_solve.py](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/linear_regression/linear_regression_solve.py)
+   in [linear_regression_solve.py](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/simple_regression/linear_regression/linear_regression_solve.py)
 
 4. Load existing simulation data and run GrainLearning for one iteration,
    in [oedo_load_and_resample.py](https://github.com/GrainLearning/grainLearning/blob/main/tutorials/oedo_compression/oedo_load_and_resample.py)

docs/source/dynamic_systems.rst

@@ -4,15 +4,18 @@ Dynamic systems
 The dynamic system module
 -------------------------
 
-The :mod:`.dynamic_systems` module is essential for GrainLearning to execute computational models
-and encapsulate simulation and observation (reference) data in a single :class:`.DynamicSystem` class.
+The :mod:`.dynamic_systems` module is essential for GrainLearning to run the predictive model(s)
+and encapsulate simulation and observation (or reference) data in a single :class:`.DynamicSystem` class.
 Currently, the :mod:`.dynamic_systems` module contains
 
-- a :class:`.DynamicSystem` class that executes simulations handles the simulation and observation data in a *Python environment*,
-- an :class:`.IODynamicSystem` class that sends instructions to *third-party software* (from the command line) and retrieves simulation data from the output files of the software.
+- a :class:`.DynamicSystem` class that handles the simulation and observation data within a *Python environment*,
+- an :class:`.IODynamicSystem` class that sends instructions to external *third-party software* (e.g., via the command line) and retrieves simulation data from the output files of the software.
 
-Note that the dynamic system classes defined in GrainLearning are also known as state-space models
-that consist of the predicted states :math:`\vec{x}_t` (:attr:`.DynamicSystem.sim_data`) and experimental observables :math:`\vec{y}_t` (:attr:`.DynamicSystem.obs_data`).
+.. note:: A dynamic system is also known as a state-space model in the literature.
+  It describes the time evolution of the state of the model :math:`\vec{x}_t` (:attr:`.DynamicSystem.sim_data`)
+  and the state of the observables :math:`\vec{y}_t` (:attr:`.DynamicSystem.obs_data`).
+  Both :math:`\vec{x}_t` and :math:`\vec{y}_t` are random variables
+  whose distributions are updated by the :mod:`.inference` module.
 
 .. math::
 
@@ -23,46 +26,63 @@ that consist of the predicted states :math:`\vec{x}_t` (:attr:`.DynamicSystem.si
 	\label{eq:obsModel}
 	\end{align}
 
-where :math:`\mathbb{F}` represents the *third-party software* model that
+where :math:`\mathbb{F}` represents the **third-party software** model that
 takes the previous model state :math:`\vec{x}_{t-1}` to make predictions for time :math:`t`. 
-If all observables :math:`\vec{y}_t` are independent and have a one-to-one relationship with :math:`\vec{y}_t`,
-the observation model :math:`\mathbb{H}` reduces to the identity matrix :math:`\mathbf{I}_d`, 
+If all observables :math:`\vec{y}_t` are independent and have a one-to-one correspondence with :math:`\vec{x}_t`,
+(meaning you predict what you observe),
+the observation model :math:`\mathbb{H}` reduces to the identity matrix :math:`\mathbb{I}_d`, 
 with :math:`d` being the number of independent observables.
 
 The simulation and observation errors :math:`\vec{\nu}_t` and :math:`\vec{\omega}_t`
-are random variables and assumed to be normally distributed with zero means.
-We consider both errors together in the covariance matrix with :attr:`.SMC.cov_matrices`.
-In fact, :math:`\vec{x}_t` and :math:`\vec{y}_t` are also random variables
-whose distributions are updated by the :mod:`.inference` module.
+are random variables and assumed to be normally distributed around zero means.
+We consider both errors together in the covariance matrix :attr:`.SMC.cov_matrices`.
 
-Interact with third-party software
-----------------------------------
+Interact with third-party software via callback function
+--------------------------------------------------------
 
-Interaction with external "software" models can be done via the callback of the :class:`.DynamicSystem` class.
-You can define your own :attr:`.DynamicSystem.callback`
-and pass samples (combinations of parameters) to the **model implemented in Python** or to the software from the **command line**.
-The following gives an example of the callback where the "software" model :math:`\mathbb{F}` is a Python function. 
+Interaction with an external "software" model can be done via the callback function of :class:`.DynamicSystem` or :class:`.IODynamicSystem`.
+You can define your own callback function
+and pass *samples* (combinations of parameters) to the **model implemented in Python** or to the software from the **command line**.
+The figure below shows how the callback function is called in the execution loop of :class:`.BayesianCalibration`. 
+
+.. _execution_loop:
+.. image:: ./figs/execution_loop.png
+  :width: 400
+  :alt: How a callback function gets executed
+
+Interact with Python software
+`````````````````````````````
+
+Let us first look at an example where the predictive model :math:`\mathbb{F}` is implemented in Python.
+The following code snippet shows how to define a callback function that runs a linear model. 
 
 .. code-block:: python
-   :caption: A callback function implemented in Python
+   :caption: A linear function implemented in Python
 
-   def run_sim(system: Type["DynamicSystem"], **kwargs):
+   def run_sim(system, **kwargs):
        data = []
+       # loop over parameter samples
        for params in system.param_data:
-           # a linear function y = a*x + b
+           # Run the model: y = a*x + b
            y_sim = params[0] * system.ctrl_data + params[1]
+           # Append the simulation data to the list
            data.append(np.array(y_sim, ndmin=2))
-       # assign model output to the dynamic system class
-       model.set_sim_data(data) 
+       # pass the simulation data to the dynamic system
+       system.set_sim_data(data)
+
+
+The function `run_sim` is assigned to the :attr:`.DynamicSystem.callback` attribute of the :class:`.DynamicSystem` class
+and is is called every time the :attr:`.DynamicSystem.run` function is called (see :ref:`the figure <execution_loop>` above).
 
 
-The IODynamicSystem class
-`````````````````````````
+Interact with non-Python software
+`````````````````````````````````
 
-The :class:`.IODynamicSystem` inherits from :class:`.DynamicSystem` and is intended to work with third-party software packages.
-The :attr:`.IODynamicSystem.run` function overrides the :attr:`.DynamicSystem.run` function of the :class:`.DynamicSystem`.
-to write samples in a text file and run the :attr:`.IODynamicSystem.callback` to execute the third-party software model at all sample points.
-Below is an example of the callback where parameter samples are passed as command-line arguments to an external executable.
+The :class:`.IODynamicSystem` class inherits from :class:`.DynamicSystem` and is intended to work with external software packages
+via the command line.
+The :attr:`.IODynamicSystem.run` function overrides the :attr:`.DynamicSystem.run` function of the :class:`.DynamicSystem` class.
+Parameter samples are written into a text file and used by :attr:`.IODynamicSystem.callback` to execute the third-party software.
+Users only need to write a for-loop to pass each parameter sample to this external software, e.g., as command-line arguments (see the example below).
 
 .. code-block:: python
    :caption: A callback function that interacts with external software
@@ -77,21 +97,27 @@ Below is an example of the callback where parameter samples are passed as comman
        curr_iter = kwargs['curr_iter']
        # loop over and pass parameter samples to the executable
        for i, params in enumerate(system.param_data):
-           description = 'Iter'+str(curr_iter)+'-Sample'+str(i).zfill(mag)
-           print(" ".join([executable, '%.8e %.8e'%tuple(params), description]))
+           description = 'Iter'+str(curr_iter)+'_Sample'+str(i).zfill(mag)
            os.system(' '.join([executable, '%.8e %.8e'%tuple(params), description]))
 
 
+.. note:: This code snippet can be used as a template to interact with any third-party software.
+  The only thing you need to do is to replace the executable name and the command-line arguments.
+  The command-line arguments are passed to the software in the order of the parameter names in :attr:`.IODynamicSystem.param_names`.
+  The last argument (optional) is a description of the current simulation, which is used to tag the output files.
+  In this example, the description is `Iter<curr_iter>_Sample<sample_ID>`.
+  The output files are read into :attr:`.IODynamicSystem.sim_data` by the function :attr:`.IODynamicSystem.load_sim_data`.
+
 Data format and directory structure
-```````````````````````````````````
+:::::::::::::::::::::::::::::::::::
 
-GrainLearning can read .npy (for backward compatibility) and plain text formats.
+GrainLearning can read plain text and .npy formats (for backward compatibility).
 When using :class:`.IODynamicSystem`, the directory :attr:`.IODynamicSystem.sim_data_dir` must exist and contains the observation data file :attr:`.IODynamicSystem.obs_data_file`.
 Subdirectories with name `iter<curr_iter>` will be created in :attr:`.IODynamicSystem.sim_data_dir`.
 In these subdirectories, you find
 
-- simulation data file: `<sim_name>_Iter<curr_iter>-Sample<sample_ID>_sim.txt`
-- parameter data file: `<sim_name>_Iter<curr_iter>-Sample<sample_ID>_param.txt`,
+- simulation data file: `<sim_name>_Iter<curr_iter>_Sample<sample_ID>_sim.txt`
+- parameter data file: `<sim_name>_Iter<curr_iter>_Sample<sample_ID>_param.txt`,
 
 where <sim_name> is :attr:`.IODynamicSystem.sim_name`, <curr_iter> is :attr:`.BayesianCalibration.curr_iter`,
 and <sample_ID> is the index of the :attr:`.IODynamicSystem.param_data` sequence.
@@ -101,34 +127,32 @@ For example, the observation data stored in a text file :attr:`.IODynamicSystem.
 .. code-block:: text
 
 	# u f
-	0		5.0
-	1		5.2
-	2		5.4
-	3		5.6
-	4		5.8
-	5		6.0
+	0	5.0
+	1	5.2
+	2	5.4
+	3	5.6
+	4	5.8
+	5	6.0
 
-Similarly, in a simulation data file `linear_Iter0-Sample00_sim.txt`, you find
+Similarly, in a simulation data file `linear_Iter0_Sample00_sim.txt`, you will find
 
 .. code-block:: text
 
 	# f
-	0.741666667
-	1.023635417
-	1.3056041669999998
-	1.587572917
-	1.869541667
-	2.151510417
-
-Note the simulation data doesn't contain the :attr:`DynamicSystem.ctrl_data` sequence.
-
-Therefore, when using :class:`.IODynamicSystem` the user needs to provide the keys to the data sequence
-of the **control** and **observation** group.
-These keys are also used to extract the corresponding data from the simulation data files.
-
-.. code-block:: python
-
-    # name of the control variable
-    "ctrl_name": 'u',
-    # name of the output variables of the model
-    "obs_names": ['f'],
+	5.0
+	5.2
+	5.4
+	5.6
+	5.8
+	6.0
+
+.. note:: The simulation data doesn't contain the sequence of :attr:`DynamicSystem.ctrl_data` at which the outputs are stored.
+  Therefore, when initializing :class:`.IODynamicSystem` the user needs to provide the keys to the data sequences
+  that belong to the **control** and the **observation** group.
+
+  .. code-block:: python
+
+      # name of the control variable
+      "ctrl_name": 'u',
+      # name of the output variables of the model
+      "obs_names": ['f'],

docs/source/examples.rst

@@ -18,6 +18,15 @@ for running the simulations, the "trials" have to be selected with an optimized
 Bayesian calibration of DEM models
 ----------------------------------
 
+
+
+Run DEM simulations guided by GrainLearning
+```````````````````````````````````````````
+
+
+Process simulation data with GrainLearning
+``````````````````````````````````````````
+
 In the case of DEM modeling of granular soils, relevant parameters could be
 Young's modulus, friction coefficient, Poisson's ratio, rolling stiffness, and rolling friction, etc.
 of a soil particle, as well as structural parameters like a particle size distribution parameterized by its moments.

docs/source/index.rst

@@ -6,18 +6,17 @@
 Welcome to GrainLearning's documentation!
 =========================================
 
-GrainLearning is a Bayesian uncertainty quantification and propagation toolbox for computer simulations
-of granular materials.
-The software is primarily used to infer and quantify parameter uncertainties in computational models of granular materials
-from observation data, which is also known as inverse analyses or data assimilation. 
-Implemented in Python, GrainLearning can be loaded into a Python environment to process the simulation and observation data,
-or alternatively, as an independent tool where simulation runs are done separately, e.g., via a shell script.
-
-If you use GrainLearning, please cite `the version of the GrainLearning software you used <https://zenodo.org/record/7123966>`_ and the following paper:
-
-H. Cheng, T. Shuku, K. Thoeni, P. Tempone, S. Luding, V. Magnanimo.
-An iterative Bayesian filtering framework for fast and automated calibration of DEM models. *Comput. Methods Appl. Mech. Eng., 350 (2019)*, pp. 268-294,
-`10.1016/j.cma.2019.01.027 <https://doi.org/10.1016/j.cma.2019.01.027>`_
+GrainLearning is a Bayesian uncertainty quantification toolbox for computer simulations of granular materials.
+The software is primarily used to infer model parameter distributions from observation or reference data,
+a process also known as inverse analyses or data assimilation. 
+Implemented in Python, GrainLearning can be loaded into a Python environment to process your simulation and observation data,
+or alternatively, used as an independent tool where simulations are run separately, e.g., from the command line.
+
+If you use GrainLearning, please cite `the version of the GrainLearning software you used <https://zenodo.org/record/7123966>`_.
+If you want to know more about how the method works, the following papers can be interesting:
+
+- H. Cheng, T. Shuku, K. Thoeni, P. Tempone, S. Luding, V. Magnanimo. An iterative Bayesian filtering framework for fast and automated calibration of DEM models. *Comput. Methods Appl. Mech. Eng., 350 (2019)*, pp. 268-294, `10.1016/j.cma.2019.01.027 <https://doi.org/10.1016/j.cma.2019.01.027>`_
+- P. Hartmann, H. Cheng, K. Thoeni. Performance study of iterative Bayesian filtering to develop an efficient calibration framework for DEM. *Computers and Geotechnics 141*, 104491,  `10.1016/j.compgeo.2021.104491 <https://doi.org/10.1016/j.compgeo.2021.104491>`_
 
 .. toctree::
    :maxdepth: 2

docs/source/installation.rst

@@ -105,7 +105,7 @@ Online
 
 You can check the online documentation `here <https://grainlearning.readthedocs.io/en/latest/>`_.
 
-Build the documentation
+Build the documentation locally
 ```````````````````````
 
 .. code-block:: bash