[build-tex] Sync main and fork version (#94)

* Add figure for discrete adjoints * [build-tex] Quantum application and comments on symbolic diff Co-authored-by: frankschae <[email protected]> Co-authored-by: facusapienza21 <facusapienza21> * Add workflow distpatch * add workflow distpatch * [build-bib] * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * Updates from Overleaf * [build-bib] Section on Symbolic Differentiation * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * [build-bib] Changes in AD VJP connection * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * Updates from Overleaf * small changes in introduction * updated figure on VJP * Mention on Stressen algorithm for matrix multiplication * [build-bib] Reorganize scientific motivation section * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * [build-bib] Reorganization of adjoints and software + appendix on stepsizing * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * Small push to sync with main * Updates from Overleaf * Small changes and looking for large marge with central Co-authored-by: heimbach <[email protected]> * [build-bib] trigger bib compilation * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * [build-bib] * 📁 Automated biblatex merge [build-latex] Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> * Restore old bibliography --------- Signed-off-by: My GitHub Actions Bot <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: frankschae <[email protected]> Co-authored-by: Author <[email protected]> Co-authored-by: heimbach <[email protected]>
ODINN-SciML · Jan 23, 2024 · 95df75f · 95df75f
1 parent 3cd1bef
commit 95df75f
Show file tree

Hide file tree

Showing 8 changed files with 31 additions and 21 deletions.
diff --git a/tex/bibliography.bib b/tex/bibliography.bib
@@ -1610,4 +1610,4 @@ @article{Ranocha_Dalcin_Parsani_Ketcheson_2022
     journal = "Communications on Applied Mathematics and Computation",
     year = "2022",
     pages = "1191–1228"
-}
+}
diff --git a/tex/main.tex b/tex/main.tex
@@ -17,7 +17,17 @@
 \usepackage[hidelinks]{hyperref} % hidelinks added to not display red box for links
 \usepackage{algorithm}
 \usepackage{enumitem} % Necesarry for enumerating with romans (i), (ii), ...
-\usepackage{todonotes} % Taking notes
+% \usepackage{todonotes} % Taking notes
+
+\usepackage{titlesec}
+
+\setcounter{tocdepth}{4}
+\setcounter{secnumdepth}{4}
+
+\titleformat{\paragraph}
+{\normalfont\small\bfseries}{\theparagraph}{1em}{}
+\titlespacing*{\paragraph}
+{0pt}{3.25ex plus 1ex minus .2ex}{1.5ex plus .2ex}
 
 \usepackage{titlesec}
 
@@ -48,8 +58,10 @@
 % add a new custom bib file inside ./bibs/
 \addbibresource{bibliography.bib}
 % Alternative bib source
-% \addbibresource{bibs/bibliography-heimbach.bib}
+% \addbibresource{bibs/bilbiography-victor.bib}
 % \addbibresource{bibs/bibliography-facu.bib}
+% \addbibresource{bibs/bibliography-heimbach.bib}
+
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %%%%%%%%%%%%% Document %%%%%%%%%%%%
@@ -70,7 +82,7 @@
 \input{sections/abstract}
 \end{abstract}
 
-\vspace{250px}
+\vspace{200px}
 \input{sections/community-statement}
 
 \normalsize
@@ -84,7 +96,6 @@ \section*{Plain language summary}
 \input{sections/plain}
 
 \section{Introduction}
-\todo[inline]{I would argue that the introduction should actually be the scientific motivation}
 \input{sections/introduction}
 
 \section{Scientific motivation}
@@ -147,9 +158,10 @@ \section*{Appendices}
 \addcontentsline{toc}{section}{Appendices}
 \renewcommand{\thesubsection}{\Alph{subsection}}
 
-\section{Statistical foundations of inverse modelling}
-\input{sections/statistical-foundations}
-
+\subsection{Statistical foundations of inverse modelling}
+% Check how to do this input correctly
+% \input{sections/statistical-foundations}
+\newpage
 \subsection{Lagrangian derivation of adjoints}
 \input{appendix/lagrangian}
 \newpage

diff --git a/tex/sections/adjoint-continuous.tex b/tex/sections/adjoint-continuous.tex
@@ -92,7 +92,7 @@ \subsubsection{Are discrete and continuous adjoint the same?}
 Different works exist regarding the consistency or inconsistency between the two approaches, and this usually depends on the ODE and equation.  
 Proofs of the consistency of discrete adjoint methods for Runge-Kutta methods have been provided in \cite{sandu2006properties, sandu2011solution}.
 Depending on the choice of the Runge-Kutta coefficients, we can have a numerical scheme that is both consistent for the original equation and consistent/inconsistent for the adjoint \cite{Hager_2000}.
-Furthermore, adjoint methods can be completely fail in chaotic systems \cite{Wang2012-chaos-adjoint}.
+Furthermore, adjoint methods can fail in chaotic systems \cite{Wang2012-chaos-adjoint}.
 
 In the more general case, both methods can give different computational results \cite{Sirkes_Tziperman_1997}.
 In \cite{Sirkes_Tziperman_1997} it is shown that discrete adjoints can lead to unstabilites and wrong gradients.
diff --git a/tex/sections/complex-step.tex b/tex/sections/complex-step.tex
@@ -31,7 +31,8 @@
 Although these methods seem similar, it is important to remark that AD gives the exact gradient, while complex step differentiation relies on numerical approximations that are valid just when the stepsize $\varepsilon$ is small. 
 In Figure \ref{fig:complex-step-AD} we show how the calculation of the gradient of $\sin (x^2)$ is performed by these two methods.
 While the second component of the dual number has the exact derivative of the function $\sin(x^2)$ with respect to $x$, it is not until we take $\varepsilon \rightarrow 0$ than we obtain the derivative in the imaginary component for the complex step method.
-The stepsize dependence of the complex step differentiation method makes it resemble more to finite differences than AD with dual numbers. 
+% The stepsize dependence of the complex step differentiation method makes it resemble more to finite differences than AD with dual numbers. 
+The dependence of the complex step differentiation method on the step size gives it a closer resemblance to finite difference methods than to AD using dual numbers.
 Further notice the complex step method involves more terms in the calculation, a consequence of the fact that second order terms of the form $i^2 = -1$ are transferred to the real part of the complex number, while for dual numbers the terms associated to $\epsilon^2 = 0$ vanish. 
 \begin{figure}[t]
     \centering

diff --git a/tex/sections/motivation.tex b/tex/sections/motivation.tex
@@ -113,14 +113,10 @@ \subsubsection{Geosciences}
 The same goals that have driven the use of sensitivity information in numerical weather prediction (optimal initial conditions for forecasts) or ocean science (state and parameter estimation) apply in the world of climate modeling.
 The recognition that ``good'' initial conditions (e.g., such that are closest to the real or observed system) will lead to improved forecasts on subseasonal, seasonal, interannual, or even decadal time scales has driven major community efforts (e.g., \cite{Meehl.2021}). However, there has been a lack so far in exploiting the use of gradient information to achieve optimal initialization for coupled Earth system models \cite{Frolov.2023}. 
 One conceptual challenge is the presence of multiple timescales in the coupled system and the utility of gradient information beyond many synoptic time scales in the atmosphere and ocean \cite{Lea:2000gv,Lea:2002cv}.
-
-TBD:
-
-Coupled GCM adjoint:
-\cite{Blessing.2014,Lyu.2018,Stammer:2018de}
-
-Neural AGCM:
-\cite{Kochkov.2023}
+Nevertheless, efforts are underway to enable adjoint-based parameter estimation of coupled atmosphere-ocean climate models, with AD again playing a crucial role in generating the corresponding adjoint model
+\cite{Blessing.2014,Lyu.2018,Stammer:2018de}.
+Complementary, recognizing the power of differentiable programming, efforts are also targeting the development of ``neural atmospheric general circulation models'' in JAX, which combine a differentiable dynamical core with neural operators as surrogate models of unresolved physics
+\cite{Kochkov.2023}.
 
 \paragraph{Transport modeling and flux inversion.}
 % ... GEOS-Chem , Kaminski, ...

diff --git a/tex/sections/plain.tex b/tex/sections/plain.tex
@@ -1,5 +1,6 @@
 Differential equations are mathematical tools that explicitly describe the processes and dynamics within various systems, based on prior knowledge. They are fundamental in many scientific disciplines for modeling phenomena such as physical processes, population dynamics, social interactions, and chemical reactions.
-\todo[inline]{here, it would be nice to explain how they differ to data-drive models. Consider:}
+% \todo[inline]{here, it would be nice to explain how they differ to data-drive models. Consider:}
+
 By contrast, data-driven models do not necessarily require a detailed understanding of the underlying processes, and learn patterns and relationships directly from data. Data-driven models are particularly useful in scenarios where the underlying processes are poorly understood or too complex to be captured by traditional mathematical models.
 The combination of mechanistic models with data-driven models is becoming increasingly common in many scientific domains. 
 In order to achieve this, these models need to leverage both domain knowledge and data, to have an accurate representation of the underlying dynamics. 

diff --git a/tex/sections/preliminaries.tex b/tex/sections/preliminaries.tex
@@ -30,7 +30,7 @@
     with $h$ being a function that quantifies the contribution of the error term at every time $t \in [t_0, t_1]$. 
     Defining a loss function where just the empirical error is penalized is known as trajectory matching. 
     Other methods like gradient matching and generalized smoothing the loss depends on smooth approximations of the trajectory and their derivatives. 
-    \todo{this is unclear}
+    % \todo{this is unclear}
     \item \textbf{Likelihood profiles.} From a statistical perspective, it is common to assume that observations correspond to noisy observations of the underlying dynamical system, $y_i = u(t_i; \theta) + \varepsilon_i$, with $\varepsilon_i$ errors or residual that are independent of each other and of the trajectory $u(\cdot ; \theta)$ \cite{ramsay2017dynamic}.
     If $p(y | t , \theta)$ is the probability distribution of $y$, maximum likelihood estimation consists in finding the parameter $\theta$ as
     \begin{equation}

diff --git a/tex/sections/recommendations.tex b/tex/sections/recommendations.tex
@@ -1,2 +1,2 @@
 For sufficient small systems of less than 100 parameters and ODEs, Forward AD is the most efficient method, outperforming sensitivity and adjoint methods \cite{ma_comparison_2021}.
-\todo[inline]{We should pay attention that this section does not overlap with \cite{ma_comparison_2021}.}
+% \todo[inline]{We should pay attention that this section does not overlap with \cite{ma_comparison_2021}.}