Code

Documentation for optimizer#89

Code

Merged

michakraus merged 163 commits intomainJuliaGNI/GeometricMachineLearning.jl:mainfrom documentation_for_optimizerJuliaGNI/GeometricMachineLearning.jl:documentation_for_optimizerCopy head branch name to clipboardDec 5, 2023

+2,399-712

Commits

Commits on Nov 10, 2023

• Copied Makefile from GI.

  benedict-96

  committed
  efad4e6

• Started adding bibliography.

  benedict-96

  committed
  c95c6d1

• Copied Makefile from GI.

  benedict-96

  committed
  7f80823

• Added more references (on symplectic integrators).

  benedict-96

  committed
  355ba99

• Started to rework this section. Fixed typos.

  benedict-96

  committed
  dca1b52

• Continued changing relevant bits and corrected typos. The file still uses old syntax.

  benedict-96

  committed
  bbe8c30

• Updated Hairer reference.

  benedict-96

  committed
  5823466

• Started adding descriptions to the various layers.

  benedict-96

  committed
  2776bc6

• Started refactoring this file. It still uses the old syntax.

  benedict-96

  committed
  daa8765

• Added routines such that the SympNet layers can now deal with NamedTuples.

  benedict-96

  committed
  7866c07

Commits on Nov 14, 2023

• File containing SympNet architecture.

  benedict-96

  committed
  bbddca5

• Added necessary tikz files.

  benedict-96

  committed
  0c86757

Commits on Nov 15, 2023

• Fixed typo.

  benedict-96

  committed
  1cf6ff7

• images -> tikz.

  benedict-96

  committed
  504e841

• Added (empty file) references.

  benedict-96

  committed
  fd4a232

Commits on Nov 18, 2023

• Added tikz picture that visualizes dependencies of various structs and types that are relevant for SympNets.

  benedict-96

  committed
  411c20a

• Updated file path.

  benedict-96

  committed
  a8e92fb

• Added the bibliography with DocumenterCitations.

  benedict-96

  committed
  7eed150

• Fixed some typos.

  benedict-96

  committed
  c3d365e

• Updated file path.

  benedict-96

  committed
  48ea7d3

• Added new files.

  benedict-96

  committed
  37b0444

• Updated pendulum script to reflect new data types.

  benedict-96

  committed
  9c840bc

• Updated SympNet architectures.:

  benedict-96

  committed
  2e82681

• Started adding the option of symplectic data to the DataLoader.

  benedict-96

  committed
  950565d

• Made layers work with symplectic data.

  benedict-96

  committed
  f543742

• Added more description and a routine that can be initialized with a NeuralNetwork.

  benedict-96

  committed
  704b39f

• Added some comments.

  benedict-96

  committed
  d3aa016

Commits on Nov 19, 2023

• Reorganized the DataLoader and Batch for NamedTuples.

  benedict-96

  committed
  4376166

• Fixed some typos in the code section.

  benedict-96

  committed
  998a1a9

• Extened routines to NamedTuples and fixed some typos.

  benedict-96

  committed
  772bd3a

Commits on Nov 20, 2023

• Forgot to output the batches before. Wrote a related comment into the test file.

  benedict-96

  committed
  b6bc346

• Changed data format so that the output is a matrix.

  benedict-96

  committed
  eb906cf

• Continued formatting script for new syntax.

  benedict-96

  committed
  8488428

• Removed empty line.

  benedict-96

  committed
  90baae2

• Updated TODO.:

  benedict-96

  committed
  7dda667

• Added Progressbar to optimization routine.

  benedict-96

  committed
  b901082

• Now saving time steps as T instead of T+1 (may have to think about possible conflict with autoencoder).

  benedict-96

  committed
  4032ba1

• Fixed typo for the outputs of the linear layers.

  benedict-96

  committed
  12b4ce8

Commits on Nov 21, 2023

• Added training routine for la sympnet.

  benedict-96

  committed
  93eafb7

• Added option to specify type.

  benedict-96

  committed
  9c0521d

• Specified type of input matrix.

  benedict-96

  committed
  4dbda54

• Added various routines that should also run on GPU including matrix-matrix multiplication and matrix-vector multiplication.

  benedict-96

  committed
  26fdacf

• Started refactoring these layers a bit such that fewer structs are needed.

  benedict-96

  committed
  356c33d

• Added a few separate tests for SymmetricMatrix.

  benedict-96

  committed
  f7b22c2

Commits on Nov 22, 2023

• Added an additional option to call the pendulum script with a NamedTuple for ics.

  benedict-96

  committed
  121b062

• Added an additional constructor s.t. SympNets can be called with DataLoader as input.

  benedict-96

  committed
  83b098d

• Updated the script s.t. calling sympnet now takes DataLoader as input and added more options for the pendulum script as well as a final conclusion.

  benedict-96

  committed
  8cbf85d

• Added a setindex! routine (needed for updating parameters).

  benedict-96

  committed
  8976d52

• put DataLoader under code comment marks.

  benedict-96

  committed
  29c7c2f

• made specifying the type of Adam an (easier) option.

  benedict-96

  committed
  90667c3

• Fixed a typo in the docs.

  benedict-96

  committed
  d9a5c39

• Updated SympNetLayers to save an extra struct and an abstract type. Stream-lined all the array routines.

  benedict-96

  committed
  366231a

• Minor refactoring to improve readability.

  benedict-96

  committed
  1a769b9

• Fixed the Makefiles s.t. they now correctly generate the pngs.

  benedict-96

  committed
  90e6097

• Reformulated various sections and fixed typos.

  benedict-96

  committed
  4ece4eb

• Added ENV line for correct plotting.

  benedict-96

  committed
  ebd35ac

• Updated according to new data structures in sympnets.jl

  benedict-96

  committed
  5af1ab9

• Updated file path to png.

  benedict-96

  committed
  1953f39

• Added comment.

  benedict-96

  committed
  e0fe713

• Split the theory and the tutorial sections up into two parts.

  benedict-96

  committed
  1008491

• Played around with the parameters a bit to make the LASympnet work (hopefully).

  benedict-96

  committed
  c45adf5

• Added the sympnet tutorial.

  benedict-96

  committed
  60673e7

• Added various cross-references and updated the tikz picture to use the same notation what is used in the text.

  benedict-96

  committed
  acdb284

• Changed the default values of the LA SympNet parameters.

  benedict-96

  committed
  87cfccc

Commits on Nov 23, 2023

• Updated the makefile to include the new logos.

  benedict-96

  committed
  acdaf0b

• Replaced the name with our logo (also dark mode).

  benedict-96

  committed
  c669e8f

• Added an optional argument (sidebar_sitename=false) to better display the logo.

  benedict-96

  committed
  350205b

• Added generation of tikz images.

  benedict-96

  committed
  c8f869e

• Added .yaml file to generate logos.

  benedict-96

  committed
  f52d258

• Got rid of extra clean command (should already be included in make all).

  benedict-96

  committed
  1e1d4db

• Added logos (you shouldn't include bitstype).

  benedict-96

  committed
  1455922

• Added logo.

  benedict-96

  committed
  a514b2a

• Added logo for darkmode.

  benedict-96

  committed
  ee17db8

• Deleted Readme workflow again.

  benedict-96

  committed
  fed1520

• type -> GeometricMachineLearning.type

  benedict-96

  committed
  9fce2b7

• Fixed a problem with tests that appeared because of the new syntax.

  benedict-96

  committed
  dd0596c

• Added an extra option just to generate logos.

  benedict-96

  committed
  6c42e9c

• Updated names of exported structs.

  benedict-96

  committed
  d707115

• Deleted test that appeared double for some reason.

  benedict-96

  committed
  b15bc2f

• type -> GeometricMachineLearning.type. This function is no longer exported.

  benedict-96

  committed
  384afac

• Remove Readme.yml.

  benedict-96

  committed
  56b08c5

Commits on Nov 24, 2023

• Started adding markdown for general manifolds.

  benedict-96

  committed
  3cbbab6

• Completed proof that SO(N) is a manifold.

  benedict-96

  committed
  d1ad571

• Added file describing basic topological concepts needed for manifolds.

  benedict-96

  committed
  3c2ff82

• Started basic description of NN optimizers.

  benedict-96

  committed
  a32fa22

Commits on Nov 26, 2023

• Added files on the existence and uniqueness theorem, the inverse function theorem and the sumersion theorem. The inverse function theorem file is essentially done whereas the existence and uniquene…

  Show description for 9f62150
  benedict-96

  committed
  9f62150

Commits on Nov 27, 2023

• Added the right reference for the Bishop book at the bottom.

  benedict-96

  committed
  9daa61f

• Remove old transformer script (Lux and Flux dependencies).

  benedict-96

  committed
  9387f2c

• Changed name to something more descriptive.

  benedict-96

  committed
  a47bdef

Commits on Nov 28, 2023

• Now only doing things once instead of multiple times (computation of various matrix-tensor products is now done with one specific function instead of many different ones.

  benedict-96

  committed
  dd31393

• Now exporting BFGS-related structs.

  benedict-96

  committed
  877f32f

• Cache and Optimizer for BFGS.

  benedict-96

  committed
  e22bc4f

• Added routines for vec, zero, get_backend, assign! and copy for the skew-symmetric matrix.

  benedict-96

  committed
  afa46c6

• Minor change of one comment and now allow different arrays to be optimized.

  benedict-96

  committed
  e3c4dfd

• Added routine for bfgs

  benedict-96

  committed
  fd5854e

• Added routines for vec (and its inverse), zero, get_backend, assign! and copy.

  benedict-96

  committed
  2790fea

• Added some comments.

  benedict-96

  committed
  80dbaa7

• Put comments in front of functions that are probably not needed.

  benedict-96

  committed
  af717fc

• Started adding documentation for BFSG.

  benedict-96

  committed
  6fa0937

• Added new transformer script explicitly for bfgs, up to now not different from regular transformer script.

  benedict-96

  committed
  c80b1c6

• Changed backend to GPU (in general) instead of CPU.

  benedict-96

  committed
  3cc8976

• Added a test for the vectorization.

  benedict-96

  committed
  dc64ce1

Commits on Nov 29, 2023

• changed some hyperparameters.

  benedict-96

  committed
  3082631

• Adjusted script to new syntax. Not working at the moment.

  benedict-96

  committed
  7426e9c

• Moved DataLoader to the front of the file <- DataLoader doesn't depend on anything in GML.

  benedict-96

  committed
  3d718df

• Changed equals sign to broadcasting operation (assign! and broadcast should serve the same purpose).

  benedict-96

  committed
  6031a94

• Changed equals sign to broadcast operation (assign! should serve the same purpose as a broadcasting operation).

  benedict-96

  committed
  e9b6741

• Added another optimize_for_one_epoch! routine for the case if we have a neural network as input.

  benedict-96

  committed
  9ddaa52

• Added an accuracy routine that takes neural network as input.

  benedict-96

  committed
  4a4b8a2

• main change is that we now add a small scalar to Y'*S to make sure it is non-zero.

  benedict-96

  committed
  5aa3bf8

• Added a constructor for optimizer if the input is a neural network.

  benedict-96

  committed
  4d3f2a1

• Added a short test script for the bfgs optimizer.

  benedict-96

  committed
  83e2702

• Now nor loading LinearAlgebra anymore.

  benedict-96

  committed
  420cd34

• Fixed typo dimesnion -> dimension.

  benedict-96

  committed
  6ec91a8

Commits on Dec 3, 2023

• Finished a rough outline of the documentation. Included a derivation based on an online comment.

  benedict-96

  committed
  5fbed13

• Fixed a typo that originated from not having defined for .

  benedict-96

  committed
  f2aa41d

• Decreased batch size.

  benedict-96

  committed
  90f59cf

• Added documentation for and .

  benedict-96

  committed
  8cd3bda

• Fixed a problem that had its origin in not being able to index GPU arrays. This now works for some reason.

  benedict-96

  committed
  7f07da4

• Added a test for bfgs on the Stiefel manifold.

  benedict-96

  committed
  4bff014

• Added an architecture for the classification transformer as a .

  benedict-96

  committed
  a33ef13

• Removed the temporary workaround and the cpu allocation.

  benedict-96

  committed
  f2099fa

• Added an additional way of computing H. These two probably are equivalent, but check first!

  benedict-96

  committed
  9740b83

Commits on Dec 4, 2023

• Added new file stiefel_projection.jl for the array of the same name.

  benedict-96

  committed
  24634f2

• Removed file. The remaining One struct is not needed anymore.

  benedict-96

  committed
  0bd6c09

• Removed a comment.

  benedict-96

  committed
  c8d6522

• Now using the newly implemented StiefelProjection struct. This gets rid of some CPU computation.

  benedict-96

  committed
  f7c9ff0

• Removed arrays/auxiliary.jl

  benedict-96

  committed
  a5c2026

• Added another constructor for the case when we just have two integers (and an optional type) as input arguments.

  benedict-96

  committed
  2823f5b

• Added a missing type dependency.

  benedict-96

  committed
  54b21bd

• Changed the way the indexing is done for the Grassmann manifold. This should still be checked!

  benedict-96

  committed
  9bde4ba

• Increased number of steps for convergence test. Was occasionally failing before.

  benedict-96

  committed
  2420ad9

• Also importing function init_optimizer_cache now because this is no longer exported by default.

  benedict-96

  committed
  015d324

• Added restriction that the type has to be a number. Needed for Julia v1.10.0.

  benedict-96

  committed
  0513445

• Merge pull request #88 from JuliaGNI/refactoring_and_documentation

  Show description for cdfd554
  michakraus

  authored
  cdfd554

• Merge branch 'main' into bfgs

  benedict-96

  authored
  41a46c4

• Got rid of CUDA and GPUArrays dependencies.

  benedict-96

  committed
  fb53122

• Merge pull request #90 from JuliaGNI/bfgs

  Show description for 3863768
  michakraus

  authored
  3863768

• Removed the with version 1. This is not in the GI Documenter.yml file and hopefully not needed.

  benedict-96

  committed
  2ea1dae

• Merge pull request #92 from JuliaGNI/bfgs

  Show description for 955912f
  michakraus

  authored
  955912f

• Started adding markdown for general manifolds.

  benedict-96

  committed
  249e4b8

• Completed proof that SO(N) is a manifold.

  benedict-96

  committed
  d77882d

• Added file describing basic topological concepts needed for manifolds.

  benedict-96

  committed
  3d47412

• Started basic description of NN optimizers.

  benedict-96

  committed
  8d6bc4a

• Added files on the existence and uniqueness theorem, the inverse function theorem and the sumersion theorem. The inverse function theorem file is essentially done whereas the existence and uniquene…

  Show description for 9e9bd25
  benedict-96

  committed
  9e9bd25

• Added the right reference for the Bishop book at the bottom.

  benedict-96

  committed
  dad1157

• Added the basic topological and analysis files as well as the bsfg optimizer.

  benedict-96

  committed
  a9a6429

• Added new references.

  benedict-96

  committed
  fca4061

• Fixed references in files. They are now all called with @Bibliography.

  benedict-96

  committed
  4f1cba9

• Fixed typo (space after #).

  benedict-96

  committed
  6b274ef

• Fixed typo.

  benedict-96

  committed
  981b04b

Commits on Dec 5, 2023

• Fixed yaml file in branch documentation_for_optimizer_yaml_test.

  benedict-96

  committed
  9ae14b8

• Added package dependencies.

  benedict-96

  committed
  883706a

• Merge branch 'documentation_for_optimizer' of https://github.com/JuliaGNI/GeometricMachineLearning.jl into documentation_for_optimizer

  benedict-96

  committed
  831144b

• Resolved merge conflicts.

  benedict-96

  committed
  ed48915

• Now making directory assets for the Documenter.

  benedict-96

  committed
  5aa570b

• Added two logos (normal and dark mode) via upload.

  benedict-96

  authored
  9445f19

• Merge branch 'documentation_for_optimizer' of https://github.com/JuliaGNI/GeometricMachineLearning.jl into documentation_for_optimizer

  Show description for 9bf1c7f
  benedict-96

  committed
  9bf1c7f

• Updated path for logos.

  benedict-96

  committed
  19b8c67

• Update README.md

  Show description for d7fc42a
  benedict-96

  authored
  d7fc42a

• Now not using local pngs for logos in README.md

  benedict-96

  authored
  6856585

• Removed pngs. Those are now saved online.

  benedict-96

  committed
  7e000dd

• fixed reference.

  benedict-96

  committed
  b64a779