Static Documentation for v0.1 #118

brunosan · 2024-01-14T11:23:39Z

This PR expands the current documentation based on #64 and tries to clarify structure.

It does not include executable documentation (Experiments, renamed tutorials) since I don't know yet how we can show the outputs of these without adding execution outputs to the repo history, nor having the CI download and execute the code. I'll PR a solution forking from here.

It also bring over the notebook from branch clay-over-ai.

We should have this merged before Wednesday presentation in Davos.

UI supports vector search & arithmetic for embeddings generated from CLAY. --------- Co-authored-by: SRM <[email protected]>

…ng space

…oring tiles

brunosan · 2024-01-14T15:56:35Z

240105-clay-v0-interpolation.ipynb
240105-clay-v0-reconstruction.ipynb
240105-location-embeddings.ipynb

Currently fail to run. I suspect @srmsoumya made this with a different version of the ClayModule, but I can't track the differences.

Error is:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
...
(
  unmasked_patches,
   unmasked_indices,
   masked_indices,
   masked_matrix,
--->  = model.model.encoder(batch)
...
File [~/code/Clay/model/docs/../src/model_clay.py:304  in Encoder.forward(self, datacube)
 cube, time, latlon = (
 datacube["pixels"],
 datacube["timestep"],
datacube["latlon"],
 )  # [B C H W]
 B, C, H, W = cube.shape
-->  patches = self.to_patch_embed(
       cube
     )  # [B G L D] - patchify & create embeddings per patch
 ...
RuntimeError: Input type (c10::Half) and bias type (float) should be the same

weiji14

Thanks, Just some suggested changes for now. Will do a proper review after #116 is merged (hopefully today), so that the diff is smaller and easier to review (22 files right now is a lot). I see also that there is duplication of documentation between the installation/basic_usage pages and the main README.md, and we should just have those instructions in one place.

@brunosan, would you like us (DevSeed) to push any changes directly to this PR branch, or would you like to handle the documentation personally for Davos? Cc @srmsoumya, as this brings in your notebooks from the docs/model branch.

src/model_clay.py

weiji14 · 2024-01-14T19:36:44Z

src/model_clay.py

@@ -581,19 +595,22 @@ def __init__(  # noqa: PLR0913
            "sar": (10, 11),
            "dem": (12,),
        },
+        shuffle=True,


@srmsoumya, did we discuss to make shuffle to be off by default (so that people running inference can avoid surprising results), and only turn this on during training?

Leaving as is for now, but created ticket #123

src/model_clay.py

docs/README

docs/_toc.yml

docs/assets/tiles.png

brunosan · 2024-01-15T06:04:31Z

Yes please make all changes you feel should be done. I can take over later today.

Co-authored-by: Wei Ji <[email protected]>

Make sure that large files don't get commited to git. Xref https://github.com/pre-commit/pre-commit-hooks/tree/v4.5.0#check-added-large-files

Don't force running all notebooks on the docs build, only run those that don't have output cells already.

Hosting the PNG images directly via GitHub's UI, instead of in the git repo.

Don't build files that aren't in the Table of Contents (https://jupyterbook.org/en/stable/structure/configure.html#disable-building-files-that-arent-in-the-table-of-contents). Can then rename README to README.md without raising a warning like `WARNING: document isn't included in any toctree`.

Some minor changes to the clay-v0-interpolation.ipynb file, and remove the large sample.gif file. Also gitignoring .gif and .png files.

Will be handled in #122

Should be 10 bands of Sentinel-2, not 13.

weiji14 · 2024-01-16T00:37:16Z

docs/_toc.yml

+    - title: Generative AI for pixel reconstruction
+      file: 240105-clay-v0-reconstruction
+    - title: Create location embeddings
+      file: 240105-location-embeddings
+    - title: Interpolating images in embedding space
+      file: 240105-clay-v0-interpolation


Probably should change these filenames, otherwise the URL comes up as https://clay-foundation.github.io/model/240105-clay-v0-reconstruction.html which doesn't look nice.

Done in 9926919

Most major issues with documentation quality have been resolved

yellowcap

Aight, lets do this!

SRM and others added 30 commits December 15, 2023 23:57

Make the clay model work on lightning cli

1009697

Update Mean & std for larger chunk of data

bcb1a06

Update model & trainer to work on multi gpu setup

1a40f56

Fix logging images to wandb

ced883f

Add specification docs

071a178

Add trainer details

8498119

Add streamlit UI to test the embeddings (#108)

6d431fc

UI supports vector search & arithmetic for embeddings generated from CLAY. --------- Co-authored-by: SRM <[email protected]>

Use rasterio.plot to show images & filter by year, tile & idx

454cc1c

Merge main with ddp

d96a638

Merge with ddp

785e7d4

Fix half:0 error by not explicitly setting float16 to cube[pixels]

e926b46

Add documentation to create the pixel reconstruction from the model

3ddd4ad

Add documention explaining location embeddings

4b5712d

Add shuffle as an argument to CLAY

a597347

Add documentation to showcase interpolation between images in embeddi…

f286205

…ng space

Adapt data pipeline to accept custom date range and local path for st…

b2c6806

…oring tiles

Add example script to run clay over custom AOI.

13c9ece

add sections to TOC

82437f8

basic use from root README

6c77a87

Install from root REAMDE

986c71d

how to build/preview docs

9185261

more narrative on release notes

ce31806

more narrative on intro

b14d1ab

Merge remote-tracking branch 'origin/clay-over-aoi' into doc-prep

00f04ea

rename intro to index, better SEO

3253f7b

reorder for clarity

c4fa077

fixed links

eb84195

fixed links

48c6b99

renamed file for clarity

7b3714e

added links and consistent title

edbf375

weiji14 previously requested changes Jan 14, 2024

View reviewed changes

brunosan and others added 17 commits January 15, 2024 08:57

Update docs/_toc.yml

c28a5f0

Co-authored-by: Wei Ji <[email protected]>

assume root folder

fa16cc9

Co-authored-by: Wei Ji <[email protected]>

Update src/model_clay.py

4dc2924

Co-authored-by: Wei Ji <[email protected]>

Update src/model_clay.py

3bbd47d

Co-authored-by: Wei Ji <[email protected]>

Update src/model_clay.py

370324b

Co-authored-by: Wei Ji <[email protected]>

Update docs/README

198f477

Co-authored-by: Wei Ji <[email protected]>

Update docs/installation.md

285d3b0

Co-authored-by: Wei Ji <[email protected]>

Merge branch 'main' into doc-prep

7043618

Let pre-commit enforce 512KB file size limit in entire repo

d44ab43

Make sure that large files don't get commited to git. Xref https://github.com/pre-commit/pre-commit-hooks/tree/v4.5.0#check-added-large-files

Remove extra args.update lines

4135737

Set execute_notebooks to cache

f09ac23

Don't force running all notebooks on the docs build, only run those that don't have output cells already.

Remove PNG files in docs/assets, replace with web version

4b35db4

Hosting the PNG images directly via GitHub's UI, instead of in the git repo.

Use huggingface ckpt, fix header, and rm sample.gif in interpolation nb

3afd8d0

Some minor changes to the clay-v0-interpolation.ipynb file, and remove the large sample.gif file. Also gitignoring .gif and .png files.

Fix some link typos and edit wording on main index page

b1c72a8

Remove run_region notebook

0a6113d

Will be handled in #122

Fix typos in model specification

d454fd1

Should be 10 bands of Sentinel-2, not 13.

weiji14 reviewed Jan 16, 2024

View reviewed changes

yellowcap and others added 4 commits January 16, 2024 09:08

Merge branch 'main' into doc-prep

69b6e1b

Rename tutorial notebooks

9926919

Exclude tutorial notebooks from execution during build.

ee96cc2

Make tutorial title consistent

3a09621

yellowcap mentioned this pull request Jan 16, 2024

Disable patch shuffling by default #123

Closed

yellowcap approved these changes Jan 16, 2024

View reviewed changes

yellowcap merged commit 959dfbd into main Jan 16, 2024
4 checks passed

yellowcap deleted the doc-prep branch January 16, 2024 09:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Static Documentation for v0.1 #118

Static Documentation for v0.1 #118

brunosan commented Jan 14, 2024

brunosan commented Jan 14, 2024 •

edited

Loading

weiji14 left a comment •

edited

Loading

weiji14 Jan 14, 2024

yellowcap Jan 16, 2024

brunosan commented Jan 15, 2024 via email •

edited

Loading

weiji14 Jan 16, 2024

yellowcap Jan 16, 2024

yellowcap left a comment

Static Documentation for v0.1 #118

Static Documentation for v0.1 #118

Conversation

brunosan commented Jan 14, 2024

brunosan commented Jan 14, 2024 • edited Loading

weiji14 left a comment • edited Loading

Choose a reason for hiding this comment

weiji14 Jan 14, 2024

Choose a reason for hiding this comment

yellowcap Jan 16, 2024

Choose a reason for hiding this comment

brunosan commented Jan 15, 2024 via email • edited Loading

weiji14 Jan 16, 2024

Choose a reason for hiding this comment

yellowcap Jan 16, 2024

Choose a reason for hiding this comment

yellowcap left a comment

Choose a reason for hiding this comment

brunosan commented Jan 14, 2024 •

edited

Loading

weiji14 left a comment •

edited

Loading

brunosan commented Jan 15, 2024 via email •

edited

Loading