Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gap refactor v0.3.0 #467

Merged
merged 94 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
80247f0
WIP: refactoring to only gap --> major implementation is done
vergauwenthomas Jul 23, 2024
ad0239f
get IO test running
vergauwenthomas Jul 24, 2024
d534fdc
fixed analysis class
vergauwenthomas Jul 25, 2024
66a45b9
fix breaking test
vergauwenthomas Jul 29, 2024
8c92416
get gapfill and test running
vergauwenthomas Jul 29, 2024
bdaca5f
version bump and template show in IO test
vergauwenthomas Jul 29, 2024
adfda24
get ploting example running
vergauwenthomas Jul 29, 2024
b5d29fc
mulit inheriance of Dataset and fix of the add and irregular test
vergauwenthomas Jul 30, 2024
b0ea46e
fix modeldata test
vergauwenthomas Jul 30, 2024
e8b9135
get qc check working
vergauwenthomas Jul 30, 2024
ba888d0
cleanup and docstring updates
vergauwenthomas Jul 31, 2024
82babab
cleanup
vergauwenthomas Jul 31, 2024
b2b5e27
logging and exmple + kwargs for interp gap
vergauwenthomas Aug 1, 2024
112cad9
upgrade the interpolation method to include higher order interpolatio…
vergauwenthomas Aug 1, 2024
37f62ed
update examples
vergauwenthomas Aug 1, 2024
ad5d735
fixed the analysis example
vergauwenthomas Aug 1, 2024
221ba55
bugfix in sync + irregular demo in docs
vergauwenthomas Aug 2, 2024
efb9b1c
fix io test and cleanup doc examples/topics
vergauwenthomas Aug 2, 2024
5af3bf0
logginmodule fix + wroking on docstrigs
vergauwenthomas Aug 2, 2024
2bee1d2
updating docstrings + print info bug fix
vergauwenthomas Aug 2, 2024
f0a8d8b
fix doctest for dataset core + update lcz map
vergauwenthomas Aug 9, 2024
f037107
fix docstrings and doctest in the Analysis class
vergauwenthomas Aug 13, 2024
7bdd212
fix doctest in visuals, qc and gaps
vergauwenthomas Aug 19, 2024
5d2e7e8
add see also for visual module
vergauwenthomas Aug 19, 2024
1396fe7
cleanup files
vergauwenthomas Aug 19, 2024
0ca192f
cleanup files
vergauwenthomas Aug 19, 2024
c8aa069
cleanup files
vergauwenthomas Aug 19, 2024
05435f5
Static and Dynamic modeldata classes for GEE interaction (#469)
vergauwenthomas Aug 29, 2024
6f8cd0b
work on the documentation
vergauwenthomas Aug 29, 2024
1b7c158
minor changes + paper notebook compatible and tracking with new versions
vergauwenthomas Aug 30, 2024
be1e7aa
savepoint: writing docstrings in the Gap class
vergauwenthomas Aug 30, 2024
6de8751
Get docsring examples running
vergauwenthomas Sep 2, 2024
603ec59
fix identations in docstrings
vergauwenthomas Sep 2, 2024
ee47609
embeded plots in the documentation
vergauwenthomas Sep 2, 2024
31dc3de
version fix
vergauwenthomas Sep 2, 2024
dfe6f40
fix gap fill test
vergauwenthomas Sep 2, 2024
a32742f
avoid tracking of py copies of notebooks
vergauwenthomas Sep 2, 2024
37402c1
fix docstring tests
vergauwenthomas Sep 2, 2024
fac5572
fix running devpipeline
vergauwenthomas Sep 2, 2024
2d9d745
rm chmod
vergauwenthomas Sep 3, 2024
15234a9
minor code review
vergauwenthomas Sep 3, 2024
a620fea
Merge branch 'gap_refactor_v0.2.2' of github.com:vergauwenthomas/vlin…
vergauwenthomas Sep 3, 2024
4f3a179
savepoint codereview
vergauwenthomas Sep 3, 2024
bea2936
test windows runner
vergauwenthomas Sep 3, 2024
98d2499
rename update settings
vergauwenthomas Sep 3, 2024
c0c4381
fix setting updater methods
vergauwenthomas Sep 4, 2024
397be47
add docstrings logging and fix docsrings for settings updater
vergauwenthomas Sep 4, 2024
f42a461
code review fixes (deleting unused lines + sys.path inserts for all t…
vergauwenthomas Sep 4, 2024
bc6c824
import dataset not a method but function
vergauwenthomas Sep 4, 2024
4c06159
fix path bug + add matplotlib in the requirements for rtd
vergauwenthomas Sep 4, 2024
4c44284
fix bug that creates the "=" file
vergauwenthomas Sep 4, 2024
7afdbb7
try rtd with poetry install
vergauwenthomas Sep 4, 2024
682846d
fix bugs in paper ipynb
vergauwenthomas Sep 4, 2024
0151a67
test with rtd yaml
vergauwenthomas Sep 4, 2024
5029e6a
new rtd yaml test
vergauwenthomas Sep 4, 2024
1527e9c
new test rtd yaml
vergauwenthomas Sep 4, 2024
018dc6a
sphinx build in poetry env
vergauwenthomas Sep 4, 2024
78614b6
test RTD yaml
vergauwenthomas Sep 4, 2024
70cc71b
fix rtd yaml
vergauwenthomas Sep 4, 2024
7959dc7
test yaml build doc
vergauwenthomas Sep 4, 2024
7339113
sytax error in rtd yaml
vergauwenthomas Sep 4, 2024
f14f2a0
testing
vergauwenthomas Sep 4, 2024
3590005
test rtd yml
vergauwenthomas Sep 4, 2024
bf6068d
test rtd build with sphinx to requirements.txt build
vergauwenthomas Sep 5, 2024
f51c904
test in pre install
vergauwenthomas Sep 5, 2024
ab9e562
requirements txt with group
vergauwenthomas Sep 5, 2024
fb843cc
new attempt
vergauwenthomas Sep 5, 2024
2dc57e2
test rtd build
vergauwenthomas Sep 5, 2024
02776b8
without sudo rtd build
vergauwenthomas Sep 5, 2024
bdc119f
without pandoc rtd build test
vergauwenthomas Sep 5, 2024
5f84bb0
without mocking dependencies
vergauwenthomas Sep 5, 2024
49d54f2
add a pip list
vergauwenthomas Sep 5, 2024
0b397ac
now with dependecy install
vergauwenthomas Sep 5, 2024
ebe437a
export req in pre_install
vergauwenthomas Sep 5, 2024
663a3de
new test
vergauwenthomas Sep 5, 2024
9cbebfe
explicit groups
vergauwenthomas Sep 5, 2024
506fe13
cleanup after all testing
vergauwenthomas Sep 5, 2024
f4c6ba4
spell checking
vergauwenthomas Sep 6, 2024
ec6d64c
spelling check on all files
vergauwenthomas Sep 9, 2024
19d10d6
ready for last tests !!
vergauwenthomas Sep 9, 2024
885eaf2
fix (last?) review points and fixes
vergauwenthomas Sep 10, 2024
715a8d0
add ipython to documentation group for fixing error
vergauwenthomas Sep 10, 2024
fe057e3
yaml workflow fix
vergauwenthomas Sep 10, 2024
8a786f3
Update main_workflow.yml
vergauwenthomas Sep 10, 2024
7eaf17c
version bump + add pip install metobs to rtd builder
vergauwenthomas Sep 10, 2024
e23e6fa
Merge branch 'gap_refactor_v0.2.2' of github.com:vergauwenthomas/vlin…
vergauwenthomas Sep 10, 2024
614dbf6
pip install of self for testing
vergauwenthomas Sep 10, 2024
b44d575
add data in the included testdata folder for the irr and __add__ tests
vergauwenthomas Sep 10, 2024
dfb8e50
make sure that notebooks are not executed when (online) building docs
vergauwenthomas Sep 10, 2024
c775183
testing
vergauwenthomas Sep 10, 2024
4db8a21
try new fix for workflows
vergauwenthomas Sep 10, 2024
408d062
update the output of the ipynb examples, so they are rendered
vergauwenthomas Sep 10, 2024
dc83b2e
do not build documentation on github
vergauwenthomas Sep 10, 2024
cb20b2d
cleanup + fix last doc issues. Ready for MERGE !!!!
vergauwenthomas Sep 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
353 changes: 214 additions & 139 deletions .github/workflows/main_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,12 @@ jobs:
poetry install --all-extras
- name: export documentation requirement group
run: |
poetry export -f requirements.txt --without-hashes --only documentation -o only_doc_req.txt
poetry export -f requirements.txt --without-hashes --with=documentation,dev,titan --format=requirements.txt -o doc_build_req.txt
- name: convert doc requirments to artifact
uses: actions/upload-artifact@v3
with:
name: documentation_requirements
path: only_doc_req.txt
path: doc_build_req.txt
- name: export titan requirement group
run: |
poetry export -f requirements.txt --without-hashes --only titan -o only_titan_req.txt
Expand Down Expand Up @@ -109,6 +109,7 @@ jobs:
python tests/push_test/breaking_test.py
#python tests/push_test/gui_launch_test.py
python tests/push_test/analysis_test.py
python tests/push_test/irregular_and_add_test.py

#--- Package os installation ---
mac_install_testing:
Expand Down Expand Up @@ -138,11 +139,11 @@ jobs:
needs: packageing
runs-on: windows-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python39
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.9"
python-version: "3.11"
- name: Download the package build
uses: actions/download-artifact@v3
with:
Expand Down Expand Up @@ -177,151 +178,225 @@ jobs:
echo "version tags are not aligned!"
exit 1

#---- Documentation build -----#
doctests:
name: documentation-test
needs: packageing
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python39
uses: actions/setup-python@v4
with:
python-version: '3.9'
#cache: 'poetry'
# You can test your matrix by printing the current Python version
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: install pandoc (system wide)
run: |
sudo apt-get -y install pandoc
- name: Download the package build
uses: actions/download-artifact@v3
with:
name: package_build
- name: Install the package
run: |
python3 -m pip install ./metobs_toolkit-*.tar.gz
- name: get documentation requirements
uses: actions/download-artifact@v3
with:
name: documentation_requirements
- name: install doc depending packages
run: |
pip install -r only_doc_req.txt
- name: Build documentation
run: |
sphinx-build -a -E docs _build
- name: deploy documentation on dev
if: |
github.ref == 'refs/heads/dev' ||
github.ref == 'refs/head/main' ||
contains(github.event.head_commit.message, 'build doc')
uses: peaceiris/actions-gh-pages@v3
with:
publish_branch: gh-pages-dev
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: _build/
force_orphan: true
# #---- Documentation build -----#
# docsbuild:
# name: documentation-test
# needs: packageing
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v3
# - name: Set up Python310
# uses: actions/setup-python@v4
# with:
# python-version: '3.10'
# #cache: 'poetry'
# # You can test your matrix by printing the current Python version
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# # - name: install pandoc (system wide)
# # run: |
# # sudo apt-get -y install pandoc
# - name: Download the package build
# uses: actions/download-artifact@v3
# with:
# name: package_build
# - name: Install the package
# run: |
# python -m pip install ./metobs_toolkit-*.tar.gz
# - name: get documentation requirements
# uses: actions/download-artifact@v3
# with:
# name: documentation_requirements
# - name: install doc depending packages
# run: |
# pip install -r doc_build_req.txt
# - name: Build documentation
# run: |
# sphinx-build -a -E docs _build
# - name: deploy documentation on dev
# if: |
# github.ref == 'refs/heads/dev' ||
# github.ref == 'refs/head/main' ||
# contains(github.event.head_commit.message, 'build doc')
# uses: peaceiris/actions-gh-pages@v3
# with:
# publish_branch: gh-pages-dev
# github_token: ${{ secrets.GITHUB_TOKEN }}
# publish_dir: _build/
# force_orphan: true


#---- Deploy documentation MAIN-----#
deploy_doc_main:
name: Deploy main documentation
needs: [doctests,run-tests,versiontest,mac_install_testing, windows_install_testing]
runs-on: ubuntu-latest
if: |
github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- name: Set up Python39
uses: actions/setup-python@v4
with:
python-version: '3.9'
#cache: 'poetry'
# You can test your matrix by printing the current Python version
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: install pandoc (system wide)
run: |
sudo apt-get -y install pandoc
- name: Download the package build
uses: actions/download-artifact@v3
with:
name: package_build
- name: Install the package
run: |
python3 -m pip install ./metobs_toolkit-*.tar.gz
- name: get documentation requirements
uses: actions/download-artifact@v3
with:
name: documentation_requirements
- name: install doc depending packages
run: |
pip install -r only_doc_req.txt
- name: Build documentation
run: |
sphinx-build -a -E docs _build
- name: deploy documentation
uses: peaceiris/actions-gh-pages@v3
with:
publish_branch: gh-pages
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: _build/
force_orphan: true
#---- Documentation build -----#
# docsbuild:
# name: documentation-building
# needs: packageing
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v3
# - name: Set up Python39
# uses: actions/setup-python@v4
# with:
# python-version: '3.10'
# #cache: 'poetry'
# # You can test your matrix by printing the current Python version
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: remove previous builds
# run: |
# rm -f dist/metobs_toolkit-*.whl
# rm -f dist/metobs_toolkit-*.tar.gz
# - name: Install poetry
# run: |
# pip install poetry
# - name: make build of the package
# run: |
# ./deploiment/build_package.sh
# - name: install the package for doc building
# run: |
# poetry install --with=documentation,dev
# poetry add .
# - name: Build documentation
# run: |
# poetry run sphinx-build -a -E docs _build
# steps:
# - uses: actions/checkout@v3
# - name: Set up Python310
# uses: actions/setup-python@v4
# with:
# python-version: '3.10'
# #cache: 'poetry'
# # You can test your matrix by printing the current Python version
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# # - name: install pandoc (system wide)
# # run: |
# # sudo apt-get -y install pandoc
# - name: Download the package build
# uses: actions/download-artifact@v3
# with:
# name: package_build
# - name: Install the package
# run: |
# python -m pip install ./metobs_toolkit-*.tar.gz
# - name: get documentation requirements
# uses: actions/download-artifact@v3
# with:
# name: documentation_requirements
# - name: install doc depending packages
# run: |
# pip install -r doc_build_req.txt
# - name: Build documentation
# run: |
# sphinx-build -a -E docs _build
# - name: deploy documentation on dev
# if: |
# github.ref == 'refs/heads/dev' ||
# github.ref == 'refs/head/main' ||
# contains(github.event.head_commit.message, 'build doc')
# uses: peaceiris/actions-gh-pages@v3
# with:
# publish_branch: gh-pages-dev
# github_token: ${{ secrets.GITHUB_TOKEN }}
# publish_dir: _build/
# force_orphan: true

# #---- Deploy documentation MAIN-----#
# deploy_doc_main:
# name: Deploy main documentation
# needs: [docsbuild,run-tests,versiontest,mac_install_testing, windows_install_testing]
# runs-on: ubuntu-latest
# if: |
# github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main'
# steps:
# - uses: actions/checkout@v3
# - name: Set up Python39
# uses: actions/setup-python@v4
# with:
# python-version: '3.9'
# #cache: 'poetry'
# # You can test your matrix by printing the current Python version
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: install pandoc (system wide)
# run: |
# sudo apt-get -y install pandoc
# - name: Download the package build
# uses: actions/download-artifact@v3
# with:
# name: package_build
# - name: Install the package
# run: |
# python3 -m pip install ./metobs_toolkit-*.tar.gz
# - name: get documentation requirements
# uses: actions/download-artifact@v3
# with:
# name: documentation_requirements
# - name: install doc depending packages
# run: |
# pip install -r doc_build_req.txt
# - name: Build documentation
# run: |
# sphinx-build -a -E docs _build
# - name: deploy documentation
# uses: peaceiris/actions-gh-pages@v3
# with:
# publish_branch: gh-pages
# github_token: ${{ secrets.GITHUB_TOKEN }}
# publish_dir: _build/
# force_orphan: true

#---- Deploy documentation dev -----#
deploy_doc_dev:
name: Deploy Dev documentation
needs: [doctests,run-tests,versiontest,mac_install_testing, windows_install_testing]
runs-on: ubuntu-latest
if: |
github.ref == 'refs/heads/dev'
steps:
- uses: actions/checkout@v3
- name: Set up Python39
uses: actions/setup-python@v4
with:
python-version: '3.9'
#cache: 'poetry'
# You can test your matrix by printing the current Python version
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: install pandoc (system wide)
run: |
sudo apt-get -y install pandoc
- name: Download the package build
uses: actions/download-artifact@v3
with:
name: package_build
- name: Install the package
run: |
python3 -m pip install ./metobs_toolkit-*.tar.gz
- name: get documentation requirements
uses: actions/download-artifact@v3
with:
name: documentation_requirements
- name: install doc depending packages
run: |
pip install -r only_doc_req.txt
- name: Build documentation
run: |
sphinx-build -a -E docs _build
- name: deploy documentation
uses: peaceiris/actions-gh-pages@v3
with:
publish_branch: gh-pages-dev
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: _build/
force_orphan: true
# deploy_doc_dev:
# name: Deploy Dev documentation
# needs: [docsbuild,run-tests,versiontest,mac_install_testing, windows_install_testing]
# runs-on: ubuntu-latest
# if: |
# github.ref == 'refs/heads/dev'
# steps:
# - uses: actions/checkout@v3
# - name: Set up Python39
# uses: actions/setup-python@v4
# with:
# python-version: '3.9'
# #cache: 'poetry'
# # You can test your matrix by printing the current Python version
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: install pandoc (system wide)
# run: |
# sudo apt-get -y install pandoc
# - name: Download the package build
# uses: actions/download-artifact@v3
# with:
# name: package_build
# - name: Install the package
# run: |
# python3 -m pip install ./metobs_toolkit-*.tar.gz
# - name: get documentation requirements
# uses: actions/download-artifact@v3
# with:
# name: documentation_requirements
# - name: install doc depending packages
# run: |
# pip install -r doc_build_req.txt
# - name: Build documentation
# run: |
# sphinx-build -a -E docs _build
# - name: deploy documentation
# uses: peaceiris/actions-gh-pages@v3
# with:
# publish_branch: gh-pages-dev
# github_token: ${{ secrets.GITHUB_TOKEN }}
# publish_dir: _build/
# force_orphan: true




# ---- delete artifacts that are not for storage -----
cleanup_artifacts:
name: delete artifacts
needs: [doctests,run-tests,versiontest,mac_install_testing, windows_install_testing, deploy_doc_main]
needs: [run-tests,versiontest,mac_install_testing, windows_install_testing]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand Down
Loading
Loading