Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build,refactor: update development tooling and package configuration #614

Merged
merged 85 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
02d3dfe
Create icon4py workspace with uv
egparedes Nov 27, 2024
2f670ac
Updating ./tools
egparedes Nov 27, 2024
8232cfb
WIP tools
egparedes Nov 29, 2024
0894864
Remove deprecated tools and fix dependency versions to avoid dependen…
egparedes Nov 29, 2024
bb87944
Fixes
egparedes Nov 29, 2024
0d20775
WIP pytest config migration
egparedes Nov 29, 2024
0f86391
Update license and versions
egparedes Nov 29, 2024
f3ea8e6
Update license and versions
egparedes Nov 29, 2024
e7068e1
WIP pre-commit
egparedes Nov 29, 2024
1762727
Format fixes
egparedes Nov 29, 2024
426563f
Finish migration of tools
egparedes Dec 2, 2024
e60d7e2
WIP common
egparedes Dec 2, 2024
e98679b
WIP migration to nox
egparedes Dec 3, 2024
87fe070
Merge branch 'main' into update-dev-tooling
egparedes Dec 4, 2024
eafa9fe
WIP adding icon4py.testing
egparedes Dec 4, 2024
4f54950
WIP2 adding icon4py.testing
egparedes Dec 4, 2024
74ea99c
WIP Fixes to pyprojects
egparedes Dec 5, 2024
9bc7b1f
Finished main work in `icon4py.testing`
egparedes Dec 5, 2024
72584af
Fixes for common tests
egparedes Dec 6, 2024
6c48671
Migrate all subpackages
egparedes Dec 9, 2024
454882d
Fix microphysics distribution package name
egparedes Dec 9, 2024
3977ab9
Add test sets in noxfile
egparedes Dec 9, 2024
a80588b
Merge branch 'main' into update-dev-tooling
egparedes Dec 9, 2024
309584f
Fixes after merging main
egparedes Dec 9, 2024
a64bfbc
More fixes after merging
egparedes Dec 10, 2024
435274c
Final fixes after merging
egparedes Dec 10, 2024
b6aa2af
Renaming data_allocation
egparedes Dec 10, 2024
d52533e
More renaming.
egparedes Dec 10, 2024
deecae5
Missing files
egparedes Dec 11, 2024
10bb890
Fixes for tests configs
egparedes Dec 11, 2024
ebdbc26
add data allocation import in common tests
halungge Dec 11, 2024
aa0d442
fix diffusion stencil tests
halungge Dec 11, 2024
3c087a2
fix common datatests and data path
halungge Dec 11, 2024
c10a872
fix diffusion datatests
halungge Dec 11, 2024
5e3b7eb
Address reviewer's comments (WIP)
egparedes Dec 11, 2024
caf0d03
Address more reviewer's comments
egparedes Dec 11, 2024
aa36905
Update code of conduct
egparedes Dec 11, 2024
1dc26ce
fix dycore stencil tests
halungge Dec 12, 2024
e2aaedf
fix import in SolveNonHydro
halungge Dec 12, 2024
33efdc5
fix advection stencils (except slow_tests)
halungge Dec 12, 2024
cebff7e
fix advection stencils (slow_tests)
halungge Dec 12, 2024
a070201
fix advection datatests
halungge Dec 12, 2024
35f8c3a
import pytest_config in conftest.py for microphysics
halungge Dec 12, 2024
515a046
fix microphysics test and fix type in filename
halungge Dec 12, 2024
4b14eec
fixing mpi tests
halungge Dec 12, 2024
2d5ddb2
fix driver tests
halungge Dec 12, 2024
a321b79
import grid fixture in conftest.py
halungge Dec 13, 2024
b1c1c20
Update READMe
egparedes Dec 13, 2024
82a7a2c
Move serialbox utils to testing
egparedes Dec 13, 2024
208dea4
Update dependencies for driver
egparedes Dec 13, 2024
6ef421b
remove slow_test marker
halungge Dec 13, 2024
b0d8ef6
remove markers from pytest.ini
halungge Dec 13, 2024
24889e5
Test config fixes
egparedes Dec 13, 2024
7d9d65d
Final test config fixes
egparedes Dec 16, 2024
fd9731e
Pre-commit and format settings
egparedes Dec 16, 2024
ac59797
Fix format with ruff & pre-commit
egparedes Dec 16, 2024
380c201
Update main docs
egparedes Dec 18, 2024
35d9a21
More documentation changes
egparedes Dec 18, 2024
0b89628
Update tools/README.md
egparedes Dec 18, 2024
df7ab61
Address review comments
egparedes Dec 18, 2024
4238c80
Update README.md
egparedes Dec 18, 2024
29c1b2c
Update README.md
egparedes Dec 18, 2024
5b32701
Address more review comments
egparedes Dec 18, 2024
bf34dee
Address more review comments
egparedes Dec 18, 2024
2b6e797
More README fixes and updates to project dependencies
egparedes Dec 18, 2024
66bd08f
More comments
egparedes Dec 18, 2024
5dc62dc
Fixes to dependencies (versions and sources), nox and CI settings
egparedes Dec 19, 2024
5d93f6d
Update github actions
egparedes Dec 19, 2024
081c89c
Stick to 3.10 for pre-commit
egparedes Dec 19, 2024
9332d92
Fixes for CI workflows
egparedes Dec 19, 2024
fd5ba7a
Install nox and uv with pip in the docker image
egparedes Dec 19, 2024
6160814
Add back uv installatino to github ci
egparedes Dec 19, 2024
0755d1b
Skip ditributed in test session (mpi4py compile error)
edopao Dec 19, 2024
d918753
fix previous commit
edopao Dec 19, 2024
4da0a7d
fix a test
havogt Dec 19, 2024
494804e
try to fix remaining tests
havogt Dec 19, 2024
a2442ea
provide cupy dependency
havogt Dec 19, 2024
e78c992
try fix tests
edopao Dec 20, 2024
703bb37
fix previous commit
edopao Dec 20, 2024
22ff7e1
fix dace test config
edopao Dec 20, 2024
cb20f1d
skip common in stencil tests
edopao Dec 20, 2024
2cfdd29
fix dace test config (1)
edopao Dec 20, 2024
97d2b1d
Fix disjoint extra dependencies in uv config
egparedes Dec 20, 2024
c377d24
Refactor ICON4PY_NOX_UV_EXTRA_ARGS
egparedes Dec 20, 2024
3e04165
Apply Edoardo's suggestion to fix DaCe
egparedes Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
135 changes: 79 additions & 56 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,57 +1,80 @@
# NOTE: pre-commit runs all hooks from the root folder of the repository,
# as regular git hooks do. Therefore, paths passed as arguments to the plugins
# should always be relative to the root folder.

default_stages: [pre-commit, pre-push]
default_language_version:
python: python3.11
minimum_pre_commit_version: 3.8.0
files: |
(?x)^(
tools/.*|
model/common/.*
)$

repos:
- repo: local
hooks:
- id: run-common-precommit
name: Run Model Common Pre-commit
entry: pre-commit run --config model/common/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- id: run-driver-precommit
name: Run Model Driver Pre-commit
entry: pre-commit run --config model/driver/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- id: run-atmosphere-advection-precommit
name: Run Model Atmosphere Advection Pre-commit
entry: pre-commit run --config model/atmosphere/advection/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- id: run-atmosphere-diffusion-precommit
name: Run Model Atmosphere Diffusion Pre-commit
entry: pre-commit run --config model/atmosphere/diffusion/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- id: run-atmosphere-dycore-precommit
name: Run Model Atmosphere Dycore Pre-commit
entry: pre-commit run --config model/atmosphere/dycore/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- id: run-atmosphere-subgrid_scale_physics-microphysics-precommit
name: Run Model Atmosphere Subgrid_scale_physics Microphysics Pre-commit
entry: pre-commit run --config model/atmosphere/subgrid_scale_physics/microphysics/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- id: run-tools-precommit
name: Run Tools Pre-commit
entry: pre-commit run --config tools/.pre-commit-config.yaml --all-files
language: system
pass_filenames: false
always_run: true

- repo: https://github.com/gauge-sh/tach-pre-commit
rev: v0.10.7
hooks:
- id: tach
name: Check inter-package dependencies
- repo: meta
hooks:
- id: check-hooks-apply
stages: [manual]
- id: check-useless-excludes
stages: [manual]

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: check-case-conflict
- id: check-merge-conflict
- id: check-shebang-scripts-are-executable
- id: check-symlinks
- id: check-yaml
- id: debug-statements
- id: destroyed-symlinks

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
rev: v2.6.0
hooks:
- id: pretty-format-ini
args: [--autofix]
- id: pretty-format-toml
args: [--autofix]
- id: pretty-format-yaml
args: [--autofix, --preserve-quotes, --indent, "2"]

- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.0.0-alpha.4
hooks:
- id: prettier
types_or: [markdown, json]

- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: v1.5.5
hooks:
- id: insert-license
name: add license for all ICON4Py Python source files
types: [python]
args: [--comment-style, "|#|", --license-filepath, HEADER.txt, --fuzzy-match-generates-todo]

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.2.2
hooks:
- id: ruff
args: [--fix]
- id: ruff-format

- repo: https://github.com/gauge-sh/tach-pre-commit
rev: v0.16.0
hooks:
- id: tach
name: Check inter-package dependencies

# - repo: local
# hooks:
# - id: mypy
# name: mypy static type checker
# entry: bash -c 'mypy tools/src/icon4pytools model/common/src/icon4py/model/common'
# language: system
# types_or: [python, pyi]
# pass_filenames: false
# require_serial: true
# stages: [pre-commit]
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11
134 changes: 134 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@

# Contributor Covenant Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
nfarabullini marked this conversation as resolved.
Show resolved Hide resolved
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, caste, color, religion, or sexual
identity and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.

## Our Standards

Examples of behavior that contributes to a positive environment for our
community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall
egparedes marked this conversation as resolved.
Show resolved Hide resolved
community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or advances of
any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address,
without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
egparedes marked this conversation as resolved.
Show resolved Hide resolved

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
egparedes marked this conversation as resolved.
Show resolved Hide resolved
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.

Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
egparedes marked this conversation as resolved.
Show resolved Hide resolved
decisions when appropriate.

## Scope

This Code of Conduct applies within all community spaces, and also applies when
egparedes marked this conversation as resolved.
Show resolved Hide resolved
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official email address,
egparedes marked this conversation as resolved.
Show resolved Hide resolved
posting via an official social media account, or acting as an appointed
representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[[email protected]](mailto:[email protected]).
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.

## Enforcement Guidelines

Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
egparedes marked this conversation as resolved.
Show resolved Hide resolved

### 1. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series of
actions.

**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or permanent
egparedes marked this conversation as resolved.
Show resolved Hide resolved
ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
egparedes marked this conversation as resolved.
Show resolved Hide resolved
Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within the
community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.1, available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].

Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].

For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
[https://www.contributor-covenant.org/translations][translations].

[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations

File renamed without changes.
File renamed without changes.
30 changes: 0 additions & 30 deletions base-requirements-dev.txt

This file was deleted.

3 changes: 0 additions & 3 deletions base-requirements.txt

This file was deleted.

2 changes: 1 addition & 1 deletion jenkins/spack/pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ markers =
slow_tests: mark test as slow.
datatest: test depending on serialized data generated by a full model run
with_netcdf: test depending on a compatible version of netCDF4
addopts = -p icon4py.model.common.test_utils.pytest_config
addopts = -p icon4py.model.testing.pytest_config
testpaths = tools/tests/f2ser
tools/tests/icon4pygen
tools/tests/liskov
Expand Down
2 changes: 2 additions & 0 deletions model/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# ICON4Py Model

[![Nox](https://img.shields.io/badge/%F0%9F%A6%8A-Nox-D85E00.svg)](https://github.com/wntrblm/nox)

This folder contains Python implementations for multiple ICON components.

It includes the following packages:
Expand Down
44 changes: 0 additions & 44 deletions model/atmosphere/advection/.flake8

This file was deleted.

Loading
Loading