-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[x] Pre-commit checks pass
- Loading branch information
0 parents
commit 180f183
Showing
80 changed files
with
6,838 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
########################################################################################## | ||
# | ||
# DESCRIPTION: Development Docker Container. Installs the following: | ||
# * miniconda | ||
# * Oh-My-Zsh | ||
# * awesome-vim | ||
# | ||
# GPU SUPPORT: In order to get GPU support the nvidia image's CUDA version must | ||
# match the version on the host computer. | ||
# | ||
# RENDERING: If rendering to the screen is required then execute the following | ||
# prior to starting the container: | ||
# | ||
# ```console | ||
# xhost +local:root | ||
# ``` | ||
# | ||
# AUTHOR: W. Li | ||
# VERSION: 1.0 | ||
# CREATED: 12/15/2023 | ||
########################################################################################## | ||
|
||
# Source Image (this should match the CUDA version on the host) | ||
# FROM nvidia/cuda:11.4.1-base-ubuntu20.04 | ||
FROM ubuntu:20.04 | ||
|
||
# Labels | ||
LABEL version="1.0" | ||
LABEL description="Miniconda Dev Image" | ||
|
||
# Expose ports here | ||
# EXPOSE 8080 | ||
|
||
# Install basic packages needed for development | ||
RUN yes Y | apt update | ||
RUN yes Y | apt-get update | ||
RUN yes Y | apt upgrade | ||
RUN yes Y | apt install vim | ||
RUN apt-get install -y --no-install-recommends \ | ||
git \ | ||
wget \ | ||
g++ \ | ||
gcc \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
# Setup shell | ||
RUN sh -c "$(wget -O- https://github.com/deluan/zsh-in-docker/releases/download/v1.1.1/zsh-in-docker.sh)" -- \ | ||
-t robbyrussell \ | ||
-p git -p ssh-agent -p 'history-substring-search' \ | ||
-a 'bindkey "\$terminfo[kcuu1]" history-substring-search-up' \ | ||
-a 'bindkey "\$terminfo[kcud1]" history-substring-search-down' | ||
|
||
# Setup Shell | ||
SHELL ["/bin/zsh", "-c"] | ||
|
||
# Miniconda requires that the path point to its python binary | ||
ENV PATH="/root/miniconda3/bin:${PATH}" | ||
ARG PATH="/root/miniconda3/bin:${PATH}" | ||
|
||
# Source python versions: https://repo.anaconda.com/miniconda/ | ||
RUN wget \ | ||
https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \ | ||
&& mkdir /root/.conda \ | ||
&& bash Miniconda3-latest-Linux-x86_64.sh -b \ | ||
&& rm -f Miniconda3-latest-Linux-x86_64.sh | ||
|
||
RUN conda init zsh | ||
|
||
# Setup Vim-Awesome | ||
RUN git clone --depth=1 https://github.com/amix/vimrc.git ~/.vim_runtime \ | ||
&& sh ~/.vim_runtime/install_basic_vimrc.sh | ||
|
||
# Setup working directory | ||
WORKDIR /projects | ||
|
||
# Copy folder/files into container (must be within context directory) | ||
# Process: local files -> artifact dir -> container dir | ||
# COPY <local_path> <container_path> | ||
|
||
# Environment | ||
ENV SHELL /bin/zsh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
# 📒 Description | ||
## 🐳 Docker | ||
* Primary method for building and maintaining **un-secure** containers. | ||
* Must have root access in order to work. | ||
* Deployable to any machine. | ||
* [Additional Information](#🐳-docker-notes) | ||
|
||
## 🦉 Apptainer | ||
* Primary method for building and maintaining **secure** containers. | ||
* Used primarily in settings where root access is not possible. | ||
* Deployable to any machine. | ||
* [Additional Information](#🦉-apptainer-notes) | ||
|
||
## 📦 Packages | ||
The default development containers will these packages by default. Only the essentials are included. | ||
|
||
* [Miniconda](https://docs.conda.io/projects/miniconda/en/latest/) | ||
* [Oh My Bash](https://github.com/ohmybash/oh-my-bash) | ||
* [Oh My Zsh](https://ohmyz.sh) | ||
* [Precommit](https://pre-commit.com) | ||
|
||
# 🐳 Docker Notes | ||
|
||
## 🎼 Docker Compose | ||
* Allows you to create multiple different builds using different Docker files. | ||
* The context directory can be set from the `docker_compose.yml` file. | ||
* More options than a standard Docker file. | ||
|
||
## 👟 Running Commands | ||
* In Docker each `RUN` command is treated as a new shell instance. This means that commands cannot persist across different RUN commands. | ||
|
||
```Docker | ||
RUN command1 # this command is run in its own shell! | ||
RUN command2 # this command is run in its own shell! | ||
``` | ||
|
||
## 🗃️ Copying Folders/Files | ||
* If you need to copy files or directories into the Docker container, first put them in an **artifacts** directory that can be seen from the context directory. | ||
|
||
```Docker | ||
COPY <artifact_directory> <container_path> | ||
``` | ||
## 📂 Context Directory | ||
|
||
* By default Dockerfiles are only aware of what is in their **context directory**. A context directory is the directory where a Dockerfile is located in. It cannot access any files outside of the context directory. | ||
|
||
* The context directory can be set from the command line or from a `docker_compose.yml` file. | ||
|
||
* It is recommended that all folders/files be copied be placed in an **artifacts** directory that is under the context directory. | ||
|
||
## 💾 Out of Memory | ||
* Occasionally you will need to clean your docker images and docker volumes in order to recover memory. | ||
|
||
```console | ||
docker volume prune | ||
docker image prune | ||
docker container prune | ||
``` | ||
|
||
# 🦉 Apptainer Notes |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
""" | ||
Run this script to build the Docker development image. | ||
""" | ||
import subprocess | ||
|
||
image_name = "development-base" | ||
|
||
# Build the image from the development Dockerfile | ||
subprocess.run(["docker", "build", "-t", image_name, "."]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at: | ||
// https://github.com/microsoft/vscode-dev-containers/tree/v0.191.0/containers/ubuntu | ||
{ | ||
"name": "dev-python", | ||
"image": "development-base", | ||
"build": { | ||
"dockerfile": "Dockerfile", | ||
}, | ||
// Add arguments that run with the container | ||
// These are arguments needed for GPU support on a Linux machine | ||
"runArgs": [ | ||
// "--net=host", | ||
// "-m=10g", | ||
// "--shm-size=10g", | ||
// "--gpus=all", | ||
// "-v", | ||
// "/tmp/.X11-unix:/tmp/.X11-unix", | ||
// "-e", | ||
// "DISPLAY", | ||
// "--device", | ||
// "/dev/dri" | ||
], | ||
"customizations": { | ||
"vscode": { | ||
// Set *default* container specific settings.json values on container create. | ||
"settings": {}, | ||
// Add the IDs of extensions you want installed when the container is created. | ||
"extensions": [ | ||
"eamodio.gitlens", | ||
"littlefoxteam.vscode-python-test-adapter", | ||
"ms-python.python", | ||
"mhutchie.git-graph", | ||
"njpwerner.autodocstring", | ||
"pkief.material-icon-theme", | ||
] | ||
} | ||
}, | ||
// Use 'forwardPorts' to make a list of ports inside the container available locally. | ||
"forwardPorts": [], | ||
// Use 'postCreateCommand' to run commands after the container is created. | ||
"postCreateCommand": "uname -a" | ||
// Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root. | ||
// "remoteUser": "vscode" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
########################################################################################## | ||
# DESCRIPTION: Run pre-commit on all files in repo. | ||
# AUTHOR: W. Li | ||
# VERSION: 1.0 | ||
# CREATED: 1/6/2024 | ||
# | ||
# References: | ||
# * https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-pythonCommon | ||
# | ||
########################################################################################## | ||
|
||
name: pre-commit | ||
|
||
on: | ||
pull_request: | ||
push: | ||
branches: [main] | ||
|
||
jobs: | ||
pre-commit: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- uses: actions/setup-python@v5 | ||
- uses: pre-commit/[email protected] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
########################################################################################## | ||
# DESCRIPTION: Runs pytests with multiple versions of python. | ||
# AUTHOR: W. Li | ||
# VERSION: 1.0 | ||
# CREATED: 1/6/2024 | ||
# | ||
# References: | ||
# * https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-pythonCommon | ||
# | ||
########################################################################################## | ||
|
||
name: pytests | ||
|
||
on: | ||
# Configure the branches you want Github actions to run on. | ||
# Leave blank if you want to run on all branches. | ||
push: | ||
branches: [ "main" ] | ||
pull_request: | ||
branches: [ "main" ] | ||
|
||
# Configure jobs | ||
jobs: | ||
test: | ||
# Set the OS and python versions | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ["3.10", "3.11"] | ||
# Checkout the repo | ||
steps: | ||
- uses: actions/checkout@v4 | ||
# Setup the python environments to use. | ||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
# Install poetry and disable virtual environments. | ||
- name: Install Poetry | ||
uses: snok/install-poetry@v1 | ||
with: | ||
virtualenvs-create: false | ||
virtualenvs-in-project: false | ||
installer-parallel: true | ||
# Install dependencies. | ||
- name: Install dependencies | ||
run: | | ||
poetry install --no-interaction --with=dev --no-root | ||
# Run pytest with coverage. | ||
- name: Test with pytest | ||
run: > | ||
pytest | ||
--doctest-modules | ||
--cov=./ | ||
--cov-report=xml | ||
--cov-report=html:pytest-results-${{ matrix.python-version }} | ||
# Save artifacts from pytests. | ||
- name: Upload pytest test results | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: pytest-results-${{ matrix.python-version }} | ||
path: pytest-results-${{ matrix.python-version }} | ||
# Use always() to always run this step to publish test results when there are test failures | ||
if: ${{ always() }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# Files to ignore | ||
*.pyc | ||
*.sif | ||
.coverage | ||
.DS_Store | ||
profile.html | ||
profile.json | ||
|
||
|
||
# Folders to ignore | ||
__pycache__ | ||
.mypy_cache | ||
.nox | ||
.pytest_cache | ||
html | ||
save | ||
runs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
repos: | ||
- repo: https://github.com/pre-commit/pre-commit-hooks | ||
rev: v4.5.0 | ||
hooks: | ||
- id: check-ast | ||
- id: check-byte-order-marker | ||
- id: check-builtin-literals | ||
- id: check-case-conflict | ||
- id: check-docstring-first | ||
- id: check-merge-conflict | ||
- id: check-symlinks | ||
- id: check-toml | ||
- id: check-xml | ||
- id: check-yaml | ||
- id: debug-statements | ||
- id: detect-private-key | ||
- id: end-of-file-fixer | ||
- id: name-tests-test | ||
- id: requirements-txt-fixer | ||
- id: sort-simple-yaml | ||
- id: trailing-whitespace | ||
- repo: https://github.com/pre-commit/pygrep-hooks | ||
rev: v1.10.0 # Use the ref you want to point at | ||
hooks: | ||
- id: python-check-blanket-noqa | ||
- id: python-check-mock-methods | ||
- id: python-no-eval | ||
- id: python-no-log-warn | ||
- id: python-use-type-annotations | ||
- id: text-unicode-replacement-char | ||
- repo: https://github.com/asottile/pyupgrade | ||
rev: v3.15.0 | ||
hooks: | ||
- id: pyupgrade | ||
- repo: https://github.com/psf/black | ||
rev: 23.12.1 # Replace by any tag/version: https://github.com/psf/black/tags | ||
hooks: | ||
- id: black | ||
language_version: python3 # Should be a command that runs python3.6+ | ||
args: ["--target-version", "py310"] | ||
- repo: https://github.com/asottile/reorder_python_imports | ||
rev: v3.12.0 | ||
hooks: | ||
- id: reorder-python-imports | ||
args: [--py3-plus] | ||
- repo: https://github.com/asottile/setup-cfg-fmt | ||
rev: v2.5.0 | ||
hooks: | ||
- id: setup-cfg-fmt | ||
- repo: https://github.com/pre-commit/mirrors-mypy | ||
rev: 'v1.8.0' # Use the sha / tag you want to point at | ||
hooks: | ||
- id: mypy | ||
- repo: https://gitlab.com/smop/pre-commit-hooks | ||
rev: 'v1.0.0' | ||
hooks: | ||
- id: check-gitlab-ci |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Commits to ignore (enter revision numbers here) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
{ | ||
"version": "0.2.0", | ||
"configurations": [ | ||
{ | ||
"name": "Python: Current File", | ||
"type": "python", | ||
"request": "launch", | ||
"program": "${file}", | ||
"console": "integratedTerminal", | ||
"cwd": "${workspaceFolder}", | ||
"env": { | ||
"PYTHONPATH": "${workspaceFolder}:${env:PYTHONPATH}" | ||
} | ||
}, | ||
{ | ||
"name": "Debug Unit Test", | ||
"type": "python", | ||
"request": "test", | ||
"justMyCode": false, | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
Mujoco | ||
SLURM | ||
Syndeo | ||
Apptainer |
Oops, something went wrong.