Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
[x] Pre-commit checks pass
  • Loading branch information
destin-v committed Sep 25, 2024
0 parents commit 180f183
Show file tree
Hide file tree
Showing 80 changed files with 6,838 additions and 0 deletions.
81 changes: 81 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
##########################################################################################
#
# DESCRIPTION: Development Docker Container. Installs the following:
# * miniconda
# * Oh-My-Zsh
# * awesome-vim
#
# GPU SUPPORT: In order to get GPU support the nvidia image's CUDA version must
# match the version on the host computer.
#
# RENDERING: If rendering to the screen is required then execute the following
# prior to starting the container:
#
# ```console
# xhost +local:root
# ```
#
# AUTHOR: W. Li
# VERSION: 1.0
# CREATED: 12/15/2023
##########################################################################################

# Source Image (this should match the CUDA version on the host)
# FROM nvidia/cuda:11.4.1-base-ubuntu20.04
FROM ubuntu:20.04

# Labels
LABEL version="1.0"
LABEL description="Miniconda Dev Image"

# Expose ports here
# EXPOSE 8080

# Install basic packages needed for development
RUN yes Y | apt update
RUN yes Y | apt-get update
RUN yes Y | apt upgrade
RUN yes Y | apt install vim
RUN apt-get install -y --no-install-recommends \
git \
wget \
g++ \
gcc \
&& rm -rf /var/lib/apt/lists/*

# Setup shell
RUN sh -c "$(wget -O- https://github.com/deluan/zsh-in-docker/releases/download/v1.1.1/zsh-in-docker.sh)" -- \
-t robbyrussell \
-p git -p ssh-agent -p 'history-substring-search' \
-a 'bindkey "\$terminfo[kcuu1]" history-substring-search-up' \
-a 'bindkey "\$terminfo[kcud1]" history-substring-search-down'

# Setup Shell
SHELL ["/bin/zsh", "-c"]

# Miniconda requires that the path point to its python binary
ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"

# Source python versions: https://repo.anaconda.com/miniconda/
RUN wget \
https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
&& mkdir /root/.conda \
&& bash Miniconda3-latest-Linux-x86_64.sh -b \
&& rm -f Miniconda3-latest-Linux-x86_64.sh

RUN conda init zsh

# Setup Vim-Awesome
RUN git clone --depth=1 https://github.com/amix/vimrc.git ~/.vim_runtime \
&& sh ~/.vim_runtime/install_basic_vimrc.sh

# Setup working directory
WORKDIR /projects

# Copy folder/files into container (must be within context directory)
# Process: local files -> artifact dir -> container dir
# COPY <local_path> <container_path>

# Environment
ENV SHELL /bin/zsh
60 changes: 60 additions & 0 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# 📒 Description
## 🐳 Docker
* Primary method for building and maintaining **un-secure** containers.
* Must have root access in order to work.
* Deployable to any machine.
* [Additional Information](#🐳-docker-notes)

## 🦉 Apptainer
* Primary method for building and maintaining **secure** containers.
* Used primarily in settings where root access is not possible.
* Deployable to any machine.
* [Additional Information](#🦉-apptainer-notes)

## 📦 Packages
The default development containers will these packages by default. Only the essentials are included.

* [Miniconda](https://docs.conda.io/projects/miniconda/en/latest/)
* [Oh My Bash](https://github.com/ohmybash/oh-my-bash)
* [Oh My Zsh](https://ohmyz.sh)
* [Precommit](https://pre-commit.com)

# 🐳 Docker Notes

## 🎼 Docker Compose
* Allows you to create multiple different builds using different Docker files.
* The context directory can be set from the `docker_compose.yml` file.
* More options than a standard Docker file.

## 👟 Running Commands
* In Docker each `RUN` command is treated as a new shell instance. This means that commands cannot persist across different RUN commands.

```Docker
RUN command1 # this command is run in its own shell!
RUN command2 # this command is run in its own shell!
```

## 🗃️ Copying Folders/Files
* If you need to copy files or directories into the Docker container, first put them in an **artifacts** directory that can be seen from the context directory.

```Docker
COPY <artifact_directory> <container_path>
```
## 📂 Context Directory

* By default Dockerfiles are only aware of what is in their **context directory**. A context directory is the directory where a Dockerfile is located in. It cannot access any files outside of the context directory.

* The context directory can be set from the command line or from a `docker_compose.yml` file.

* It is recommended that all folders/files be copied be placed in an **artifacts** directory that is under the context directory.

## 💾 Out of Memory
* Occasionally you will need to clean your docker images and docker volumes in order to recover memory.

```console
docker volume prune
docker image prune
docker container prune
```

# 🦉 Apptainer Notes
9 changes: 9 additions & 0 deletions .devcontainer/build.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""
Run this script to build the Docker development image.
"""
import subprocess

image_name = "development-base"

# Build the image from the development Dockerfile
subprocess.run(["docker", "build", "-t", image_name, "."])
44 changes: 44 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.191.0/containers/ubuntu
{
"name": "dev-python",
"image": "development-base",
"build": {
"dockerfile": "Dockerfile",
},
// Add arguments that run with the container
// These are arguments needed for GPU support on a Linux machine
"runArgs": [
// "--net=host",
// "-m=10g",
// "--shm-size=10g",
// "--gpus=all",
// "-v",
// "/tmp/.X11-unix:/tmp/.X11-unix",
// "-e",
// "DISPLAY",
// "--device",
// "/dev/dri"
],
"customizations": {
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {},
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"eamodio.gitlens",
"littlefoxteam.vscode-python-test-adapter",
"ms-python.python",
"mhutchie.git-graph",
"njpwerner.autodocstring",
"pkief.material-icon-theme",
]
}
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "uname -a"
// Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
// "remoteUser": "vscode"
}
25 changes: 25 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
##########################################################################################
# DESCRIPTION: Run pre-commit on all files in repo.
# AUTHOR: W. Li
# VERSION: 1.0
# CREATED: 1/6/2024
#
# References:
# * https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-pythonCommon
#
##########################################################################################

name: pre-commit

on:
pull_request:
push:
branches: [main]

jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: pre-commit/[email protected]
65 changes: 65 additions & 0 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
##########################################################################################
# DESCRIPTION: Runs pytests with multiple versions of python.
# AUTHOR: W. Li
# VERSION: 1.0
# CREATED: 1/6/2024
#
# References:
# * https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-pythonCommon
#
##########################################################################################

name: pytests

on:
# Configure the branches you want Github actions to run on.
# Leave blank if you want to run on all branches.
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

# Configure jobs
jobs:
test:
# Set the OS and python versions
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11"]
# Checkout the repo
steps:
- uses: actions/checkout@v4
# Setup the python environments to use.
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
# Install poetry and disable virtual environments.
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: false
virtualenvs-in-project: false
installer-parallel: true
# Install dependencies.
- name: Install dependencies
run: |
poetry install --no-interaction --with=dev --no-root
# Run pytest with coverage.
- name: Test with pytest
run: >
pytest
--doctest-modules
--cov=./
--cov-report=xml
--cov-report=html:pytest-results-${{ matrix.python-version }}
# Save artifacts from pytests.
- name: Upload pytest test results
uses: actions/upload-artifact@v3
with:
name: pytest-results-${{ matrix.python-version }}
path: pytest-results-${{ matrix.python-version }}
# Use always() to always run this step to publish test results when there are test failures
if: ${{ always() }}
17 changes: 17 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Files to ignore
*.pyc
*.sif
.coverage
.DS_Store
profile.html
profile.json


# Folders to ignore
__pycache__
.mypy_cache
.nox
.pytest_cache
html
save
runs
57 changes: 57 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: check-ast
- id: check-byte-order-marker
- id: check-builtin-literals
- id: check-case-conflict
- id: check-docstring-first
- id: check-merge-conflict
- id: check-symlinks
- id: check-toml
- id: check-xml
- id: check-yaml
- id: debug-statements
- id: detect-private-key
- id: end-of-file-fixer
- id: name-tests-test
- id: requirements-txt-fixer
- id: sort-simple-yaml
- id: trailing-whitespace
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.10.0 # Use the ref you want to point at
hooks:
- id: python-check-blanket-noqa
- id: python-check-mock-methods
- id: python-no-eval
- id: python-no-log-warn
- id: python-use-type-annotations
- id: text-unicode-replacement-char
- repo: https://github.com/asottile/pyupgrade
rev: v3.15.0
hooks:
- id: pyupgrade
- repo: https://github.com/psf/black
rev: 23.12.1 # Replace by any tag/version: https://github.com/psf/black/tags
hooks:
- id: black
language_version: python3 # Should be a command that runs python3.6+
args: ["--target-version", "py310"]
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.12.0
hooks:
- id: reorder-python-imports
args: [--py3-plus]
- repo: https://github.com/asottile/setup-cfg-fmt
rev: v2.5.0
hooks:
- id: setup-cfg-fmt
- repo: https://github.com/pre-commit/mirrors-mypy
rev: 'v1.8.0' # Use the sha / tag you want to point at
hooks:
- id: mypy
- repo: https://gitlab.com/smop/pre-commit-hooks
rev: 'v1.0.0'
hooks:
- id: check-gitlab-ci
Empty file added .vscode/.env
Empty file.
1 change: 1 addition & 0 deletions .vscode/configs/.git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Commits to ignore (enter revision numbers here)
22 changes: 22 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"cwd": "${workspaceFolder}",
"env": {
"PYTHONPATH": "${workspaceFolder}:${env:PYTHONPATH}"
}
},
{
"name": "Debug Unit Test",
"type": "python",
"request": "test",
"justMyCode": false,
}
]
}
4 changes: 4 additions & 0 deletions .vscode/ltex.dictionary.en-US.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Mujoco
SLURM
Syndeo
Apptainer
Loading

0 comments on commit 180f183

Please sign in to comment.