[Cloud Deployment I]: Main dockerfile

dockerfiles/neuroconv_developer_build_auto_config

@@ -0,0 +1,8 @@
+FROM python:3.10
+LABEL org.opencontainers.image.source=https://github.com/catalystneuro/neuroconv
+ADD /neuroconv /neuroconv
+RUN cd neuroconv \
+ && pip install -e .[full]
+CMD echo "$YAML_STREAM" > run.yml \
+  && cat run.yml \
+  && neuroconv run.yml

dockerfiles/neuroconv_developer_build_dockerfile

@@ -0,0 +1,6 @@
+FROM python:3.10
+LABEL org.opencontainers.image.source=https://github.com/catalystneuro/neuroconv
+ADD /neuroconv /neuroconv
+RUN cd neuroconv \
+ && pip install -e .[full]
+CMD ["python3 -m"]

dockerfiles/neuroconv_latest_release_dockerfile

@@ -0,0 +1,4 @@
+FROM python:3.10
+LABEL org.opencontainers.image.source=https://github.com/catalystneuro/neuroconv
+RUN pip install neuroconv[full]
+CMD ["python3 -m"]

dockerfiles/rclone_auto_config_dockerfile

@@ -0,0 +1,5 @@
+FROM rclone/rclone:latest
+CMD printf "[$DRIVE_NAME]\ntype = drive\nscope = drive\ntoken = {\"access_token\":\"$TOKEN\",\"token_type\":\"Bearer\",\"refresh_token\":\"$REFRESH_TOKEN\",\"expiry\":\"$EXPIRY\"}" > /mnt/data/rclone.conf \
+  && cat /mnt/data/rclone.conf \
+  && rclone $RCLONE_COMMAND
+ENTRYPOINT [""]

docs/developer_guide.rst

@@ -15,5 +15,6 @@ The most common contribution is for a user/developer to help us integrate a new
     Testing Suite <developer_guide/testing_suite>
     Coding Style <developer_guide/style_guide>
     Building the Documentation <developer_guide/building_documentation>
+    Building the Docker Image <developer_guide/docker_images.rst>
 
 Or feel free to raise a `bug report <https://github.com/catalystneuro/neuroconv/issues/new?assignees=&labels=bug&template=bug_report.yml&title=%5BBug%5D%3A+>`_ or `feature request <https://github.com/catalystneuro/neuroconv/issues/new?assignees=&labels=enhancement&template=feature_request.yml&title=%5BFeature%5D%3A+>`_ for our maintainers to prioritize!

docs/developer_guide/docker_images.rst

@@ -0,0 +1,94 @@
+Manually Build Docker Images
+----------------------------
+
+.. note: It is highly recommended to build the docker image on the same basic system or architecture type that you intend to run it on, *i.e.*, AWS Linux AMI 64-bit (x86), as it may experience difficulties running on other radically different systems (like an M1 Mac).
+
+.. note: The NeuroConv docker container comes prepackaged with all required installations, *i.e.*, equivalent to `pip install neuroconv[full]`. As such it is fairly heavy, so be sure that whatever system (or specifically CI environment) you build with has sufficient disk space.
+
+
+Latest Release
+~~~~~~~~~~~~~~
+
+.. code:
+
+    docker build -f neuroconv_latest_release_dockerfile -t neuroconv_latest_release .
+
+
+Dev Branch
+~~~~~~~~~~
+
+Checkout to a specific branch on a local clone, then
+
+.. code:
+
+    docker build -f neuroconv_developer_build_dockerfile -t neuroconv_dev .
+
+
+
+Publish Container to GitHub
+---------------------------
+
+The `LABEL` is the important item here; it determines the host repository on the GitHub Container Registry (GHCR). In each docker file we wish to publish on the GHCR, we will add this label right after the `FROM` clause
+
+.. code:
+
+    FROM PARENT_IMAGE:TAG
+    LABEL org.opencontainers.image.source=https://github.com/catalystneuro/neuroconv
+
+After building the image itself, we can publish the container with
+
+.. code:
+
+    docker tag IMAGE_NAME ghcr.io/catalystneuro/IMAGE_NAME:TAG
+    export CR_PAT="<YOUR GITHUB SECRET TOKEN>"
+    echo $CR_PAT | docker login ghcr.io -u <YOUR GITHUB USERNAME> --password-stdin
+    docker push ghcr.io/catalystneuro/IMAGE_NAME:TAG
+
+.. note: Though it may appear confusing, the use of the `IMAGE_NAME` in these steps determines only the _name_ of the package as available from the 'packages' screen of the host repository; the `LABEL` itself ensured the upload and linkage to the NeuroConv GHCR.
+
+
+
+Running Docker Container on local YAML (Linux)
+----------------------------------------------
+
+You can either perform a manual build locally following the instructions above, or pull the container from the GitHub Container Registry (GHCR) with
+
+.. code:
+
+    docker pull ghcr.io/catalystneuro/neuroconv:latest
+
+and can then run the entrypoint (equivalent to the usual CLI usage) on a YAML specification file (named `your_specification_file.yml`) with
+
+.. code:
+
+    docker run -it --volume /your/local/drive/:/desired/alias/of/drive/ ghcr.io/catalystneuro/neuroconv:latest neuroconv /desired/alias/of/drive/your_specification_file.yml
+
+
+
+Running Docker Container with a YAML_STREAM (Linux)
+---------------------------------------------------
+
+An alternative approach that simplifies usage on AWS Batch is to specify the YAML contents as an environment variable. The file is constructed in the first step of the container launch. The only potential downside with this usage is maximum file size (~13,000 characters; current example files do not come remotely close to this).
+
+Otherwise, the YAML file transfer will have to managed separately, likely as a part of the data transfer or an entirely separate transfer step.
+
+To use manually outside of AWS Batch,
+
+.. code:
+
+    export YAML_STREAM="<copy and paste contents of YAML file (manually replace instances of double quotes with single quotes)>"
+    docker run -it --volume /your/local/drive/:/desired/alias/of/drive/ ghcr.io/catalystneuro/neuroconv:dev_auto_yaml
+
+To use automatically via a Python helper function (coming in a separate PR)
+
+.. code:
+    import os
+
+    with open(file="my_yaml_file.yml") as file:
+        yaml_stream = "".join(file.readlines()).replace("\"", "'")
+
+    os.environ["YAML_STREAM"] = yaml_stream
+
+.. note:
+
+    When  using YAML files through the docker containers, always be sure that the NWB file paths are absolute paths stemming from the mounted volume; otherwise, the NWB file will indeed be written inside the container but will not be accessible outside of it.