Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimised multi-stage docker build for cogstacksystems/medcat-trainer #127

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

vvcb
Copy link

@vvcb vvcb commented Mar 26, 2023

This is an optimised multi-stage docker build for cogstacksystems/medcat-trainer as described in detail in comments in response to #92.

  • Reduce uncompressed docker image size of cogstacksystems/medcat-trainer:v2.6.0 from 7.71GB to 6.13GB
    • Separate stage for building frontend saves ~450MB by only using the dist folder that is the output of npm build
    • Disable pip cache with pip install --no-cache-dir to save >900MB
    • Copy only required folders/files to save ~20MB
  • Create a non-root user to install dependencies and run container without elevated permissions
  • Fix path to manage.py in run.sh

Tested and working with docker-compose.yml with cogstacksystems/medcat-trainer-nginx:v2.6.0 and solr:8 (with default cdb.dat and vocab.dat).

Optimised multi-stage docker build for `cogstacksystems/medcat-trainer`

- Reduces uncompressed docker image size from 7.71GB to 6.13GB through
  - Separate stage for building frontend
  - Disable pip cache
  - Copy only required folders/files
- Run container as non-root
- Fix path to manage.py in run.sh

Tested and working with docker-compose.yml with `cogstacksystems/medcat-trainer-nginx:v2.6.0` and `solr:8`.
@tomolopolis
Copy link
Member

hi @vvcb - thanks for the contribution here. I tried building and running locally:
$ docker compose -f docker-compose-dev.yml up

when I navigate to http://localhost:8001/ - the UI doesn't come up, /admin/ does appear though.

Also on the api container - can we still have vim installed?

I've found it handy in a pinch

Copy link
Member

@tomolopolis tomolopolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small fixes before the change can be merged plz

FROM python:3.10

# Update and upgrade everything
RUN apt-get -y update && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we could include a vim install for container / image debugging

ARG SPACY_MODELS="en_core_web_md"
RUN for SPACY_MODEL in ${SPACY_MODELS}; do python -m spacy download ${SPACY_MODEL}; done
RUN pip install pip --upgrade --no-cache-dir \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice with --no-cache-dir , makes sense to add

@@ -14,7 +14,7 @@ echo "from django.contrib.auth import get_user_model
User = get_user_model()
if User.objects.count() == 0:
User.objects.create_superuser('admin', '[email protected]', 'admin')
" | python manage.py shell
" | python /home/api/manage.py shell

if [ $LOAD_EXAMPLES ]; then
python /home/load_examples.py &
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think WORKDIR used to be in /home/api so running python manage.py would have relative path serving up the static content directly for container available debugging purposes.

Copy link
Member

@tomolopolis tomolopolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small fixes before the change can be merged plz

@tomolopolis tomolopolis mentioned this pull request Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants