Separate code from docker setup

[derneuere]

Related to #158

It is currently not possible to mount the directory of the code from the developer machine to the docker container, because multiple dependencies are directly saved into the code folder at docker build time.

We have to move the logs and the protected media to a different place and update the code:

librephotos/docker-compose.yml

Lines 49 to 50 in 205a040

	- ${proMedia}:/code/protected_media
	- ${logLocation}:/code/logs

We also have to move the models and data of the machine learning applications:

librephotos/Dockerfile

Lines 46 to 48 in 205a040

	RUN curl -SL https://s3.eu-central-1.amazonaws.com/ownphotos-deploy/places365_model.tar.gz | tar -zxC /code/api/places365/
	RUN curl -SL https://s3.eu-central-1.amazonaws.com/ownphotos-deploy/im2txt_model.tar.gz | tar -zxC /code/api/im2txt/
	RUN curl -SL https://s3.eu-central-1.amazonaws.com/ownphotos-deploy/im2txt_data.tar.gz | tar -zxC /code/api/im2txt/

We have to either change this to config or put it in a seperate folder:

librephotos/Dockerfile

Line 61 in 205a040

RUN mv /code/config_docker.py /code/config.py

Also, a spacy model gets downloaded into the code folder. Not sure why we need that:

librephotos/Dockerfile

Line 54 in 205a040

RUN /miniconda/bin/python -m spacy download en_core_web_sm

[parruc]

I was starting to try to work on this.

Are there specific reasons why we are not using nginx official (and mantained) image and python official images as we do with postgres instead of custom proxy image and ubuntu + miniconda installed images?

If you could open a repo under libreoffice only for the docker part I can start working on it.

[derneuere]

Never looked to hard at the nginx stuff. You are right, the current implementation looks redundant, because you could just mount the .conf file. Also there are two more .conf files in the backend and frontend repo. Not sure what they do.

I am all for switching back to python. That would also make #59 possible I think.

What kind of files do you want to put into the docker repository? All the docker-compose.yml stuff or other files too? Should I call it librephotos-docker or librephotos-docker-compose?

[parruc]

I don't have a strong opinion on the repository name. I'm putting in it basically docker-compose dockerfiles, entrypoint.sh for both frontend and backend and nginx config. I'm also trying to create 2 docker-compose.yml the default that uses prebuilt images and the other (for dev) that builds the images. The dev one also adds some tty, volume and could add an env var that we could use in django settings to decide if debug is true or false, logging verbosity and other dev perks.

Is there some reason I don't understand we are compiling dlib from source instead of simply installing python package? In case there is not one I suggest to put it in requirements.txt so it gets normally installed. Also there is a ton of python package in requirements. Are we really using them all? Doing some cleaning would allow us to have a slimmer and faster image to work with.

BTW It is becoming a major rewrite so some testing/help will be probably needed if I manage to not break everything ^_^

[derneuere]

Alright here is the new repo: https://github.com/LibrePhotos/librephotos-docker
Just open up a pull request with your changes :D

We have to keep in mind to also update the readme and the ci workflow.

Dlib has to be compiled afaik. The documentation says that the setup.py will choose the compile parameters based on the build hardware. But of course our users have different systems, so we have to disable certain instruction sets. Maybe there is a way to just add it to the requirements.txt, but I don't know how :D

Yes, there are a lot of dependencies. There are probably a couple of not needed dependencies, but I haven't found a way to show unused dependencies in a requirements.txt. If you want, you can absolutely try to figure out which ones are needed and which ones aren't :D

I will test it! If you open up a pull request I will try your dockerfile and docker-compose file!

[parruc]

I did A LOT of work on both containers and structure/features etc.

There are now 3 pull request: one per project. Now it's your time to have fun and review all that work ;-)

[alexislefebvre]

What would be the benefits to use several GitHub repositories? Would it make development of LibrePhotos more complicated if it means cloning several repos?

[derneuere]

We can more easily deploy the code in our dev environment. Strict separation of concerns also leads to other benefits. For example, it will be easier to change the deployment type, because it is clear which part is code and always needed and which part is deployment specific.

[alexislefebvre]

Ok thanks. I still think having sub-folders in one repo is easier than managing code in several repos.