Add full local setup with docker

[Newbas]

• Add separate configuration for s3 endpoint
• Add dockerfile to build backend image
• Add docker-compose with minio and rabbitmq
• Add MacOS setup guide

[Newbas]

Hello everyone, let me first appreciate your work. I was setting everything on my computer and thought that it can be helpful and simplify process of local setup. Also added instruction for MacOS, only issue on my Mac psycopg2==2.8.6 does not work, after update to latest (2.9.2) pip install worked fine. With docker installation 2.8.6 works fine.

If you think it is valuable I can add description for setup through docker.

[biblicabeebli]

This is awesome. Docker infrastructure, especially w/respect the work on the Forest runtime environment and task dispatch, is on my wish list and I guess I wished really hard.

High level comments and a couple tasks for you. Definitely ping me if you need something:

• Sooooo the eli/django branch is about to become main. I'm doing device testing right now, and it is a substantial developer quality of life, tech fundamentals, and tech stack upgrade; also there are any tests and some are even good. I don't think there will be conflicts exactly, but will need to be updated for to that. (Sorry about the worst timing here.) You can track updates on the Drop Flask entirely for Django #255 issue, expect flurries of activity.
• Python 3.7 - Python 3.6 -> 3.8 platform upgrade #262 - I just added a comment there, the eli/django branch is Python 3.8 compatible as of Monday. (I believe I know exactly the bug involving pycrypto that you may have run into that stuck you on 3.7. The underlying issue is documented in encryption.py on the django branch.)
• Directions: Yes, this would be amazing, especially if you could also update the Readme to include the necessary Brew installs. (we've had the issue for a while, but lacked a mac to do testing.) Edit: does this make more sense to include in the Readme, or would a wiki page be better? If so I can give you write access to the wiki.

Other comments:

• I had that psycopg2 error too when I was reviewing pip installs recently on the eli/django branch, my comment in requirements.in is "You have to upgrade django to greater than 2.2 (apparently) in order to upgrade psycopg2 to 2.9.x". Also, I'm considering swapping to psycopg2-binary to reduce the requirements bullcrap.
• Were some of your issues about mac support an issue because we don't have directions for the required Brew installs?

Question:
I did not know there was a parameter for a specific S3 target url, do you know if that would let us point at S3 compatible services like Linode's Object Storage or Wasabi?

[Newbas]

Great to hear that it is helpful.

• I checked django branch already, since having flask and django as dependencies made me wondering is it django or flask in the end.
• I tried to set 3.8 and got module 'time' has no attribute 'clock' so after merge should not be hard to update docker image.
• I updated the doc for brew setup, at least steps I did. Set postgre (and psycopg2 version) and it worked. Could be that something was installed already because of other developments. libpq-dev does not exist in brew, but it worked without it.

Answer:
I have not tried to use it with different providers in python only with minio, had experience using Scala, I tried to use it with Digital Ocean Storage, and had issues with part of the APIs. Bulk functions did not work.

I will:

• Add description for Docker setup.
• Crontab for celery is missing here, but went over production setup, can be added there.

Question:

• i saw that installing cron sets forest, but i have not checked it much yet. Is it also Celery tasks? Or something else should be added.

For Cluster setup (if someone needed), can be used docker-swarm, or some load balancer. But i will leave it a side for the moment.

If you think something is missed let me know.

[biblicabeebli]

Seen but not reviewed, simply busy elsewhere, just didn't get to this during work hours today.
Several pushes today on various branches, including a merge into the django branch of main, and a bugfix I had forgotten to merge is now on main. Chugging through other things so it may be a few days before I have some time.

[biblicabeebli]

Stiiiiill busy...