Add docker files

[whitehara]

Thank you for providing nice application.

I added docker related files.

Key features:

• You can use pre-build container from dockerhub, or build your image.
• You can use environment variables for setting parameters in config.txt
• models and outputs are stored in a persistent volume.

Please also see docker.md for details.

I also removed 'realpath' from 'generate_temp_filename'.
If the real path is not under the application folder but is symlinked under the application folder, it shows "Not allowed" error because the symlinked path is not used.

EDIT @mashb1t: closes #98

[mashb1t]

relates to #1154

[mashb1t]

using a virtual volume here will increase the loading speed of models in Fooocus. Please test and adjust accordingly. See https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose
Please also consider using a mount for the outputs directory as well as the python cache dir (most likely /home/user/.cache) as keeping cache non-persistent will lead to the container having to download many models on container start after destruction again.

[whitehara]

Thank you for your comment.

• Unfortunately, I can't find the word, "virtual volume" in the URL. What is the "virtual volume"?
• "models" directory is in the persistent volume. So you don't need to download them after the second boot.
• "outputs" directory is also in the persistent volume. So you can see your previous outputs after the second boot.
• Generate python precompiled cache dir __pycache__ in Dockefile. (5b30983). However, even without this cache, the .pyc files will be generated with a very short time.

[mashb1t]

Maybe "virtual volume" was a bit misleading, what i meant was as non-disk-mounted volume with definition in the docker-compose's volume section (not external), see example in the provided link.

Some external models, not checkpoints, will be downloaded to /home/user/.cache, such as the CompVis safety checker (which arguably hasn't been merged yet and has 1GB). This would have to be downloaded on every container start or on usage of specific features, which would delay the process.
This might not be the case for all users though and can be easily adjusted by affected users.

[mashb1t]

@whitehara one addition: did you use --share by intention? --listen is sufficient in this case to use Fooocus in Docker locally + via port on the client (+ internet if forwarded and intended) and not everybody wants / is expecting their instance to be publicly available in the internet via share link.
Anyways, good job!

[whitehara]

@mashb1t Thank you for your comments again.

• Do you mean "myapp" is "virtual volume" in the below sample ,do you?

services:
  frontend:
    image: node:lts
    volumes:
      - myapp:/home/node/app      
volumes:
  myapp:

If so, "myapp" is created in the "/var/lib/docker/volumes"(default linux docker setting) and the performance of the volume depends on the location's storage(ssd, hdd, etc.)
However, I don't recommend using this, because many users don't have a large space in /var/lib/docker, but we need a lot of space for model s, and users should make a space in the different partition.

• Users who want to make /home/user/.cache as a persistent volume, They can use this.

  volumes:
    - /content/data/cache:/home/user/.cache

Using other places are also OK. It depends on their needs, and users can modify docker-compose.yml easily.

• Yes, I use ---share as default because I want to make user can try easily. As you said, users (including me) who want to keep the container running should use --listen. But they need to change their network settings. That is not so easy as using --share.

[mashb1t]

Thank you for the explanation. I totally get your points, but still have my doubts concerning default speed and privacy.

1. A volume mount to host without delegation is about 20x slower than a docker volume without mount to the host (even using an NVMe SSD, due to how file system events are handled in Docker), so i'd strongly suggest to use no volume mount to host by default.
   Are you fine with adding the volume mount as commented out variant with potential additional remark in the README.md?
2. As using --share can lead to other users unexpectedly accessing one's instance and privacy breaches, we should also not make this the default but use --listen.
   Same here, would you be fine with adding a line to comment in with --share and a comment?

[whitehara]

Thank you for your comments again.
Because I only use docker on Linux, I hadn't known the slow "bind mount" problem on Mac/Windows. I added notes in docker.md.
I also added notes about not using --share for using a long time.

[realies]

Volume mounts are the standard for appdata such as models, regardless of the speed limitations. It would be better if this is the default and if anyone has any concerns about speed, they can switch to volume without mount.

[whitehara]

OK. Now 'docker-compose.yml' is changed not to use the bind mount.
In this usage, users need to use local built image by using docker compose up --build for preventing the permission error.

[mashb1t]

@whitehara Currently doing a re-review of this great PR as I'm now permitted to merge this feature.
Here are my findings:

• use LF instead of CLRF as line separator or entrypoint.sh will not be found / executable. This is not committable in git afaik. No change needed here, but readme entry should be added.
  
• container always auto updates Fooocus to latest version. I think it's better to use launch.py in entrypoint instead of entry_with_update.py and add instructions to git pull and docker compose build to the docker readme
• container building works perfectly fine on an empty fooocus isntallation, but can basically not be built in an environment which already has models installed as build context size explodes. I'd propose to add the models and outputs folder to the .dockerignore as they're not needed in the image and use a volume mount (bind to host) to models.orig, then copying to docker volume. This will lead to double the model space requiremets, but compared to the current setup (3x model space needed, 1. existing models if present, 2. image, 3. docker volume) this still is acceptible.

What do you think?

[whitehara]

@mashb1t Thank you for your review again.

• use LF instead of CLRF as line separator or entrypoint.sh will not be found / executable. This is not committable in git afaik. No change needed here, but readme entry should be added.

I'm using LF in my environment. So I can not reproduce this problem and don't know how to fix. But I modified CMD in Dockerfile to suspect to change the behavior.

• container always auto updates Fooocus to latest version. I think it's better to use launch.py in entrypoint instead of entry_with_update.py and add instructions to git pull and docker compose build to the docker readme

I modified entrypoint.sh to launch launch.py and also modified docker.md

• container building works perfectly fine on an empty fooocus installation, but can basically not be built in an environment which already has models installed as build context size explodes. I'd propose to add the models and outputs folder to the .dockerignore as they're not needed in the image and use a volume mount (bind to host) to models.orig, then copying to docker volume. This will lead to double the model space requiremets, but compared to the current setup (3x model space needed, 1. existing models if present, 2. image, 3. docker volume) this still is acceptible.

I added import function to import files from models, outputs folder.
For preventing copying the unnecessary files from the current folder, original files are cloned from GitHub when building the container. So I use my GitHub repo temporary. It can be modified when you merge.

[mashb1t]

@whitehara great to still have you on board! Sadly i don't have full access to this repository and can't set up secrets for the actions, which is why we'd have to either remove the action or set up a new Dockerhub account and mirror Fooocus somehow. I'd propose to go for route 1(not yet merging the action), but keep the rest.

[whitehara]

@whitehara great to still have you on board! Sadly i don't have full access to this repository and can't set up secrets for the actions, which is why we'd have to either remove the action or set up a new Dockerhub account and mirror Fooocus somehow. I'd propose to go for route 1(not yet merging the action), but keep the rest.

@mashb1t No problem. You can omit the GitHub Actions for building and releasing the container image. I (and maybe others, too) will keep doing that in our own repo.
Please add it as a new issue, so we can close this PR after your merge.

[realies]

@lllyasviel @mashb1t, setting up a couple of secrets can't be that hard, right?

[mashb1t]

@whitehara we have two possibilities now:

1. not write

   Run git pull before executing docker compose up --build to update Fooocus.

   but rather

   Run docker compose build --no-cache && docker compose up to update Fooocus.

2. tag the image manually and change version number in docker-compose.yml, so the build will correctly trigger on command execution.

   image: fooocus:2.1.865

As there is no feedback from @lllyasviel yet I'd propose to go for the 1st route and remove the github action for now, which also doesn't build for multi-arch arm64 + amd64, so M1/M2 mac users can't use it in the current state.

What do you think?

[whitehara]

@mashb1t Thank you for your checking again.

1. not write

   Run git pull before executing docker compose up --build to update Fooocus.

   but rather

   Run docker compose build --no-cache && docker compose up to update Fooocus.

I recommend to keep "Run git pull before executing...". Because without this, users will miss the updates about the docker related files like Dockerfile, etc.

which also doesn't build for multi-arch arm64 + amd64, so M1/M2 mac users can't use it in the current state.

I modified module requirements for building the linux/arm64 image, and I just start to provide the image on my dockerhub repo "whitehara/fooocus". But, unfortunately, I don't have any test environment about arm64. If you have the environment, please try it.

[mashb1t]

@whitehara thank you for the adjustments.
I was able to successfully pull the image from whitehara/fooocus and after removing driver: nvidia and the config below tty: true the image could be started on my M1 Macbook. There seems to be a conceptual issue though:

Torch Error

app-1  | Exception in thread Thread-4 (worker):
app-1  | Traceback (most recent call last):
app-1  |   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
app-1  |     self.run()
app-1  |   File "/usr/lib/python3.10/threading.py", line 953, in run
app-1  |     self._target(*self._args, **self._kwargs)
app-1  |   File "/content/app/modules/async_worker.py", line 25, in worker
app-1  |     import modules.default_pipeline as pipeline
app-1  |   File "/content/app/modules/default_pipeline.py", line 1, in <module>
app-1  |     import modules.core as core
app-1  |   File "/content/app/modules/core.py", line 1, in <module>
app-1  |     from modules.patch import patch_all
app-1  |   File "/content/app/modules/patch.py", line 5, in <module>
app-1  |     import ldm_patched.modules.model_base
app-1  |   File "/content/app/ldm_patched/modules/model_base.py", line 2, in <module>
app-1  |     from ldm_patched.ldm.modules.diffusionmodules.openaimodel import UNetModel, Timestep
app-1  |   File "/content/app/ldm_patched/ldm/modules/diffusionmodules/openaimodel.py", line 15, in <module>
app-1  |     from ..attention import SpatialTransformer, SpatialVideoTransformer, default
app-1  |   File "/content/app/ldm_patched/ldm/modules/attention.py", line 9, in <module>
app-1  |     from .sub_quadratic_attention import efficient_dot_product_attention
app-1  |   File "/content/app/ldm_patched/ldm/modules/sub_quadratic_attention.py", line 27, in <module>
app-1  |     from ldm_patched.modules import model_management
app-1  |   File "/content/app/ldm_patched/modules/model_management.py", line 118, in <module>
app-1  |     total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
app-1  |   File "/content/app/ldm_patched/modules/model_management.py", line 87, in get_torch_device
app-1  |     return torch.device(torch.cuda.current_device())
app-1  |   File "/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py", line 674, in current_device
app-1  |     _lazy_init()
app-1  |   File "/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py", line 239, in _lazy_init
app-1  |     raise AssertionError("Torch not compiled with CUDA enabled")
app-1  | AssertionError: Torch not compiled with CUDA enabled
app-1  | Running on local URL:  http://0.0.0.0:7865
app-1  | 
app-1  | To create a public link, set `share=True` in `launch()`.

pytorch/pytorch#81224

It is highly unlikely that accelerated inference will ever be available via Docker on Apple Silicon. This is due to Apple's implementation of Docker, where it https://github.com/ml-explore/mlx/discussions/257with a weirdly virtualised ARM CPU that does not expose the underlying CPU/GPU capabilities. Asahi Linux pytorch/pytorch#81224.
However, I have an M1 Mac and I would like to use the hardware with text-generation-webui. Unfortunately, Apple does not provide any appropriate containerisation or virtualisation technologies - so I will be exploring different packaging options. If I have any developments worth sharing, I will post them here.

Atinoda/text-generation-webui-docker#22 (comment)

I don't think there will be support for MPS with docker any time soon...

After adding --always-cpu to CMDARGS i still wasn't able to successfully start the container, receiving this error:

Process Kill Error

app-1  | To create a public link, set `share=True` in `launch()`.
app-1  | Total VRAM 9946 MB, total RAM 9946 MB
app-1  | xformers version: 0.0.22
app-1  | Set vram state to: DISABLED
app-1  | Always offload VRAM
app-1  | Device: cpu
app-1  | VAE dtype: torch.float32
app-1  | Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --attention-split
app-1  | Refiner unloaded.
app-1  | model_type EPS
app-1  | UNet ADM Dimension 2816
app-1  | /content/entrypoint.sh: line 33:    16 Killed                  python launch.py $*
app-1 exited with code 137

After increasing from 10GB => 20GB memory as well as 1GB => 4GB swap in Docker Desktop generation still doesn't start due to an issue after loading the model:

Error after model load

pp-1  | [Fooocus] Processing prompts ...
app-1  | [Fooocus] Encoding positive #1 ...
app-1  | [Fooocus] Encoding positive #2 ...
app-1  | [Parameters] Denoising Strength = 1.0
app-1  | [Parameters] Initial Latent shape: Image Space (896, 1152)
app-1  | Preparation time: 3.98 seconds
app-1  | Using lcm scheduler.
app-1  | [Sampler] refiner_swap_method = joint
app-1  | [Sampler] sigma_min = 0.39970141649246216, sigma_max = 14.614640235900879
app-1  | Requested to load SDXL
app-1  | Loading 1 new model
app-1  | /content/entrypoint.sh: line 33:    16 Killed                  python launch.py $*
app-1 exited with code 137

This is also when i gave up to make it work on the Macbook. It seems to be working with neither MPS nor --always-cpu.

[whitehara]

@mashb1t Thank you for your testing.

This is also when i gave up to make it work on the Macbook. It seems to be working with neither MPS nor --always-cpu.

I think so, too. The "linux/arm64" image has a binary compatibility, but it still needs some tweaks for running on M1 Mac.
Because I don't have the machine, I can't debug now. Sorry.