[SUGGESTION] Remove /var/ folder from composer installation in docker volume

[Wingmk]

Currently the TYPO3 backend is slow in docker installations. Even if a delegated volume is mounted.
If TYPO3 is setup as composer we can stop mounting the /var/ folder to the docker container.

I ran some Tests after removing the folder from mounting to docker and it runs a lot faster, especially the backend.

There might also be reasons to remove the typo3temp folder from mounting inside the docker container, but currently I don't even know what or why TYPO3 stores still data inside that directory.

[mediaessenz]

Can you explain, how do you did this exclusion?

[Wingmk]

@mediaessenz I changed the volumes inside the docker-compose.yaml from

volumes:
  - ./app/:/app/:delegated

to

volumes:
  - ./app/config:/app/config
  - ./app/vendor:/app/vendor
  - ./app/web:/app/web

This way the app folder itself and therefore the /var/ folder does sync itself with the docker system.
This does also means that the root of the project is not complete. e.g. /var/, composer.json, composer.lock

[mediaessenz]

Thank you for the explaination.

To bad that there is no easier way to exclude some files or subfolder, like it is possible in eg. .gitignore files

[htuscher]

You can do this:

version: '3.7'
services:
  app:
    volumes:
      - ./app:/app:cached
      - app-cache:/app/var:nocopy
volumes:
  app-cache:

[Wingmk]

@hhoechtl Does this setup work for you?
It looks like the /app/var directory inside the container is not writable by the TYPO3 application.
Having the folder empty on the clientside and filled by the application inside the docker container is the desired outcome.

[htuscher]

Yes that works, but you have to do chown 1000:1000 /app/var inside of the container once of course.

[Wingmk]

@hhoechtl can you put this inside a Dockerfile so this is executed on container creation?
A manual chown would not be a great solution.

[htuscher]

Of course you could. I'm personally not supporting TYPO3 anymore. Am Do., 26. März 2020 um 08:46 Uhr schrieb TimoWinkler < [email protected]>:

…

@hhoechtl <https://github.com/hhoechtl> can you put this inside a Dockerfile so this is executed on container creation? A manual chown would not be a great solution. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#202 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGXLBG6DVS65NAOJT7W4UTRJMB4DANCNFSM4JNMZBAQ> .

-- *Onedrop GmbH & Co. KG* Dr.-Leo-Ritter-Str. 2 93049 Regensburg Tel. +49 (0) 941 / 206 064 - 40 1drop.de [email protected] <[email protected]> http://www.facebook.com/onedropsolutions http://twitter.com/1drop

[Wingmk]

@hhoechtl I did some reading and I don't think this stops synchronising the /var folder.
It instead pushes the volume location for synchronising the var folder to docker.

[htuscher]

Not it won't stop the synchronisation, but as you don't make changes locally in the var folder and it's overlayed inside the container with the nocopy volume, it doesn't matter. The nocopy volume is docker engine internally and therefore fast.

[mediaessenz]

@hhoechtl
I have added this to my Dockerfile.development, right after WORKDIR /app/:

RUN mkdir /app/var && chown 1000:1000 /app/var

Additional I added your suggestion to my docker-compose.yml:

version: '3.7'
services:
  app:
    volumes:
      - ./app:/app:cached
      - app-cache:/app/var:nocopy
volumes:
  app-cache:

After all I rebuild the app container and started all again and get this error when opening the app in the browser:

Fatal error: Uncaught RuntimeException: Could not create directory "/app/var/log/"! in /app/public/typo3/sysext/core/Classes/Utility/GeneralUtility.php on line 2169

Did I forget/missunderstand something?

[htuscher]

Creating the directory upfront doesn't work.
The volume mount will override that and still belongs to root

[mediaessenz]

But how can I change the owner/group later?

[htuscher]

docker exec {container} chown 1000:1000 /app/var

[mediaessenz]

Is there also a way to do this automaticaly after "docker-compose up" with the right container name?

[htuscher]

We have that in our Makefile

up: ## Start Project
	docker-compose up -d --remove-orphans --force-recreate
	docker-compose exec app chown -R 1000:1000 /app/var/

[mediaessenz]

I see, thanks for sharing.

Any idea, why I get this errors in the backend:

Warning: rename(/app/var/transient/preview_1066312516667750181.jpg,/app/public/fileadmin/_processed_/c/7/preview_Alexander_Grein_Avatar_d04235402e.jpg): Operation not permitted in /app/public/typo3/sysext/core/Classes/Resource/Driver/LocalDriver.php on line

When I check the owner/group of this directory inside the app container, all files inside there and the subfolder itself have application:application with 644 (files) 2775 (folder)

It seems the system couldn't delete the files inside /app/var/transient/ after doing some image processing. I can see the processed file inside /app/public/fileadmin/processed/c/7/, but there is also still this old, unnecessary file /app/var/transient/preview_1066312516667750181.jpg

[mediaessenz]

After some research I found a way to fix this problem. Unfortunately it needs a TYPO3 core patch :-(
This code in line 787 of TYPO3\CMS\Core\Resource\Driver\LocalDriver::addFile() is the problem:

                $result = rename($localFilePath, $targetPath);

When I change it to this, it works fine:

                $result = copy($localFilePath, $targetPath);
                unlink($localFilePath);

I have no idea why this is necessary, but it seams to be a bug in TYPO3 (or PHP?).
I just checked some /app/var/transient/ dirs on my live systems and they are all packet with this temporary files!

[mediaessenz]

One more, hopefully last question to this issue:
I added the docker-compose exec app chown -R 1000:1000 /app/var/ to my Makefile, which works fine for my development system.
If I want to do the same on my production system, where I use the same docker setup (with production settings, of course), what about the restart: always entries inside my docker-compose.yml file. Are they startup the system after a reboot with the fixed rights of the /app/var/ folder?

[htuscher]

No a restart does not execute a command that it's not aware of.

But the rights remain as long as don't recreate the named volume that is mounted.

[mediaessenz]

Cool, thanks for helping

[mediaessenz]

Is there any reason, why this volume "trick" doesn't work on a production system?
I added exact the same things to the docker-compose.yml of a staging system (ubuntu) with the exact same docker/docker-compose versions. If I rebuild this docker system, the /app/var dir is still synced by docker with the local filesystem.
Any idea what could be the reason for this?

[htuscher]

No, but why would you care. On Linux that's just irrelevant performance wise.

[mediaessenz]

Okay, good to know :-)

[mediaessenz]

It's me again:
I have the problem that my volume mapping only works for a short moment.

My steps:

1. Stop all docker containers
2. Remove local app/var folder by rm -rf app/var
3. Remove app container with docker-compose rm app
4. Start containers by make up with the upper descipted settings
5. Open shell to running app container by make shell and checked correct /app/var owner -> application/application
6. Open the app (TYPO3 page) in a browser and visit some pages who create cache files inside e.g. /var/app/cache/code
7. Check local app/var, which normaly should not exists -> folder exists and is full of generated files
8. Go into the app container by "make shell" and check owner of /app/var -> 501:dialout

What's going wrong here?

[mediaessenz]

Okay, I think I have found my problem.
After removing this setting from my Development-Context-Configuration it works like expected:
$GLOBALS['TYPO3_CONF_VARS']['BE']['createGroup'] = 'dialout';