Volume doesn't sync when you delete a folder in Windows #5550

coolfarmer · 2020-01-22T18:42:34Z

I have tried with the latest version of my channel (Stable)
I have uploaded Diagnostics
Diagnostics ID: F038DED9-684B-479E-98A8-29477AD4CF30/20200122183814

Expected behavior

Sync should work, if I delete a folder in Windows, ls -lsa should show me the correct result.

Actual behavior

Actually I have the problem when phpfpm and nginx exist in my docker-compose. If I only have the service who run my symfony the sync will work.

Information

Is it reproducible? Yes, every time
Is the problem new? Yes, only with 2.2.0.0
Did the problem appear with an update? Yes, 2.2.0.0
A reproducible case if this is a bug, Dockerfiles with reproduction inside is best.
Windows Version: Windows 10 Version 1903 Build 18362.592
Docker Desktop Version: 2.2.0.0
Are you running inside a virtualized Windows e.g. on a cloud server or on a mac VM: No

Steps to reproduce the behavior

Create a simple docker-compose file like this:

version: '2'

services:
  bitnami3:
    image: 'bitnami/symfony:1'
    volumes:
      - '.:/app'
    environment:
      - TZ=America/New_York
      - SYMFONY_PROJECT_NAME=bitnami3
    depends_on:
      - nginx
    networks:
      - default
  nginx:
    image: 'bitnami/nginx:1.16'
    ports:
      - '8784:8080'
    networks:
      - default
    depends_on:
      - phpfpm
    volumes:
      - '.:/app'
      - './bitnami3.conf:/opt/bitnami/nginx/conf/server_blocks/bitnami3.conf'
  phpfpm:
    image: 'bitnami/php-fpm:latest'
    networks:
      - default
    volumes:
      - '.:/app'

networks:
  default:
    driver: bridge

Create at the same level an Nginx conf file "bitnami3.conf"

server {
    server_name localhost;
    root /app/bitnami3/public;
    listen 0.0.0.0:8080;

    location / {
        try_files $uri /index.php$is_args$args;
    }

    location ~ ^/index\.php(/|$) {
		fastcgi_pass phpfpm:9000;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $document_root;
        internal;
    }

    location ~ \.php$ {
        return 404;
    }
}

Run docker-compose up -d
Normally you should access the demo page of symfony at localhost:8784
In your project directory, go to /var/cache/ and delete the folder "dev"
Refresh the web browser to recreate the cache folder
Delete again the folder dev in /var/cache
Open the container with docker exec -ti {ID} /bin/bash
Go to /app/bitnami2/var/cache then run ls -lsa
The folder "dev" is not there, run now "mkdir dev"
- mkdir: cannot create directory 'dev': File exists
This result that symfony can't recreate the cache folder and if your refresh your browser this time you will have:
Warning: file_put_contents(/app/bitnami2/var/cache/dev/App_KernelDevDebugContainerDeprecations.log): failed to open stream: No such file or directory

I don't know why that happen only with php-fpm and nginx. If I remove them and ONLY have a service with symfony, I can delete the cache folder as many time I want and every refresh Symfony will success to recreate the cache folder.

My english is not the best i'm sorry for that.
If more information is needed I'll keep an eye on this issue.

The text was updated successfully, but these errors were encountered:

djs55 · 2020-01-23T12:38:43Z

@coolfarmer thanks for the report (and your english is fine!)

I tried it on my Mac and on my Windows host (since the file sharing is different). I can't make one of the steps work:

Step 2: (I removed the -d so I could see the logs)

bitnami2_1  | Added MariaDB container credentials to .env.local
bitnami2_1  | Symfony app created
bitnami2_1  | symfony successfully initialized
bitnami2_1  | nami    INFO  Initializing php
bitnami2_1  | nami    INFO  php successfully initialized
bitnami2_1  | INFO  ==> Starting symfony...

Step 3:

$ curl http://localhost:8784
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.16.1</center>
</body>
</html>

and in the logs:

nginx_1     | 2020/01/23 12:35:12 [error] 31#0: *4 directory index of "/opt/bitnami/nginx/html/" is forbidden, client: 172.19.0.1, server: , request: "GET / HTTP/1.1", host: "localhost:8784"
nginx_1     | 172.19.0.1 - - [23/Jan/2020:12:35:12 +0000] "GET / HTTP/1.1" 403 153 "-" "curl/7.54.0"

I think this error is preventing the app from rebuilding the cache?

Unfortunately I'm not very familiar with Symfony so I don't know how to fix this. Possibly an nginx config file or an ACL rule or file permission somewhere?

corpsepk · 2020-01-23T12:47:13Z

@coolfarmer Look at this, I think you have similar problem #5530 (comment)

coolfarmer · 2020-01-23T18:01:52Z

@djs55 , omg I'm so sorry, I cut too much things to simplify my demo code that I have forgot the nginx conf file. I edited my post and steps.

@corpsepk , yes I think it's the samething. Sometime I delete the cache folder on the host, and when I do a "ls -lsa" in my container I can see the folder (who should no longer be there). This happen only since 2.2.0.0. I just rollback on 2.1.0.5 and the problem is gone.

mat007 · 2020-01-28T16:07:57Z

@coolfarmer I’ve been trying to reproduce on 2.2.0.0 but wasn’t really successful.
What I did after creating the compose and conf files was run this script in a loop

#!/bin/sh
set -ex
curl -s http://localhost:8784/ > /dev/null
rm -r bitnami3/var/cache/dev
docker exec -it pinata_bitnami3_1 ls /app/bitnami3/var/cache

It fails quite quickly after a few iterations but not exactly how I was expecting.

$ repeat ./bla.sh                                                          
trying...                                                                  
+ curl -s http://localhost:8784/                                           
+ rm -r bitnami3/var/cache/dev                                             
+ docker exec -it pinata_bitnami3_1 ls /app/bitnami3/var/cache     
trying...                                                                  
+ curl -s http://localhost:8784/                                           
+ rm -r bitnami3/var/cache/dev                                             
+ docker exec -it pinata_bitnami3_1 ls /app/bitnami3/var/cache
trying...                                                                  
+ curl -s http://localhost:8784/                                           
+ rm -r bitnami3/var/cache/dev                                             
+ docker exec -it pinata_bitnami3_1 ls /app/bitnami3/var/cache
trying...                                                                  
+ curl -s http://localhost:8784/                                           
+ rm -r bitnami3/var/cache/dev                                             
rm: cannot remove 'bitnami3/var/cache/dev': No such file or directory

And when I refresh the page in my browser:

The cache is however gone both on the host and in the container

$ ls -la bitnami3/var/cache
total 0
drwxr-xr-x 1 mathi 197610 0 Jan 28 16:59 ./
drwxr-xr-x 1 mathi 197610 0 Jan 28 16:10 ../

$ docker exec -it pinata_bitnami3_1 ls -la /app/bitnami3/var/cache
total 0
drwxrwxrwx 1 root root 0 Jan 28 10:59 .
drwxrwxrwx 1 root root 0 Jan 28 10:10 ..

I then need to compose down and up to restart everything.

I’m not overly familiar with Symfony, is this expected or part of the issue?

coolfarmer · 2020-01-28T18:04:49Z

With symfony it's pretty simple, when you delete the cache folder "dev", at the next refresh Symfony will rebuild all the cache and will create a new "dev" folder under /cache/. This is the only thing you need to know about symfony for this issue.

The error you have on your screenshot is part of the problem yeah, if you have that error, you have successfully reproduced the issue.

Warning: file_put_contents(/app/bitnami2/var/cache/dev/App_KernelDevDebugContainerDeprecations.log): failed to open stream: No such file or directory

This error mean that symfony can't access/delete/create the cache folder (aka permission error).
Try something, run your script until you get the same error when you refresh your browser, then look into your container to see if a dev folder exist under /cache/.

If you don't have the dev folder, try to create an empty one with "mkdir dev", you will receive "mkdir: cannot create directory 'dev': File exists".

But the directory doesn't exist when you run ls -lsa, that is so weird.
I'm not the best in english and I don't know how to better explain the problem.

One thing is sure, if you got that error when your refresh your browser, you perfectly reproduce the problem. Symfony MUST ALWAYS re-build the cache folder when you delete it, and this work correctly before 2.2.0.0.

mat007 · 2020-01-28T20:46:28Z

Ah I see!
I can confirm I was able to reproduce the issue, we’ll take a deeper look, thanks for the nice repro instructions.

mat007 · 2020-01-28T20:57:54Z

In the meanwhile you can deactivate the caching by creating an empty file %APPDATA%\Docker\disable-filesystem-caching as it appears to fix the issue.
This might degrade performance a bit, but that’ll make it at least usable until we fix it.

coolfarmer · 2020-01-28T20:59:31Z

I have already downgraded to 2.1.0.5 but thanks for the workaround. I think I'll wait a stable 2.2.0.0.

mat007 · 2020-01-28T21:00:04Z

No worries, thanks for your help!

stephen-turner · 2020-02-01T14:28:34Z

Another instance at #5572 (comment)

stephen-turner · 2020-02-06T17:37:21Z

We have a new build that we hope fixes this error. You can get it from https://download-stage.docker.com/win/stable/42644/Docker%20Desktop%20Installer.exe. We had to switch one of the software libraries we use for an alternative, so we would appreciate any testing that people are able to do on it. Thank you for your help.

coolfarmer · 2020-02-06T20:28:23Z

@stephen-turner , I just tested your new build (2.2.0.2) and I have good and bad news! Problem is 50% fixed. If I delete the folder inside Windows Explorer, now it work! YAYYY!

BUT, with the same folder, if I delete it with my IDE (Jetbrain PhpStorm 2019.3.1), the same problem occure. I tested it with my personal project AND with my demo project (Bitnami3) that you can find in my first post : the result is the same, I can reproduce the same bug but now with my IDE only.

Again my english in not the best so please don't hesitate if you want more details.

djs55 · 2020-02-10T20:42:10Z

@coolfarmer thanks for all your help so far. We have another build (version 2.2.0.3 (42716)), which I hope will help: https://download-stage.docker.com/win/stable/42716/Docker%20Desktop%20Installer.exe

I tested it by starting your compose example with docker-compose up and then running a loop of:

#!/bin/sh
set -ex

while true; do
  # container builds a cache
  if curl -s http://localhost:8784/ | grep "Internal Server Error"; then
    # cache was in the middle of being deleted
    echo 500 Internal Server error
    exit 1
  fi
  # host removes the cache
  rm -rf bitnami3/var/cache/dev
  # cache removals are being propagated to the container now
  winpty docker exec -it folder-sync_bitnami3_1 ls //app/bitnami3/var/cache
done

I then observe a sequence of HTTP 404s in the logs, ending in a 500 when the deletes from rm -rf on the host propagate to the container while PHP is rebuilding its cache, causing an expected error (writing to the same files from both the host and container will cause this kind of transient glitch).

The important difference is that if I

refresh the page in the browser, or
manually execute mkdir dev

it now works as I would expect. Host and container quickly get back in sync and the app can repopulate the cache.

Previously (as you observed) you could get into a stuck out-of-sync state where the directory could not be created because it already existed from the container's point of view and yet it could not be removed because it didn't exist from the host's point of view. I've fixed a number of filesystem event propagation bugs so hopefully both host and container will remain in sync (modulo the time taken to propagate changes)

coolfarmer · 2020-02-10T21:35:55Z

You guys are machines! Now it work even with my IDE. :)

I tried to test the limits of the system and the error come back only if I am too fast like refreshing the browser ~500ms after the folder was deleted. But this time, if I get the error, I just have to refresh again and the cache is recreated without problem!

So yeah, I think the bug is gone, until next issue (I hope not haha), I don't find any other ways to break that thing.

Nice job and thanks again! :)

stephen-turner · 2020-02-11T13:35:05Z

Thanks for confirming, @coolfarmer. We have just released 2.2.0.3 publicly, so closing this ticket. In case any Edge users are watching this ticket, it will be on the Edge channel tomorrow too.

coolfarmer · 2020-02-12T07:29:45Z

@stephen-turner , hey Stephen, just a little message to let you know that the problem was back again, to fix it I had to reboot my computer (Docker Restart didn't help). I hope it will not do that every day.

stephen-turner · 2020-02-12T10:43:26Z

@coolfarmer That's weird. If it happens again, could you send us new diagnostics please?

docker-robott · 2020-07-13T03:03:59Z

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

mat007 added status/triage version/2.2.0.0 labels Jan 23, 2020

mat007 added status/acknowledged and removed status/triage labels Jan 28, 2020

stephen-turner mentioned this issue Feb 1, 2020

Under 2.2.0.0, windows file paths (~260 chars) fail to work properly via container mount #5572

Closed

2 tasks

This was referenced Feb 5, 2020

"transport endpoint is not connected" error after a write to a mounted volume crashes docker container. Docker Destop 2.2.0.0 #5584

Closed

WorkingDir of subdir of a volume mount is not found by Python's os.getcwd() #5698

Closed

stephen-turner closed this as completed Feb 11, 2020

docker-robott added the lifecycle/locked label Jul 13, 2020

docker locked and limited conversation to collaborators Jul 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Volume doesn't sync when you delete a folder in Windows #5550

Volume doesn't sync when you delete a folder in Windows #5550

coolfarmer commented Jan 22, 2020 •

edited

Loading

djs55 commented Jan 23, 2020

corpsepk commented Jan 23, 2020

coolfarmer commented Jan 23, 2020

mat007 commented Jan 28, 2020 •

edited

Loading

coolfarmer commented Jan 28, 2020

mat007 commented Jan 28, 2020

mat007 commented Jan 28, 2020

coolfarmer commented Jan 28, 2020 •

edited

Loading

mat007 commented Jan 28, 2020

stephen-turner commented Feb 1, 2020 •

edited

Loading

stephen-turner commented Feb 6, 2020

coolfarmer commented Feb 6, 2020 •

edited

Loading

djs55 commented Feb 10, 2020

coolfarmer commented Feb 10, 2020 •

edited

Loading

stephen-turner commented Feb 11, 2020

coolfarmer commented Feb 12, 2020

stephen-turner commented Feb 12, 2020

docker-robott commented Jul 13, 2020

Volume doesn't sync when you delete a folder in Windows #5550

Volume doesn't sync when you delete a folder in Windows #5550

Comments

coolfarmer commented Jan 22, 2020 • edited Loading

Expected behavior

Actual behavior

Information

Steps to reproduce the behavior

djs55 commented Jan 23, 2020

corpsepk commented Jan 23, 2020

coolfarmer commented Jan 23, 2020

mat007 commented Jan 28, 2020 • edited Loading

coolfarmer commented Jan 28, 2020

mat007 commented Jan 28, 2020

mat007 commented Jan 28, 2020

coolfarmer commented Jan 28, 2020 • edited Loading

mat007 commented Jan 28, 2020

stephen-turner commented Feb 1, 2020 • edited Loading

stephen-turner commented Feb 6, 2020

coolfarmer commented Feb 6, 2020 • edited Loading

djs55 commented Feb 10, 2020

coolfarmer commented Feb 10, 2020 • edited Loading

stephen-turner commented Feb 11, 2020

coolfarmer commented Feb 12, 2020

stephen-turner commented Feb 12, 2020

docker-robott commented Jul 13, 2020

coolfarmer commented Jan 22, 2020 •

edited

Loading

mat007 commented Jan 28, 2020 •

edited

Loading

coolfarmer commented Jan 28, 2020 •

edited

Loading

stephen-turner commented Feb 1, 2020 •

edited

Loading

coolfarmer commented Feb 6, 2020 •

edited

Loading

coolfarmer commented Feb 10, 2020 •

edited

Loading