No E-Tag received from server, check Proxy/Gateway [$150 awarded]

[anagno]

Hi all,

Using the desktop client i am getting "No E-Tag received from server, check Proxy/Gateway" and a large portion of files is re-downloaded. (similar with #1269)

Server configuration

Operating system: Ubuntu Server 12.04 LTS

Web server: Apache 2.2.22

Database: MySQL 5

ownCloud version: 6.0 (stable)

Client configuration

OS: ubuntu 12.04

Client: 1.5

Thanks in advance

--- The **[$150 bounty](https://www.bountysource.com/issues/1383937-no-e-tag-received-from-server-check-proxy-gateway?utm_campaign=plugin&utm_content=tracker%2F216457&utm_medium=issues&utm_source=github)** on this issue has been claimed at [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F216457&utm_medium=issues&utm_source=github).

[lachlan-00]

Are there archives in that list? (Zip, tar.gz, etc?) I am having a similar issue.

All compressed files on windows/Linux clients will upload on the source PC but will not sync to the other computers linked to the account.

The same etag error pops up saying the file was downloaded but no data is transferred. No files are created.

If I delete the file and sync again on a different PC I get the same result. Once the file has uploaded the same error repeats on the source PC.

Debian testing
Apache
Linux/windows client @1.5

[sokol99]

I am also having this issue from two of my four clients using the same account. The client keeps trying to sync over and over and only syncs folders but no files.

ownCloud 6.0 on Mint 15 box. Clients are the latest builds also from Mint 15 boxes. ..Can't make the transition from Dropbox without this working. I do have two other clients able to sync with files to the same account. One is Mint 16 and the other is OS X. .. I personally don't see anything obvious in the owncloud MySQL data but I'm a novice in that dept...

[anagno]

Unfortunately, in my case, it syncs the entire content of the folders and there aren`t archives in the lists to blame the archives.

[lachlan-00]

I might open a separate issue, mine is only occuring with compressed files.

[ottok]

I had the same error: a JPG image was downloaded again and again by my OwnCloud client installed (version: 5.0.4debian-0ubuntu1~ubuntu12.04). I removed the JPG image server side (via browser) and after that the client continued and synced all other files normally.

[ottok]

Sorry for the false report, actually the error still persists. Now there is a bunch of .txt files OwnCloud client redownloads over and over again..

I hope there was some bounty system so I could through 50€ in a pool to fund fixing this. It is a pitty the syncing client is unusable at the moment.

[sokol99]

I rolled back to ownCloud 5.0 from 6.0 and seem to be having much better luck. At least basic syncing seems to work. :/ Using Ubuntu 12.04 LTS.

[danimo]

So it might be a server issue. Summoning @DeepDiver1975 @schiesbn @icewind1991.

[justlilpete]

I think I may have a slightly different issue, but I'm hoping someone can tell me if it is or not before I raise a separate report.

My ownCloud server is 6.0.0a and my client(s) are 1.5.0.

My setup is to use Apache to host the ownCloud instance, with nginx acting as a reverse proxy to allow access through one port to several servers. Neither seems to be using pagespeed from what I can tell.

It looks as if all of my .htm and .html files have the same issue as above where an ETag is not provided to the client during sync, so it never downloads any of these files. All other files seem unaffected.

If I access the Apache instance directly, then the files sync correctly, so I'm guessing nginx is stripping out the etag header (despite "proxy_pass_header ETag;" being set) for htm/html files, which I am (purely guessing) is because it's caching them...? I can't seem to find a way to prevent this behaviour, although I'm far from an expert.

This never seemed to be an issue before, I've first noticed it after setting up the client on a new machine with a blank folder structure so all files were downloaded from the server. My existing machine demonstrates an identical problem with any new files added to the server, but existing ones it's happy with.

[dragotin]

@ottok no need to open your wallet, how about providing us with client log files and even better a wireshark report of what's being transmitted over the wire? Thx.

[ottok]

Client side log looks like this:

01-04 21:34:07:716 !!! Mirall::CheckQuotaJob created for  QUrl( "http://xxx/owncloud" )  querying "/" 
01-04 21:34:13:224 "DELETE FROM downloadinfo WHERE path=?" "04-thievery_corporation_-_interlude-tronik.mp3" 
01-04 21:34:13:224 void Mirall::CSyncThread::transferCompleted(const Mirall::SyncFileItem&) "04-thievery_corporation_-_interlude-tronik.mp3" 3 "No E-Tag received from server, check Proxy/Gateway" 
01-04 21:34:13:225 "INSERT OR REPLACE INTO downloadinfo (path, tmpfile, etag, errorcount) VALUES ( ? , ?, ? , ? )" "Mediakomppania » Kassa.png" ".Mediakomppania » Kassa.png.~46b52dfb" "" 0 
01-04 21:34:13:225 Transaction Start  "download file start" 
01-04 21:34:13:445 static void Mirall::PropagateDownloadFile::install_content_reader(ne_request*, void*, const ne_status*) No E-Tag reply by server, considering it invalid  
01-04 21:34:22:170 "DELETE FROM downloadinfo WHERE path=?" "Mediakomppania » Kassa.png" 
01-04 21:34:22:171 void Mirall::CSyncThread::transferCompleted(const Mirall::SyncFileItem&) "Mediakomppania » Kassa.png" 3 "No E-Tag received from server, check Proxy/Gateway" 
01-04 21:34:22:171 "INSERT OR REPLACE INTO downloadinfo (path, tmpfile, etag, errorcount) VALUES ( ? , ?, ? , ? )" "clientsync/Kyocera tulostin asennus.pdf" "clientsync/.Kyocera tulostin asennus.pdf.~1e23ef54" "" 0 
01-04 21:34:22:171 Transaction Start  "download file start" 
01-04 21:34:22:531 static void Mirall::PropagateDownloadFile::install_content_reader(ne_request*, void*, const ne_status*) No E-Tag reply by server, considering it invalid  
01-04 21:34:23:946 "DELETE FROM downloadinfo WHERE path=?" "clientsync/Kyocera tulostin asennus.pdf" 
01-04 21:34:23:946 void Mirall::CSyncThread::transferCompleted(const Mirall::SyncFileItem&) "clientsync/Kyocera tulostin asennus.pdf" 3 "No E-Tag received from server, check Proxy/Gateway" 
01-04 21:34:23:950 "INSERT OR REPLACE INTO downloadinfo (path, tmpfile, etag, errorcount) VALUES ( ? , ?, ? , ? )" "clientsync/packages-that-depend-on-mysql-client.txt" "clientsync/.packages-that-depend-on-mysql-client.txt.~7f6af5df" "" 0 
01-04 21:34:23:950 Transaction Start  "download file start" 
01-04 21:34:24:230 static void Mirall::PropagateDownloadFile::install_content_reader(ne_request*, void*, const ne_status*) No E-Tag reply by server, considering it invalid  
01-04 21:34:24:232 "DELETE FROM downloadinfo WHERE path=?" "clientsync/packages-that-depend-on-mysql-client.txt" 
01-04 21:34:24:232 void Mirall::CSyncThread::transferCompleted(const Mirall::SyncFileItem&) "clientsync/packages-that-depend-on-mysql-client.txt" 3 "No E-Tag received from server, check Proxy/Gateway" 

Server log data/owncloud.log does not have anything with matching timestamps.

I updated from server 5.0.13 to 6.0.0a and the problem still persists.

[ottok]

Captured HTTP request and response with Wireshark:

HTTP    339 GET /owncloud/remote.php/webdav/muistio.txt HTTP/1.1 
    User-Agent: Mozilla/5.0 (Linux) csyncoC/0.91.4 neon/0.29.6\r\n
    Connection: TE\r\n
    TE: trailers\r\n
    Host: xxx\r\n
    Proxy-Connection: Keep-Alive\r\n
    Accept-Encoding: gzip\r\n
    Cookie: 512cb4a347514=jou7h53nt6v5pdmfvk92agsmh6; \r\n

    HTTP/1.1 200 OK\r\n
    Date: Sat, 04 Jan 2014 20:08:36 GMT\r\n
    Server: Apache/2.2.15 (CentOS)\r\n
    X-Powered-By: PHP/5.3.10\r\n
    Expires: Thu, 19 Nov 1981 08:52:00 GMT\r\n
    Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0\r\n
    Pragma: no-cache\r\n
    Last-Modified: Tue, 26 Feb 2013 13:31:02 GMT\r\n
    Content-Length: 42\r\n
    Connection: close\r\n
    Content-Type: text/plain; charset=utf-8\r\n
    \r\n
(and then the file contents)

So really there is no Etag present in the headers, thus the files keep downloading over and over again..

[Stebalien]

NGINX strips off ETags when gzipping. In my case, turning gzip off fixed it.

[justlilpete]

@Stebalien Thanks, I've changed the setting and it's stopped the problem. Apologies for potentially muddying the issue, I figured that as it had only recently started happening it might be related.

[willyaranda]

I had the same problem and @Stebalien provided the correct way to workaround it.

Thanks!

[ottok]

@willyaranda Note that the this issue was originally raised for the setup with Apache, the Nginx proxy issue was a side track.

So for Apache users the issue remains unresolved.

[lionants02]

@Stebalien Thank for ngnix config
gzip off;

[ccverg]

@ottok in case you're still interested, you can start a bounty here: https://www.bountysource.com/issues/1383937. Bountysource has been used to fund another bounty on Owncloud in the past: https://www.bountysource.com/issues/831597-gallery-app-can-t-share-public-via-link.

[ottok]

I upgraded on the server side to 6.0.1 and on the client side, for some strange reason, apt-get notified that there is a new version available of "http://download.opensuse.org/repositories/isv:ownCloud:desktop/xUbuntu_12.04/ owncloud-client 1.5.0" (even though it was at 1.5.0 already previously) so I also upgraded it (By the way, why are the OBS builds not signed? Apt-get shows ugly warning of untrusted download.)

The problem still presist. Funny. I'd guess my setup is fairly common, with CentOS+Apache+OwnCloud server and Ubuntu 12.04+OwnCloud client. Most of the files are redownloaded very 5 minutes or so. Terrible.

@ccvergara I now spent $165 on Bountysource (https://www.bountysource.com/issues/1383937), paid extra to get it into the Bountysource newsletter and I also promoted it myself via social media.

If anybody else suffers from this bug, consider contributing to the bounty!

[dragotin]

@ottok We fixed something in the client package, that is why you got an update.

I also checked again which code path results in the error message No E-Tag reply by server, considering it invalid This happens directly after the GET request is coming in, and the GET should have an ETag header which it has not in your case.
I still think there is something filtering your requests and removing custom headers. We have to find out what that is. Do you have any special authentication solutions in between for example? I know that for example the old Novell iChain also does header filtering. Any scan software? Firefox module maybe?

[ottok]

@dragotin I am pretty sure there is no filter. The server is directly on public internet and so is client too. Anyway, even if there would be some connection error, the software should fail gracefully and not go nuts and re-download same files every 5 minutes. I've never had this kind of situation while using Ubuntu One or Dropbox.

If you send me an email to otto ät seravo.fi I can give you credentials to the server in question so you can check it out yourself. The server is for test use only, there is no private data.

[sulazix]

I got the same issue (only with shared folders) OC 6.0.0a (and 6.0.1) with an apache server pagespeed_module disable
I solved this by downgrading the sync-client, we're about 30 users who were using sync client 1.5.0 with shared folders, I asked everybody to downgrad and now everything is okay.
For Mac : http://download.owncloud.com/download/ownCloud-1.4.2.dmg
For Windows : http://download.owncloud.com/download/ownCloud-1.4.2-setup.exe

[blind-coder]

I'm having the same issue with owncloud 5.0.13 and sync client 1.5.0.
Funny thing is, lots of .zip files and about 50 .epub (out of several hundred) don't have an E-Tag, while many other, similiarly named and sized files do.
This sucked my data volume on my mobile phone dry yesterday...

[blind-coder]

I "fixed" this with this SQL:
UPDATE oc_filecache SET etag = MD5(fileid) WHERE etag = '';

The question is, obviously, why these files didn't have an etag in the first place.

[ottok]

@blind-coder Thanks! Indeed the files that are re-downloaded over and over have an empty etag in the table oc_filecache, and only them.

SQLite does not have function md5() built in and adding it manually as an external function is not trivial, so I will not do it this time (argh, OC should really not have SQLite as default).

sqlite> UPDATE oc_filecache SET etag = MD5(fileid) WHERE etag = ''; Error: no such function: MD5

Anyway this is the conclusion:

• the PHP code somewhere has a bug so that it has failed to input the etag,
• the database schema has a bug that it allows empty etags and
• the download code isn't resilient, it should not send files again to client just because the etag field is empty.

This bug can be marked done when somebody adds code that does at least one of the things above.

As a work around I deleted the files from OC, and re-uploaded them. Now they have etags in the database.

[blind-coder]

You can actually use any value in the etag. I just used MD5 out of habit. You can even set them all to "1".
As long as it's not null or a zero-length string, it should be fine.
I checked in owncloud source code and it just calls PHPs uniqid() function to "generate" an etag.

[guruz]

For the record:
1.7 and very-new-unreleased-server-versions will introduce a header OC-ETag with our expected Etag value and hopefully the webservers not messing with it.

12e2bac
owncloud/core#10725

@DeepDiver1975 Can this be in the 7.x server? The bug here is one of our most-commented ones.

[ruddyconsult]

I have installed the sync client on a fresh Ubuntu 14.04 machine, and it synced successfully. (However encryption is not working, and I have posted that as a separate issue).

[benemoto]

Iam a new owncloud user. I use Ubuntu 12.04, client version 1.7.0 and server version 7.0.3. (Sqlite)
I get the "No E tag" error message all the time and have no idea how to fix it. None of my files are synchronised. I tried to disable gzip via .htaccess but without success. I do not have the possibility to access to the server via console etc. I do not have much knowledge about coding or server administration. Could anyone help me?

[guruz]

Could you check if on your server the file filesplugin.php contains this line: https://github.com/owncloud/core/pull/10725/files#diff-048b55ec399ee022be98577796d4f0b0R112

[benemoto]

Thank you for responding.

No. I can not find this line. Unfortunately I can not upload my filesplugin.php due to restrictions of github, however here are the lines of my filesplugin.php that look similar to what you refered to:

    if ($node instanceof OC_Connector_Sabre_Node) {
        $fileId = $node->getFileId();
        if (!is_null($fileId)) {
            $this->server->httpResponse->setHeader('OC-FileId', $fileId);

[guruz]

@benemoto Thanks. I suspect you should ask your server provider about this, I don't know what is happening but something makes owncloud not workable there.
Do you see anything in the owncloud.log in the data/ directory on the server?

[benemoto]

@guruz No. I can not see anything particular that might explain the error.

[blaskovic]

Same problem. I cannot sync .zip files successfuly.

No suspicious logs in data/owncloud.log. Running OwnCloud git version ownCloud 7.0.3 RC3 (git) on CentOS6 with Apache 2.2.

[guruz]

@blaskovic The oC client does not treat .zip files any different from any other file.
So I suggest you look into your server setup.
I also heard that Google Pagespeed (or similar) httpd modules sometimes mess with the files.

[blaskovic]

I have more-less default httpd for centos6.

[sulazix]

@blaskovic Did you check for :
Header unset ETag
FileETag None
in any .htaccess ? Those line are sometimes used to optimized navigation speed... and they cause an "No-etag received from server ..." issue
Check also your /etc/httpd/conf.d/* With some services like Open-Xchange, they suggest to disable Etag (http://oxpedia.org/wiki/index.php?title=Open-Xchange_Installation_Guide_for_CentOS_6_622#Configure_services)

[blaskovic]

Nothing like that.

$ pwd
/path/to/web
$ find . -name '.htaccess' | xargs grep -i tag
$ 

$ pwd
/etc/httpd/conf.d
$ grep -nriHIF tag *
$

[ogoffart]

Owncloud 8 has a OC-Etag header that should solve this problem.

In any cases, i'm closing this old issue because it has too many comments and this is not practical to track.

If you still have problem with a recent owncloud, please open a new issue (following the issue template)

[blaskovic]

So.. This will NOT be fixed for owncloud 7?

[danimo]

@blaskovic correct.

[ottok]

@ogoffart If you consider this fixed, you can collect 150 USD from the bounty at https://www.bountysource.com/issues/1383937-no-e-tag-received-from-server-check-proxy-gateway

[ogoffart]

Someone just told me that the issue is actually not fixed in owncloud 8 because the fix of the server was reverted -> owncloud/core#9005
Maybe this is going to be in owncloud 8.1

Anyway, the client part is done as we read the oc-etag header.
There is not much more we can do. So just make sure you don't have a proxy that remove the headers Or just track the server issue.

[moscicki]

@ogoffart: is the intention to replace standard Etag by OC-Etag in the future completely?

[Germano0]

I confirm this problem on Owncloud 8.0.3 / mirall-1.7.1-2.fc22.x86_64

[Germano0]

I disabled gzip but I am still experiencing the problem
nginx-1.6.3-2.fc21.x86_64
owncloud-8.0.3-2.fc21.noarch

# cat /etc/nginx/nginx.conf
# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log;
#error_log  /var/log/nginx/error.log  notice;
#error_log  /var/log/nginx/error.log  info;

pid        /run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on; questa riga è l'originale. Ho messo off per via del bug di Owncloud con gli e-tag
    gzip off;
    index   index.html index.htm;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80 default_server;
        server_name  localhost;
        root         /usr/share/nginx/html;

        #charset koi8-r;

        #access_log  /var/log/nginx/host.access.log  main;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        # redirect server error pages to the static page /40x.html
        #
        error_page  404              /404.html;
        location = /40x.html {
        }

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
        }
    }
}

[ghost]

To avoid double-posts: This post above was also posted in #2980 (comment)