Rack attack sees wrong IP address when configured with addtional proxy

[jeinwag]

We're running gitlab behind an addtional nginx proxy (used for Let's Encrypt support/SSL termination).
Configuration looks like this:

server {
       listen 0.0.0.0:443 ssl;
       server_name my.githost.com;
       ssl_certificate /etc/dehydrated/certs/my.githost.com/fullchain.pem;
       ssl_certificate_key /etc/dehydrated/certs/my.githost.com/privkey.pem;
       server_tokens off;
       ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
       ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
       client_max_body_size 200m;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header Host $http_host;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Ssl on;
       proxy_set_header X-Forwarded-Proto $scheme;
       location /.well-known/acme-challenge {
        alias /var/www/dehydrated;
       }

       location / {
        proxy_pass http://gitlab;
       }
}

Even though the X-Forwarded-For header is set, it seems rack attack only sees the same IP address for all requests (the address of the machine running gitlab and nginx). Blacklisting that IP address will of course make gitlab unusable for everyone.

I guess this is due to the second nginx proxy running inside the gitlab container.

Any ideas how to work around this issue?

[lukas-hetzenecker]

I'm using Traefik for SSL termination and can confirm this behaviour. Unfortunately I haven't found a workaround yet.

[stmh]

I see this too with haproxy in front of gitlab.

[jeinwag]

I guess using the nginx real IP module with the inner nginx inside the container could help:
http://nginx.org/en/docs/http/ngx_http_realip_module.html

[stepan-romankov]

We have the same problem!

[jeinwag]

I just noticed that this was fixed in #1137, you can solve the issue by configuring NGINX_REAL_IP_RECURSIVE and NGINX_REAL_IP_TRUSTED_ADDRESSES accordingly.

[stepan-romankov]

@jeinwag This solution doesn't work well.
We use https://github.com/jwilder/nginx-proxy and proxy itself has dynamic ip address given by docker.

is it possible that gitlab just take into account X-Real-IP?

[Nuxij]

I have this exact problem. I also use my nginx to do SSL management. Doesn't it make more sense to just use the X-* HTTP headers? It's a pretty standard mechanism.

[jeinwag]

Currently, the X-Real-IP header gets set/overwritten by the proxy running inside the gitlab container. I guess you could remove it from the config there and have it set by your outer proxy.

But I'm not sure what other usecases might break by removing the X-Real-IP from the configuration of the inner proxy.

[stepan-romankov]

@jeinwag could you configure nginx in a way that if NGINX_REAL_IP_RECURSIVE is set to true and NGINX_REAL_IP_TRUSTED_ADDRESSES is not set, then it takes into account X-Real-IP otherwise not.

this should not break existing behaviour because for now NGINX_REAL_IP_TRUSTED_ADDRESSES is mandatory if NGINX_REAL_IP_RECURSIVE is set to true.

Or simple another environment flag NGINX_USE_REAL_IP_HEADER :)

Would appreciate a lot!