Add CrowdSec and/or ModSecurity

[Joly0]

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[Zoey2936]

Thanks for the request, I think it should be possible, but one question: What do they exactly do? I didn't understand it, yeet? What is a use case?

[Joly0]

Hey, i will try my best to explain it:
CrowdSec is similar to Fail2Ban (not sure if you know this), but it basically can scan logs of different applications and if it finds malicious activity, the connection is automatically blocked. So far, fail2ban should be enough, though CrowdSec has a very important feature: Sharing malicious activity with others automatically.
So if i find something, i can share it with the whole network of user that use CrowdSec and make sure, everyone is a bit safer.
This is a simple explanation, a better one by CrowdSec is found here https://docs.crowdsec.net/docs/intro/#:~:text=CrowdSec%20is%20an%20open%2Dsource,infrastructural%2C%20system%2C%20applicative).

Now to ModSecurity: ModSecurity is basically a Module for various Web-Servers (Apache, Nginx, etc) which is used as a WAF (Web-Application-Firewall) to protect against and to help setting up protection against different attacks like Cross-Site-Scripting, SQL Injection, or Session Hijacking.

I guess the use case for both should be clearer now, but in general, both of them can improve the overall security of the system at the web-server (or reverse-proxy in our case) level by scanning logs and network traffic, checking them against pre-defined rules and taking action against malicious activity. Both of these systems have a lot of pre-configured scenarious to check, but they still have to be used by the user.

I tried to make this explanation as simple as possible with my (limited) knowledge, but i hope its still understandable :)

And a quick side-note to CrowdSec: It only detects activity, to apply these a bouncer is needed which there are a larger number of different ones for different systems found here https://docs.crowdsec.net/docs/bouncers/intro

[Zoey2936]

do they need to be configured in a nginx config file? and would it possible to built them into nginx and make them manually configurable?

[Joly0]

For CrowdSec after installation a nginx.conf should be placed here "/etc/nginx/conf.d/crowdsec_nginx.conf" which is so far pre-configured to just run (thats with the bouncer-installation, not with crowdsec on its own, as i said earlier, crowdsec doesnt do much on its own). More information about this can be found here https://doc.crowdsec.net/docs/bouncers/nginx/

For ModSecurity (for this libmodsecurity and the nginx-connector is needed) there are a few directives that need to be configured in the nginx.conf files of the different hosts.

[Joly0]

Btw, there are 2 pr´s for nginx-proxy-manager for this
NginxProxyManager#2677
NginxProxyManager#1867
But it doesnt look like this will be merged, so maybe it could be added here. Wanted to mention that, because maybe there is some useful information found in those pr´s

[Zoey2936]

I will do it (maybe this or next week), but how do I test if they work?

[Joly0]

Hm, thats not so easy. For crowdsec i would guess either the "cscli metrics"-command would work, which shows some metric information or simpler "cscli decisions add --ip your.ip.address --duration 5m --reason "testing"" could work. The last command blocks your ip address for 5 minutes.
For modsecurity i am out of ideas so i simply googled and found this https://serverfault.com/questions/642804/test-whether-mod-security-is-actually-working

[Zoey2936]

I think mod security should be possible very easily, but crowdsec could be hard, since it is not optimized for alpine linux, which this fork is based on

[Joly0]

Are you sure about the crowdsec part? Even the official crowdsec docker is based off of alpine https://github.com/crowdsecurity/crowdsec/blob/master/Dockerfile
I might be wrong, just quickly googled that because i knew i have seen another project using both of them already

[Zoey2936]

Even the official crowdsec docker is based off of alpine https://github.com/crowdsecurity/crowdsec/blob/master/Dockerfile

that is just crowdsec, no nginx...

[Joly0]

Ohh yes, you are totally right. I am sorry, havent looked closer.
If crowdsec isnt possible, or harder, than maybe try to integrate modsecurity and maybe at a later point crowdsec might be possible?

[Zoey2936]

I could try to do the steps of install.sh manually

[LePresidente]

Crowdsec has a requirement for lua scripting language as well so that will need to be installed, npm normally uses openresty instead of vanilla nginx

I did a similar pr for jlesage/docker-nginx-proxy-manager that contains everything needed in one PR

jlesage/docker-nginx-proxy-manager#238

there is also a caddy module
https://caddyserver.com/docs/modules/crowdsec

[Zoey2936]

But it should work with vanilla nginx also, right? since I don't get openresty built to a binary without shared libaries, I switched to plain nginx, because I got it built static

[LePresidente]

I would say it is doable but you will need to add lua and a few modules to nginx

https://docs.crowdsec.net/docs/bouncers/nginx/

[Zoey2936]

crowdsec would come with a massive size increment: https://pkgs.alpinelinux.org/package/edge/testing/x86_64/crowdsec 70MB!

[Zoey2936]

I did a similar pr for jlesage/docker-nginx-proxy-manager that contains everything needed in one PR

where did you add the cscli?

[LePresidente]

No, that is the full crowdsec package, what you add to nginx is the bouncer

https://github.com/crowdsecurity/cs-nginx-bouncer (About 20kb)

this talks to the crowdsec server and blocks based on a ban list in crowdsec

[LePresidente]

i'll have a look at your build process and try get this working, did the rest of them so shouldn't be hard

[Zoey2936]

I did a similar pr for jlesage/docker-nginx-proxy-manager that contains everything needed in one PR

where did you add the cscli?

https://github.com/crowdsecurity/cs-nginx-bouncer/blob/8a8f40f83a3495b973fd5a4b402a48cacd34460b/install.sh#L44 since the install script makes requests to cscli

[LePresidente]

I'll have to port some of my changes from here, and make it docker friendly

https://github.com/crowdsecurity/cs-openresty-bouncer

[Zoey2936]

I got nginx build with lua support, but

1. the binary can't be static (so it requires dynamic libs)
2. I'm thinking about using openresty, since they don't recommend using their lua without openresty
3. BUT openresty is not built from the nginx-quic repo... So that would be needed to be changed by me
4. BUT if nginx-quic uses a newer nginx version than openresty, the build will fail (I did exactly this in the past and I don't want to fork openresty again)

[Zoey2936]

What would you all do?

1. use nginx-quic with lua
2. fork openresty again
3. remove quic, but use lua

[Zoey2936]

@LePresidente what would you say?

[LePresidente]

Hi, if this fork was done for quic then i say leave it as is without crowdsec, when quic does make it officially into nginx/openresty then make the move back to openresty,

[Zoey2936]

with #266 I got lua working, so crowsec should be possible

[Zoey2936]

I've got crowdsec working

[Zoey2936]

[Zoey2936]

I will do a last test and then push it to latest, then I will work on modsecurity
When nginx-quic is inside openresty, I will switch the project to openresty, until then I will use the nginx-quic repo and add all modules manually

[Zoey2936]

@LePresidente @Joly0 I've added some docs to the readme and pushed it to latest, could you please test it?

[Joly0]

Hey, thank you for your work. I am currently in vacation, so quite difficult for me to test that, but maybe i find a way

[Zoey2936]

But one question from my side: Is it normal that with crowdsec enabled, nginx takes long to launch and uses a lot of cpu while launching/reloading (one complete cpu core)? and after the full launch/reload of nginx, nginx uses only a few percent cpu?

[Zoey2936]

modsec is building: https://github.com/ZoeyVid/nginx-quic/actions/runs/5012341255

[Zoey2936]

got it working, a last fix is building here: https://github.com/ZoeyVid/nginx-quic/actions/runs/5015297036/jobs/8990672365 when this build is finished, I will do a last test and push it to latest, modsec will be always enabled

[Zoey2936]

It works, but I wont will push it to latest, because of this: coreruleset/coreruleset#3216, if you need it you can use the zoeyvid/nginx-proxy-manager:develop image, but note: this image has always the latest commit from this repo and breaks very often

[Zoey2936]

I will publish modsec, when coreruleset/coreruleset#3218 is merged, I will also add options for manual config (already in readme)

[dontcrash]

Wow, progress movies fast on this repo! Great work @Zoey2936

[Zoey2936]

thanks

[theraw]

is this modesecurity checking 127.0.0.1 or something else ? i think localhost should be whitelisted..

[Zoey2936]

didyou got a 403 error?

[theraw]

nope, its just that its checking requests from localhost i believe it shouldn't

[Zoey2936]

whrer did you find that logs?

[Zoey2936]

since if modsec does block something, it will return a 403 error

[theraw]

/usr/local/nginx/conf # cat nginx.conf
user root;
daemon off;
pcre_jit on;
worker_processes auto;
error_log /data/logs/error.log debug;
...

since if modsec does block something, it will return a 403 error

yes didn't block anything, but its still checking localhost requests made to /api i don't think its necessary to check requests coming from localhost. i believe modsec has option for ip whitelist to add the private ip ranges used by localhost/docker

[Zoey2936]

You can whitelist it yourself if you want, config files are free to edit in /opt/npm/etc/modsecurity, but I wouldn't do that (ip overriding while portforwarding/wrong proxiing/wrong forwarded for headers, etc.)

[Zoey2936]

just want to add another thing: nginx released today nginx v1.25.0 with HTTP/3 / Quic support, so if openresty will update to it in some weeks/months I will switch to openresty, I switched today from cloning the branch to using the published tar file

[theraw]

You can whitelist it yourself if you want, config files are free to edit in /opt/npm/etc/modsecurity, but I wouldn't do that (ip overriding while portforwarding/wrong proxiing/wrong forwarded for headers, etc.)

yeah sorry looks like i was using an older image and i couldn't find /data/etc/modsecurity

just want to add another thing: nginx released today nginx v1.25.0 with HTTP/3 / Quic support, so if openresty will update to it in some weeks/months I will switch to openresty, I switched today from cloning the branch to using the published tar file

i'll make a copy and save it i don't like openresty 🤷‍♂️

[Zoey2936]

i'll make a copy and save it i don't like openresty 🤷‍♂️

can you explain, why? I like to stick to nginx, but I think openresty has the better lua implementation...

[theraw]

can you explain, why? I like to stick to nginx, but I think openresty has the better lua implementation...

nothing special just personal preference, i use nginx plus in production with some lua scripting to filter traffic and communicate with backends/apis/redis and thats been running perfectly for me so i don't need more stuff, ngx-lua module was meant for nginx but its maintainers decided to get a copy of nginx and make openresty and keep on suggesting people to install openresty i feel like they are forcing it thats why i don't like it

[Zoey2936]

I understand what you mean, but I think that the lua implementation is better when using openresty and openresty also has good patches...

[theraw]

openresty also has good patches...

true and they tend to release stuff faster then nginx aswell, but nginx has a better and more stable development process and is less likely to cause bugs so its more suitable for production use.. imo

[Zoey2936]

I've pushed this now to latest.