Add CrowdSec OpenResty bouncer, Admin Dashboard logging and ModSecurity #1867
Conversation
…enable. /data/crowdsec/crowdsec-openresty-bouncer.conf is the configuration file for it. Add Admin dashboard logging to the /data/logs/admin-panel_[access/error].log folder - ENV: ADMIN_PANEL_LOG = 1 to enable. Add OpenResty error log (fallback_error.log) debug level - ENV: OPENRESTY_DEBUG = 1 to enable.
…included in the 99_crowdsec-openresty-bouncer.sh file.
baudneo
changed the title
Unknown how to pass jenkins tests...
|
@baudneo I'm trying to use your image and I get the following error with no existing config. I see you pushed an update to the docker container 20mins ago? |
|
Sorry, it should be fixed in next 20 mins or so, everything building. Added PROXY PR and upgraded openresty bouncer. |
|
Image is up and works, but it seems it built from an old builder and didn't propagate all changes. It will be another couple hours until a new build is out with all the new stuff. This current :testing tag will work for now. Add SSL_CERTS_PATH to the environment and set it to "/etc/ssl/certs/GTS_Root_R1.pem" if it complains about that same error. In the new image SSL_CERTS_PATH will be set to "/etc/ssl/certs/GTS_Root_R1.pem" by default (can be overridden). Another fork maintainer found a memory issue when using crowdsec' openresty bouncer and reCAPTCHA remediation. Using Googles root CA specifically to reach out to Google for reCAPTCHA validation was the fix. PROXY protocol from a PR was added with UI changes to configure it. Brotli compression, xslt and image_filter modules have also been added. |
|
Testing image is working but FYI theres a typo in the default value for SSL_CERTS_PATH (lower case 'r' in 'Root') so it complains about that cert not existing. Fixed by manually setting that var though |
|
Yes sorry about that, I reused a buildx builder instance and the new changes did not propogate. It's rebuilding multi arch images from scratch right now with the upgrades. The crowdsec html templates are also missing until this build completes. EDIT: Templates are located at |
|
Something has changed ? Because this morning I get an issue trying to log in the WebUI; Click to expand... |
I tried to recreate your error and am unsuccessful. Try editing Another thing to try is to stop the container, remove it and then If you want to revert to the base image, feel free. Template pathsoutput from me pulling and starting the imageWhen I remove template paths |
|
Thanks @baudneo, I will try later. |
OK, I give another try as you proposed. Then still the same issue which look like to be closed to; |
|
Hmm the thing though is your error says the file is missing completely, not that there are missing elf headers. I am perplexed but I will try and take a look into it as soon as I can. I don't have arm64 SoC to test with at the moment but I will see if I can virtualize something if possible to do some testing. Sorry about that. Can you try |
The file is here... I do not find the I have tried Pretty useless test, but different result ! I was good !! |
|
Another issue with the CAPTCHA I made at Google !? Here are my parameters (all are externally define from the |
The CROWDSEC_RECAP_XXXC are environment variables used only when the file is first created on a user's very first time running npm without having anything in the data/crowdsec directory. Change them to The way to use the CROWDSEC_RECAP vars is to set them as the VALUE of those keys. You would set CROWDSEC_RECAP_SITE and CROWDSEC_RECAP_SECRET as environment variables in your docker compose with the keys as values and then set your config like so On boot of the container the 99 init file should replace those environment variables. BUT, for now I would just set the keys on the crowdsec bouncer .conf. So once you manually.installed bcrypt using node package manager it started working? Thats.... Odd. That means everytime you start that container you will need to install bcrypt from node package manager. I'll try and take a look into what's going on there. It seems weird that on arm64 there is a bcrypt library that isn't registered but the files are there. Edit: the captcha stuff isn't needed if you have remediation set to ban. Changing those 2 keys will get the reCAPTCHA set up, then you can change the "bounce on type" to all/captcha to use captcha. To test that crowdsec is working properly you can add your IP to the decision list manually on your crowdsec LAPI host using Once you are done testing use |
|
There is something else which locks the running ! Then nothing more and not working (more than 10 minutes waiting...) |
|
The logs there seem correct. They don't output much noise besides "SSL renew complete" and a few other things when it's running. Is the admin panel responding? (Port 81) |
|
No, no answer, it is why I state it as not running. |
|
Well I would use the jc22 image for now until I can take a look at the arm64 build. I don't know what's going on with that specific version of the build as the amd54 build is working great, low ram usage and bans/captchas as it should while still serving Admin dashboard and everything else. You are absolutely certain you deleted the old images and pulled new ones? |
Yes, I think so. |
|
Trying to install your image as a direct replacement for another docker instance of npm, but it fails with the following: |
|
I am away for the last little bit and for a bit longer dealing with some personal family stuff and don't have access to my machine to fix builds. You can try the older testing tag that has modsec and an older version of the crowdsec bouncer for now. I will update when 8 return and I will fix these issues. Sorry people. :( |
|
No worries @baudneo - we all have to take care of our family stuff. Best of luck - and we'll see you when you return. |
|
Please be aware that ModSecurity has known memory leaks. I have done a lot of research and work to minimize the memory leak. The ModSec maintainer does not seem to be actively working on fixing the memory leaks. I changed how modsec was compiled and where modsec directives are placed for MODSEC_ENABLE=1. Before the directives were placed in the default server {} block, now it is in the ROOT http {} block. The memory leak increases every time If you only set the MODSEC_ENABLE=1 env var and do not add any modsec directives in the Advanced tab, then the memory leak should only be 10-12 MB per reload. If you have MODSEC_ENBABLE=0 and then use the Voice your concerns about the memory leak in the issue I opened ⇒ owasp-modsecurity/ModSecurity#2817 |
Typos, add info about bullseye image and trivy scans.
Wording.
…dsec memory leaks.
…ecurity # Conflicts: # docker/rootfs/etc/cont-init.d/97_modsecurity.sh # docker/rootfs/etc/nginx/nginx.conf
…security_crowdsecurity
Add notice
UPDATES
ModSec has a known memory leak!
I have no control over fixing that, the modsec maintainer does not seem to be working on fixing the memory leak. For now, all I can recommend is to restart the container every so often.
Do not add multiple
modsecurity on;andmodsecurity_rules_file <path/to/rules/file.conf>directives in the advanced tab of your proxy hosts if you have enabled modsec using MODSEC_ENABLE=1 env var. Modsec is enabled in the root http {} block, meaning it will protect all HTTP servers/locations.To turn modsec off for certain hosts while MODSEC_ENABLE=1. Add a
modsecurity off;directive in your Advanced tab.Voice your concerns about the memory leak on the issue I opened in modsecurity GitHub ⇒ Memory leak on nginx -s reload, even after mem leak fixes of v3/master owasp-modsecurity/ModSecurity#2817
PROXY protocol support
real_ip_headerdirective can only be called once. So if you use CloudFlare and Allow PROXY protocol you can't setreal_ip_header CF-Connecting-IP;cs-openresty-bouncer
$CROWDSEC_RECAP_SECRET$CROWDSEC_RECAP_SITESSL_CERTS_PATHto control CA cert used for reCAPTCHA validation. ca-certificates.pem caused memory issues.docker-compose.yaml
Timezone
Environment Variables
TZ- Set to your timezone. Example:TZ=America/ChicagoConfiguration
TZyou can mount/etc/localtimeinto the docker containerCrowdSec OpenResty Bouncer
NOTE
docker logs --follow [name of your NPM container]. There will be a log line like ->nginx: [alert] [lua] init_by_lua:8: [Crowdsec] Initialisation doneEnvironment Variables
CROWDSEC_BOUNCER=1- Enable CrowdSec OpenResty bouncer, still needs to be configured.CROWDSEC_LAPI=[URL]- configure CrowdSec local API URLCROWDSEC_KEY=[API KEY]- configure CrowdSec API keyCROWDSEC_RECAP_SECRET=[SECRET KEY]- configure reCAPTCHACROWDSEC_RECAP_SITE=[SITE KEY]- configure reCAPTCHASSL_CERTS_PATH- CA certificate used to communicate with Google for reCAPTCHA validationConfiguration
data/crowdsec/crowdsec-openresty-bouncer.conf/crowdsec/templates/inside the containerAdmin dashboard logging / OpenResty DEBUG level logging
Environment Variables
ADMIN_DASHBOARD_LOG=1- Enable admin (Port 81) dashboard loggingOPENRESTY_DEBUG=1- Enable DEBUG level logging for the default OpenRestyERRORlogConfiguration
data/logs/admin-panel_access.loganddata/logs/admin-panel_error.logfallback_access.logandfallback_error.log.DEBUGlevel will be set on the error log, it is set toWARNby default.ModSecurity
ModSecurity WAF is installed and loaded by default, OWASP-CoreRuleSet is installed and used as the default rule set. The user is responsible for configuring modsecurity via config/CLI.
MODSECURITY HAS KNOWN MEMORY LEAKS! Whenever
nginx -s reloadis issued expect RAM to increase by 10 to several hundred MB (depending on if you have custom rule files or are double enabling modsec). Nginx is reloaded every time a host is created/modified and when the SSL cert renewal timer resets. The only way to get modsec to release REAM is to restart nginx from inside the running container usingnginx -s stop; nginx -s reloador by restarting the container usingsudo docker restart [Name of NPM container]Environment Variables
MODSEC_CREATE=1- Force recreating the default modsecurity config, This should never be neededMODSEC_ADMIN_PANEL=1- Enable ModSec for the admin panel (Port 81 web interface)MODSEC_ENABLE=1- Enable ModSec in the ROOT http {} block (Enabled ModSec for ALL HTTP servers)The minimum directives that need to be added to enable modsec.
Tips to enable
modsecurity off;in the Advanced tab.modsecurity off;in a location {} block in the Advanced tab.modsecurity_rules_file <path/to/rules/file.conf>;in Advanced tab. Either in the root of the Advanced tab (for all locations) or inside of location {} blocks (for certain locations). MAY INCREASE MEMORY LEAK SIZE!!!Configuration
data/logs/modsec_audit.logdata/modsecanddata/modsec/rulesetdata/modsec/modsecurity.confis the main modsec config file.data/modsec/main.confis the main rules file, it hasIncludedirectives that load the actual rulesdata/modsecis symbolically linked to/etc/nginx/modsecdocker-compose.yaml