Alert context appsec

[buixor]

Add alert context support to appsec rules. Closes #3286

When we release 1.6.4, we should add the following alert context to the appsec collection(s) to be iso with what exists:

context:
  id:
   - match.id
  name:
   - match.name
  method:
   - match.method
  uri:
   - match.uri
  matched_zones:
   - match.matched_zones
  msg:
   - match.msg

However, we're now exposing both match (types.MatchedRule) and req (http.Request) to the user's alert context rules. This allows extracting more relevant context:

context:
#extract the content of specific header
  smth:
     - req.Header.Get("foobar")
 #extract all cookies
  cookies:
     - req.Cookies()
  #parse the cookie and extract a specific one
  foobar_cookie:
     - find(req.Cookies(), {#.Name == "yop"}).String()

Todo:

• Coverage

[github-actions]

@buixor: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind fix
• /kind chore
• /kind dependencies

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions]

@buixor: There are no area labels on this PR. You can add as many areas as you see fit.

• /area agent
• /area local-api
• /area cscli
• /area appsec
• /area security
• /area configuration

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[buixor]

todo: use expr lib

[codecov]

Codecov Report

Attention: Patch coverage is 70.13889% with 43 lines in your changes missing coverage. Please review.

Project coverage is 58.31%. Comparing base (d8bc17b) to head (37c0447).
Report is 10 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/acquisition/modules/appsec/utils.go	24.32%	19 Missing and 9 partials ⚠️
pkg/alertcontext/alertcontext.go	85.41%	12 Missing and 2 partials ⚠️
pkg/acquisition/modules/appsec/appsec_runner.go	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #3288       +/-   ##
===========================================
+ Coverage   39.27%   58.31%   +19.04%     
===========================================
  Files         480      349      -131     
  Lines       62217    37389    -24828     
===========================================
- Hits        24437    21805     -2632     
+ Misses      35124    13697    -21427     
+ Partials     2656     1887      -769     

Flag	Coverage Δ
bats	41.43% <36.80%> (+10.00%)	⬆️
unit-linux	34.05% <70.13%> (-0.71%)	⬇️
unit-windows	29.29% <63.88%> (+4.53%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[buixor]

/area appsec
/kind feature

[LaurenceJJones]

Testing PR:

Added contents to appsec_base.yaml provided via the hub

root@bookworm:/opt/crowdsec# cat /etc/crowdsec/contexts/appsec_base.yaml
context:
  test_header:
   - req.Header.Get("x-test")
  id:
   - match.id
  name:
   - match.name
  method:
   - match.method
  uri:
   - match.uri
  matched_zones:
   - match.matched_zones
  msg:
   - match.msg
## These already existed below ##
  rules:
    - evt.Meta.rule_name
  target_uri:
    - evt.Meta.target_uri
## ---- ##

produces the following context with the request curl -H "X-Test: mytest" http://192.168.121.18/.env

 - Context  :
╭───────────────┬─────────────────────────────────╮
│      Key      │              Value              │
├───────────────┼─────────────────────────────────┤
│ id            │ 3256131465                      │
│ matched_zones │ REQUEST_FILENAME                │
│ method        │ GET                             │
│ msg           │ crowdsecurity/vpatch-env-access │
│ name          │ crowdsecurity/vpatch-env-access │
│ test_header   │ mytest                          │
│ uri           │ /.env                           │
╰───────────────┴─────────────────────────────────╯

Some things to note is the msg and name seem the be the exact same so we might want to only provide one or the other depending if custom rules will no produce the same duplicate effect.

To test directly the reason for the PR I added the following to the context file:

context:
  host:
   - req.Host

Sending these 2 curl commands:

curl -H "X-Test: mytest" http://192.168.121.18/.env -vv
curl -H "X-Test: mytest" -H "Host: test.example.com" http://192.168.121.18/.env -vv

Produces the following contexts:

root@bookworm:/opt/crowdsec# cscli alerts inspect 6

################################################################################################

 - ID           : 6
 - Date         : 2024-10-20T08:41:28Z
 - Machine      : c1a6b6ec59fa4f10977727267fd20e4d
 - Simulation   : false
 - Remediation  : false
 - Reason       : crowdsecurity/vpatch-env-access
 - Events Count : 0
 - Scope:Value  : Ip:192.168.121.1
 - Country      :
 - AS           :
 - Begin        : 2024-10-20 08:41:28 +0000 UTC
 - End          : 2024-10-20 08:41:28 +0000 UTC
 - UUID         : bfd99f7b-a806-452e-be42-597d7eb3b6c9


 - Context  :
╭───────────────┬─────────────────────────────────╮
│      Key      │              Value              │
├───────────────┼─────────────────────────────────┤
│ host          │ 192.168.121.18                  │
│ id            │ 3256131465                      │
│ matched_zones │ REQUEST_FILENAME                │
│ method        │ GET                             │
│ msg           │ crowdsecurity/vpatch-env-access │
│ name          │ crowdsecurity/vpatch-env-access │
│ test_header   │ mytest                          │
│ uri           │ /.env                           │
╰───────────────┴─────────────────────────────────╯
root@bookworm:/opt/crowdsec# cscli alerts inspect 7

################################################################################################

 - ID           : 7
 - Date         : 2024-10-20T08:41:51Z
 - Machine      : c1a6b6ec59fa4f10977727267fd20e4d
 - Simulation   : false
 - Remediation  : false
 - Reason       : crowdsecurity/vpatch-env-access
 - Events Count : 0
 - Scope:Value  : Ip:192.168.121.1
 - Country      :
 - AS           :
 - Begin        : 2024-10-20 08:41:51 +0000 UTC
 - End          : 2024-10-20 08:41:51 +0000 UTC
 - UUID         : b8ab4385-5c5f-41c8-be6e-462b6eac04ca


 - Context  :
╭───────────────┬─────────────────────────────────╮
│      Key      │              Value              │
├───────────────┼─────────────────────────────────┤
│ host          │ test.example.com                │
│ id            │ 3256131465                      │
│ matched_zones │ REQUEST_FILENAME                │
│ method        │ GET                             │
│ msg           │ crowdsecurity/vpatch-env-access │
│ name          │ crowdsecurity/vpatch-env-access │
│ test_header   │ mytest                          │
│ uri           │ /.env                           │
╰───────────────┴─────────────────────────────────╯

Example showing context on console

[buixor]

👍 for keeping only name or msg. Is your statement valid too for modsec native rules ?

Todo:

• Create the AC PR for appsec