Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zeek log severity scoring #19

Closed
mmguero opened this issue Sep 9, 2020 · 2 comments
Closed

Zeek log severity scoring #19

mmguero opened this issue Sep 9, 2020 · 2 comments
Labels
enhancement New feature or request logstash Relating to Malcolm's use of Logstash

Comments

@mmguero
Copy link
Collaborator

mmguero commented Sep 9, 2020

From Malcolm created by mmguero: cisagov#125

The idea is that we assign a severity rating to logs (all logs? some logs?)

So, imagine 1 - not severe at all (blue or green), 5 - super severe (red)

in Logstash enrichment we'd do stuff like:

cleartext password - 5
connection to naughty country - 5
certain notices - 5
insecure or old versions of protocols - 4
file transfers of certain mime types - 3
connection within subnet - 1
connection to other subnet - 2
connection to outside world - 3
etc.

Of course those are just examples. I'd need to hammer out a real list.

Then in some of the dashboards, we can have "number of red events" "number of green events" etc.

@mmguero mmguero added enhancement New feature or request logstash Relating to Malcolm's use of Logstash labels Sep 9, 2020
@mmguero
Copy link
Collaborator Author

mmguero commented Sep 9, 2020

I'm going to start spitballing a list of categories that could be assigned to logs. I'll edit this list as we come up with more of them, we'll assign severities later. Also, note that a lot of these same things are already visualizations in the security overview dashboards.

One more note: these are on a per-log basis. We're not talking about bursts, trends, baselines, buckets, or anything here. Those kinds of things are a bigger issue and require some other machine learning or statistical analysis tool, and will be covered in another issue.

  • file transfer of low-risk types (0 severity by default)
  • file transfer of medium-risk types
  • file transfer of high-risk types
    • executables, scripts, etc.
  • notice.log generated
    • probably with different severities for different notice categories
  • signatures.log generated
  • weird.log or dpd.log generated
  • clear text transmission of passwords
  • outdated/insecure protocol versions
  • internal-to-external network connections
  • cross-subnet network connections
  • external-to-internal network connections
    • higher severity with "remote control" protocols like rsh, telnet, ssh, rdesktop, vns, etc.
  • countries on some sort of "higher severity" list? (n. korea, china, russia, etc.?)
  • possible DGA malware based on randomness/entropy score
  • rank connections by size
    • connection duration
    • bytes
  • connection state
  • protocols on non-standard ports
  • tunneled traffic

the issue with these ones below is the list is so, so varied from protocol to protocol I don't think it'll be possible to characterize all possible options. it might be possible to do it for particular protocols/actions though (like, say, for HTTP or LDAP or others that have a finite list of codes). may come back to this

  • rank "action" for all protocols that populate it
    • this is quite a list
    • in general writes/sets are more severe than reads/gets, etc.
  • any kind of access denied/file not found/login success or failure
    • ssh, telnet, http, ftp, kerberos, ldap, ntlm, radius, rdp, rfb, whatever
    • success and failure ranked differently?
  • server/client information stuff from software.log
    • certain software categories, vendors or versions?
    • specific user agents strings?

@mmguero
Copy link
Collaborator Author

mmguero commented Sep 9, 2020

Some notes:

  • elastic common schema (if we're going to make a new field for severity we might as well use something standard going forward)

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 16, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 17, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 17, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 17, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 17, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 17, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 18, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 18, 2021
@mmguero mmguero changed the title "security posture" by assigning severity to events Zeek log severity scoring Aug 18, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 18, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 19, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 19, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 19, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 19, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 20, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 26, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 26, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Sep 2, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Sep 2, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Sep 2, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Sep 2, 2021
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Sep 2, 2021
mmguero added a commit to cisagov/Malcolm that referenced this issue Sep 3, 2021
* New features
    * Automatically create some broadly useful anomaly detectors when initializing Kibana
        * connection size
        * file transfer MIME type
        * action and result (by application protocol)
    * Configurable [event severity scoring](https://github.com/cisagov/malcolm/tree/main#Severity) (idaholab#19) and new **Severity** dashboard

* Other changes
    * vagrant-based ISO build can now work with either VirtualBox or libvirt providers
    * change wording of terms such as "master"/"slave" to "client"/"server" as instructed by DHS directive

* Version updates
    * Update base image for Debian-based Docker images from 10 (buster) to 11 (bullseye)
    * Update Yara to 4.1.2
    * Update Capa to 2.0.0
    * Update Spicy to 1.2.1
    * Update remainder of python 2 code to python 3
@mmguero mmguero closed this as completed Sep 16, 2021
@mmguero mmguero added this to Malcolm Oct 14, 2024
@mmguero mmguero moved this to Released in Malcolm Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request logstash Relating to Malcolm's use of Logstash
Projects
Status: Released
Development

No branches or pull requests

1 participant