Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AWS Security Lake output #387

Merged
merged 1 commit into from
Jan 3, 2023
Merged

AWS Security Lake output #387

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,10 @@ It works as a single endpoint for as many as you want `Falco` instances :
- [**Node-RED**](https://nodered.org/)
- [**WebUI**](https://github.com/falcosecurity/falcosidekick-ui) (a Web UI for displaying latest events in real time)

### SIEM

- [**AWS Security Lake**](https://aws.amazon.com/security-lake/)

### Other
- [**Policy Report**](https://github.com/kubernetes-sigs/wg-policy-prototypes/tree/master/policy-report/falco-adapter)

Expand Down Expand Up @@ -288,6 +292,7 @@ aws:
# accesskeyid: "" # aws access key (optional if you use EC2 Instance Profile)
# secretaccesskey: "" # aws secret access key (optional if you use EC2 Instance Profile)
# region : "" # aws region (by default, the metadata are used to get it)
# checkidentity: true # check the identity credentials, set to false for locale developments (default: true)
lambda:
# functionname : "" # Lambda function name, if not empty, AWS Lambda output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
Expand All @@ -306,6 +311,14 @@ aws:
# bucket: "falcosidekick" # AWS S3, bucket name
# prefix : "" # name of prefix, keys will have format: s3://<bucket>/<prefix>/YYYY-MM-DD/YYYY-MM-DDTHH:mm:ss.s+01:00.json
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
securitylake.:
# bucket: "" # Bucket for AWS SecurityLake data, if not empty, AWS SecurityLake output is enabled
# region: "" # Bucket Region
# prefix: "" # Prefix for keys
# accountid: "" # Account ID
interval: 5 # Time in minutes between two puts to S3 (must be between 5 and 60min) (default: 5min)
batchsize: 1000 # Max number of events by parquet file (default: 1000)
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
kinesis:
# streamname: "" # AWS Kinesis Stream Name, if not empty, Kinesis output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
Expand Down Expand Up @@ -758,6 +771,14 @@ care of lower/uppercases**) : `yaml: a.b --> envvar: A_B` :
_enabled_
- **AWS_S3_PREFIX** : Prefix name of the object, keys will have format: s3://<bucket>/<prefix>/YYYY-MM-DD/YYYY-MM-DDTHH:mm:ss.s+01:00.json
- **AWS_S3_MINIMUMPRIORITY** : minimum priority of event for using this output,
- **AWS_SECURITYLAKE_BUCKET** : Bucket for AWS SecurityLake data, if not empty, AWS SecurityLake. output is _enabled_
- **AWS_SECURITYLAKE_REGION** : Bucket Region (mandatory)
- **AWS_SECURITYLAKE_PREFIX** : Prefix for keys (mandatory)
- **AWS_SECURITYLAKE_ACCOUNTID** : Account ID (mandatory)
- **AWS_SECURITYLAKE_INTERVAL** : Time in minutes between two puts to S3 (must be between 5 and 60min) (default: 5min)
- **AWS_SECURITYLAKE_BATCHSIZE** : Max number of events by parquet file (default: 1000)
- **AWS_SECURITYLAKE_PREFIX** : Prefix name of the object, keys will have format: s3://<bucket>/<prefix>/YYYY-MM-DD/YYYY-MM-DDTHH:mm:ss.s+01:00.json
- **AWS_SECURITYLAKE_MINIMUMPRIORITY** : minimum priority of event for using this output,
order is
- **AWS_KINESIS_STREAMNAME** : AWS Kinesis Stream Name, if not empty, Kinesis output is enabled
- **AWS_KINESIS_MINIMUMPRIORITY** : minimum priority of event for using
Expand Down
17 changes: 17 additions & 0 deletions config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ func getConfig() *types.Configuration {
v.SetDefault("AWS.AccessKeyID", "")
v.SetDefault("AWS.SecretAccessKey", "")
v.SetDefault("AWS.Region", "")
v.SetDefault("AWS.CheckIdentity", true)

v.SetDefault("AWS.Lambda.FunctionName", "")
v.SetDefault("AWS.Lambda.InvocationType", "RequestResponse")
Expand All @@ -154,6 +155,14 @@ func getConfig() *types.Configuration {
v.SetDefault("AWS.S3.Prefix", "falco")
v.SetDefault("AWS.S3.MinimumPriority", "")

v.SetDefault("AWS.SecurityLake.Bucket", "")
v.SetDefault("AWS.SecurityLake.Region", "")
v.SetDefault("AWS.SecurityLake.Prefix", "")
v.SetDefault("AWS.SecurityLake.Interval", 5)
v.SetDefault("AWS.SecurityLake.BatchSize", 1000)
v.SetDefault("AWS.SecurityLake.AccountID", "")
v.SetDefault("AWS.SecurityLake.MinimumPriority", "")

v.SetDefault("AWS.Kinesis.StreamName", "")
v.SetDefault("AWS.Kinesis.MinimumPriority", "")

Expand Down Expand Up @@ -507,6 +516,13 @@ func getConfig() *types.Configuration {
}
}

if c.AWS.SecurityLake.Interval < 5 {
c.AWS.SecurityLake.Interval = 5
}
if c.AWS.SecurityLake.Interval > 60 {
c.AWS.SecurityLake.Interval = 60
}

if c.ListenPort == 0 || c.ListenPort > 65536 {
log.Fatalf("[ERROR] : Bad port number\n")
}
Expand Down Expand Up @@ -538,6 +554,7 @@ func getConfig() *types.Configuration {
c.AWS.SQS.MinimumPriority = checkPriority(c.AWS.SQS.MinimumPriority)
c.AWS.SNS.MinimumPriority = checkPriority(c.AWS.SNS.MinimumPriority)
c.AWS.S3.MinimumPriority = checkPriority(c.AWS.S3.MinimumPriority)
c.AWS.SecurityLake.MinimumPriority = checkPriority(c.AWS.SecurityLake.MinimumPriority)
c.AWS.CloudWatchLogs.MinimumPriority = checkPriority(c.AWS.CloudWatchLogs.MinimumPriority)
c.AWS.Kinesis.MinimumPriority = checkPriority(c.AWS.Kinesis.MinimumPriority)
c.Opsgenie.MinimumPriority = checkPriority(c.Opsgenie.MinimumPriority)
Expand Down
23 changes: 16 additions & 7 deletions config_example.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,12 +114,13 @@ aws:
# accesskeyid: "" # aws access key (optional if you use EC2 Instance Profile)
# secretaccesskey: "" # aws secret access key (optional if you use EC2 Instance Profile)
# region : "" # aws region (by default, the metadata are used to get it)
# checkidentity: true # check the identity credentials, set to false for locale developments (default: true)
lambda:
# functionname : "" # Lambda function name, if not empty, AWS Lambda output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
# functionname : "" # Lambda function name, if not empty, AWS Lambda output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
sqs:
# url : "" # SQS Queue URL, if not empty, AWS SQS output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
# url : "" # SQS Queue URL, if not empty, AWS SQS output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
sns:
# topicarn : "" # SNS TopicArn, if not empty, AWS SNS output is enabled
rawjson: false # Send Raw JSON or parse it (default: false)
Expand All @@ -129,9 +130,17 @@ aws:
# logstream : "" # AWS CloudWatch Logs Stream name, if empty, Falcosidekick will try to create a log stream
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
s3:
# bucket: "falcosidekick" # AWS S3, bucket name
# prefix : "" # name of prefix, keys will have format: s3://<bucket>/<prefix>/YYYY-MM-DD/YYYY-MM-DDTHH:mm:ss.s+01:00.json
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
# bucket: "falcosidekick" # AWS S3, bucket name
# prefix : "" # name of prefix, keys will have format: s3://<bucket>/<prefix>/YYYY-MM-DD/YYYY-MM-DDTHH:mm:ss.s+01:00.json
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
securitylake.:
# bucket: "" # Bucket for AWS SecurityLake data, if not empty, AWS SecurityLake output is enabled
# region: "" # Bucket Region (mandatory)
# prefix: "" # Prefix for keys (mandatory)
# accountid: "" # Account ID (mandatory)
interval: 5 # Time in minutes between two puts to S3 (must be between 5 and 60min) (default: 5min)
batchsize: 1000 # Max number of events by parquet file (default: 1000)
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
kinesis:
# streamname: "" # AWS Kinesis Stream Name, if not empty, Kinesis output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
Expand Down
7 changes: 7 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ require (
github.com/aws/aws-sdk-go v1.44.89
github.com/cloudevents/sdk-go/v2 v2.11.0
github.com/eclipse/paho.mqtt.golang v1.4.1
github.com/embano1/memlog v0.4.3
github.com/emersion/go-sasl v0.0.0-20211008083017-0b9dcfb154ac
github.com/emersion/go-smtp v0.15.0
github.com/google/uuid v1.3.0
Expand All @@ -26,6 +27,8 @@ require (
github.com/streadway/amqp v1.0.0
github.com/stretchr/testify v1.8.0
github.com/wavefronthq/wavefront-sdk-go v0.10.3
github.com/xitongsys/parquet-go v1.6.2
github.com/xitongsys/parquet-go-source v0.0.0-20220723234337-052319f3f36b
golang.org/x/oauth2 v0.0.0-20220822191816-0ebed06d0094
google.golang.org/api v0.94.0
google.golang.org/genproto v0.0.0-20220829175752-36a9c930ecbf
Expand Down Expand Up @@ -54,6 +57,9 @@ require (
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/apache/arrow/go/arrow v0.0.0-20200730104253-651201b0f516 // indirect
github.com/apache/thrift v0.14.2 // indirect
github.com/benbjohnson/clock v1.3.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/caio/go-tdigest v3.1.0+incompatible // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
Expand All @@ -69,6 +75,7 @@ require (
github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.3 // indirect
github.com/google/gnostic v0.6.9 // indirect
github.com/google/go-cmp v0.5.8 // indirect
github.com/google/go-querystring v1.1.0 // indirect
Expand Down
Loading