Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add OTLP/Traces output support #613

Merged
merged 13 commits into from
Feb 6, 2024
Merged

Conversation

jjo
Copy link
Contributor

@jjo jjo commented Aug 31, 2023

What type of PR is this?
/kind feature

Any specific area of the project related to this PR?
/area outputs

What this PR does / why we need it:
Implements OTLP/Traces (OpenTelemetry) output.

Folding Falco events into traces allows them to be represented as "hierarchical" views.
Using a TraceID taken from container.id (or any arbitrary output fields configured by the user), all Falco events will be "rooted" under the container (or fieldset) where it occurred, enabling a tree~ish view of these.

Which issue(s) this PR fixes:

Special notes for your reviewer:
Will provide some screenshots of these hierarchical views, as shown by an OTLP Traces backend like Jaeger or Grafana Tempo.

@Issif
Copy link
Member

Issif commented Aug 31, 2023

Hi,

This is a terrific idea and PR!!! Thanks a lot. I'm in holidays til next Monday, I'll review asap.

@jjo
Copy link
Contributor Author

jjo commented Aug 31, 2023

Using below docker-compose.yaml, and querying Grafana UI at http://localhost:3000/explore with Tempo datasource and {} query:
image

  • On the left panel you see the four containers "containing" Falco events

  • On the right, after clicking one of the above (chose the event-generator)

  • docker-compose.yaml:

---
version: "3.9"
services:
  falco:
    image: falcosecurity/falco-no-driver:latest
    privileged: true
    command: "falco --modern-bpf -r /etc/falco/rules"
    volumes:
      - /var/run/docker.sock:/host/var/run/docker.sock
      - /dev:/host/dev
      - /proc:/host/proc:ro
      - /boot:/host/boot:ro
      - /lib/modules:/host/lib/modules:ro
      - ./etc/falco:/etc/falco:ro
  falcosidekick:
     #image: falcosecurity/falcosidekick:latest
     # NB: below image built from github.com/jjo/falcosidekick jjo/OTLP branch
    volumes:
      - ./etc/falco:/etc/falco:ro
    image: xjjo/falco-sidekick:2.28.0-jjo07
    command: -c /etc/falco/falcosidekick.yaml
    ports:
      - 2801:2801
    environment:
      # - OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://jaeger:4318/v1/traces
      - OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://tempo:4318/v1/traces
      - OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc
      - OTEL_EXPORTER_OTLP_HEADERS=X-Scope-OrgID=1
      - OTLP_TRACES_SYNCED=true
#  jaeger:
#    image: jaegertracing/all-in-one
#    command: [
#      "--collector.otlp.http.host-port=0.0.0.0:4318",
#      "--collector.otlp.http.tls.enabled=false"
#    ]
#    ports:
#      - 4318:4318
#      - 16686:16686
  tempo:
    image: grafana/tempo:latest
    ports:
      - 4317
      - 4318
      - 3200
    volumes:
      - ./etc/tempo:/etc/tempo:ro
    command: "-config.file /etc/tempo/config.yaml"
    restart: always
  grafana:
    image: grafana/grafana:10.0.3
    volumes:
      - ./etc/grafana/provisioning:/etc/grafana/provisioning:ro
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_DISABLE_LOGIN_FORM=true
      - GF_FEATURE_TOGGLES_ENABLE=traceqlEditor
    ports:
      - "3000:3000"
  event-generator:
    image: falcosecurity/event-generator
    command: run
    restart: always
  trigger:
    image: alpine
    command: ["sh", "-c", "while true; do cat /etc/shadow > /dev/null; sleep 5; done"]

@jjo
Copy link
Contributor Author

jjo commented Aug 31, 2023

Hi,

This is a terrific idea and PR!!! Thanks a lot. I'm in holidays til next Monday, I'll review asap.

Huh 1st thought it was kinda crazy to fold these as Traces, much appreciate your feedback!

@jjo
Copy link
Contributor Author

jjo commented Aug 31, 2023

And a Jaeger screenshot (using same docker-compose.yaml file, edited to enable Jaeger and disable Tempo)

  1. UI landing:
    image

  2. After clicking one the trace corresponding to the event-generator
    image

Dockerfile Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
handlers.go Outdated Show resolved Hide resolved
otlp_init.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
otlp_init.go Outdated Show resolved Hide resolved
@Issif
Copy link
Member

Issif commented Sep 4, 2023

I did a first review, still have to read the whole logic and understand how it works

outputs/otlp.go Outdated Show resolved Hide resolved
@Issif Issif added this to the 2.29.0 milestone Sep 4, 2023
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
otlp_init.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
config_example.yaml Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
stats.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
outputs/otlp_init.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Show resolved Hide resolved
outputs/otlp.go Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
outputs/otlp.go Outdated Show resolved Hide resolved
@jjo jjo requested a review from Issif September 6, 2023 21:43
config.go Outdated Show resolved Hide resolved
outputs/otlp.go Show resolved Hide resolved
config.go Outdated Show resolved Hide resolved
outputs/otlp.go Show resolved Hide resolved
Signed-off-by: JuanJo Ciarlante <[email protected]>
@jjo
Copy link
Contributor Author

jjo commented Feb 5, 2024

I will create a PR to change that, you will just have to rebase on master then

Thanks, doing

@jjo
Copy link
Contributor Author

jjo commented Feb 5, 2024

Did you test my changes? I guess we're closed to the merge, can you rebase on the master and squash your commits after having applied my suggestions. thanks

Update: stack deployed via docker-compose working ok now (after the "syscalls" fix), as shown by 9353d05 added screenshots.

@jjo jjo requested a review from Issif February 5, 2024 19:06
Issif
Issif previously approved these changes Feb 6, 2024
Copy link
Member

@Issif Issif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@poiana poiana added the lgtm label Feb 6, 2024
@poiana
Copy link

poiana commented Feb 6, 2024

LGTM label has been added.

Git tree hash: 309a97a9393afed7a5257a7b4e90675864bbb40b

@poiana poiana added the approved label Feb 6, 2024
@jjo
Copy link
Contributor Author

jjo commented Feb 6, 2024

🎉

woOOOT ... thanks @Issif especially for the advice and patience towards making this PR a reality 😃!

@Issif
Copy link
Member

Issif commented Feb 6, 2024

🎉

woOOOT ... thanks @Issif especially for the advice and patience towards making this PR a reality 😃!

it remains 2 lint errors and a build issue, can you check them please

Signed-off-by: JuanJo Ciarlante <[email protected]>
@jjo
Copy link
Contributor Author

jjo commented Feb 6, 2024

🎉

woOOOT ... thanks @Issif especially for the advice and patience towards making this PR a reality 😃!

it remains 2 lint errors and a build issue, can you check them please

on them

Signed-off-by: JuanJo Ciarlante <[email protected]>
@Issif
Copy link
Member

Issif commented Feb 6, 2024

/lgtm

@poiana poiana added the lgtm label Feb 6, 2024
@poiana
Copy link

poiana commented Feb 6, 2024

LGTM label has been added.

Git tree hash: f96ce03f839d3a6c1fb80d205b4bf445fa9e24ba

@poiana poiana added the approved label Feb 6, 2024
@poiana
Copy link

poiana commented Feb 6, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Issif, jjo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana poiana merged commit 253113a into falcosecurity:master Feb 6, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants