Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APPSEC-8115] create AppSec::Monitor to subscribe to internal app sec events #2617

Merged
merged 13 commits into from
Feb 23, 2023
Merged

[APPSEC-8115] create AppSec::Monitor to subscribe to internal app sec events #2617

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
a27ebc2
create new reactive component to listen to set_user events
GustavoCaso Feb 14, 2023
ccee5a0
store waf context on the local thread. Make sure to reset it at the e…
GustavoCaso Feb 14, 2023
b6130cf
refactor contrib gateway watch method and split each subscribition in…
GustavoCaso Feb 14, 2023
fd15c5c
rack watch for user_id events
GustavoCaso Feb 14, 2023
cb552de
update rbs signature
GustavoCaso Feb 14, 2023
96394a1
rename current_context to active_context
GustavoCaso Feb 17, 2023
0ea7be4
introduce Processor#activate_context and Processor#deactive_context
GustavoCaso Feb 17, 2023
00e6205
create Appsec::Monitor to handle internal appsec events agnostic to i…
GustavoCaso Feb 17, 2023
a9fa5a2
move monitor initialization to AppSec::Component
GustavoCaso Feb 22, 2023
d280590
skip steep check for lib/datadog/appsec/component.rb
GustavoCaso Feb 22, 2023
e625807
raise AlreadyActiveContextError if trying to activate a new context w…
GustavoCaso Feb 23, 2023
b174cfb
move appsec monitor watch to rack patcher
GustavoCaso Feb 23, 2023
4abbd89
remove sorbet type checking job from CircleCI
GustavoCaso Feb 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 0 additions & 21 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,10 +138,6 @@ step_rubocop: &step_rubocop
# The workaround is to use `cpu.shares / 1024`:
# https://discuss.circleci.com/t/environment-variable-set-to-the-number-of-available-cpus/32670/4
command: PARALLEL_PROCESSOR_COUNT=$((`cat /sys/fs/cgroup/cpu/cpu.shares` / 1024)) bundle exec rake rubocop
step_sorbet_type_checker: &step_sorbet_type_checker
run:
name: Run sorbet type checker
command: bundle exec rake typecheck
Comment on lines -141 to -144
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Adding this for future reference) See #2641 for details on why we're removing Sorbet.

step_appraisal_install: &step_appraisal_install
run:
name: Install Appraisal gems
Expand Down Expand Up @@ -348,16 +344,6 @@ orbs:
keys:
- bundle-{{ .Environment.CIRCLE_CACHE_VERSION }}-{{ checksum ".circleci/images/primary/binary_version" }}-<<parameters.ruby_version>>-{{ checksum "lib/ddtrace/version.rb" }}-{{ .Branch }}-{{ checksum ".circleci/bundle_checksum" }}
- *step_rubocop
sorbet_type_checker:
<<: *test_job_default
steps:
- restore_cache:
keys:
- '{{ .Environment.CIRCLE_CACHE_VERSION }}-bundled-repo-<<parameters.ruby_version>>-{{ .Environment.CIRCLE_SHA1 }}'
- restore_cache:
keys:
- bundle-{{ .Environment.CIRCLE_CACHE_VERSION }}-{{ checksum ".circleci/images/primary/binary_version" }}-<<parameters.ruby_version>>-{{ checksum "lib/ddtrace/version.rb" }}-{{ .Branch }}-{{ checksum ".circleci/bundle_checksum" }}
- *step_sorbet_type_checker
coverage:
<<: *test_job_default
steps:
Expand Down Expand Up @@ -564,11 +550,6 @@ workflows:
name: lint
requires:
- build-2.7
- orb/sorbet_type_checker:
<<: *config-2_7-small
name: sorbet_type_checker
requires:
- build-2.7
- orb/coverage:
<<: *config-2_7-small
name: coverage
Expand Down Expand Up @@ -766,7 +747,6 @@ workflows:
<<: *filters_all_branches_and_tags
requires:
- lint
- sorbet_type_checker
- test-2.1
- test-2.2
- test-2.3
Expand All @@ -786,7 +766,6 @@ workflows:
<<: *filters_only_release_tags
requires:
- lint
- sorbet_type_checker
- test-2.1
- test-2.2
- test-2.3
Expand Down
2 changes: 2 additions & 0 deletions Steepfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ target :appsec do
# check 'lib/datadog/kit'

ignore 'lib/datadog/appsec/contrib'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is ignored because of the dependency on actual Sinatra/Rails/Rack constants and methods, which are not typed. We can relax that ignore by minimally typing just what we need in vendor/rbs.

ignore 'lib/datadog/appsec/monitor'
ignore 'lib/datadog/appsec/component.rb'
Comment on lines +10 to +11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were there any challenges in typing these that resulted in them being ignored?


library 'pathname', 'set'
library 'cgi'
Expand Down
218 changes: 111 additions & 107 deletions lib/datadog/appsec/contrib/rack/gateway/watcher.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,140 +14,144 @@ module Rack
module Gateway
# Watcher for Rack gateway events
module Watcher
# rubocop:disable Metrics/AbcSize
# rubocop:disable Metrics/MethodLength
# rubocop:disable Metrics/CyclomaticComplexity
# rubocop:disable Metrics/PerceivedComplexity
def self.watch
Instrumentation.gateway.watch('rack.request', :appsec) do |stack, request|
block = false
event = nil
waf_context = request.env['datadog.waf.context']

AppSec::Reactive::Operation.new('rack.request') do |op|
trace = active_trace
span = active_span

Rack::Reactive::Request.subscribe(op, waf_context) do |result, _block|
if result.status == :match
# TODO: should this hash be an Event instance instead?
event = {
waf_result: result,
trace: trace,
span: span,
request: request,
actions: result.actions
}

span.set_tag('appsec.event', 'true') if span

waf_context.events << event
class << self
def watch
gateway = Instrumentation.gateway

watch_request(gateway)
watch_response(gateway)
watch_request_body(gateway)
end

def watch_request(gateway = Instrumentation.gateway)
gateway.watch('rack.request', :appsec) do |stack, request|
block = false
event = nil
waf_context = request.env['datadog.waf.context']

AppSec::Reactive::Operation.new('rack.request') do |op|
trace = active_trace
span = active_span

Rack::Reactive::Request.subscribe(op, waf_context) do |result, _block|
if result.status == :match
# TODO: should this hash be an Event instance instead?
event = {
waf_result: result,
trace: trace,
span: span,
request: request,
actions: result.actions
}

span.set_tag('appsec.event', 'true') if span

waf_context.events << event
end
end

_result, block = Rack::Reactive::Request.publish(op, request)
end

_result, block = Rack::Reactive::Request.publish(op, request)
end
next [nil, [[:block, event]]] if block

next [nil, [[:block, event]]] if block
ret, res = stack.call(request)

ret, res = stack.call(request)
if event
res ||= []
res << [:monitor, event]
end

if event
res ||= []
res << [:monitor, event]
[ret, res]
end

[ret, res]
end

Instrumentation.gateway.watch('rack.response', :appsec) do |stack, response|
block = false
event = nil
waf_context = response.instance_eval { @waf_context }

AppSec::Reactive::Operation.new('rack.response') do |op|
trace = active_trace
span = active_span

Rack::Reactive::Response.subscribe(op, waf_context) do |result, _block|
if result.status == :match
# TODO: should this hash be an Event instance instead?
event = {
waf_result: result,
trace: trace,
span: span,
response: response,
actions: result.actions
}

span.set_tag('appsec.event', 'true') if span

waf_context.events << event
def watch_response(gateway = Instrumentation.gateway)
gateway.watch('rack.response', :appsec) do |stack, response|
block = false
event = nil
waf_context = response.instance_eval { @waf_context }

AppSec::Reactive::Operation.new('rack.response') do |op|
trace = active_trace
span = active_span

Rack::Reactive::Response.subscribe(op, waf_context) do |result, _block|
if result.status == :match
# TODO: should this hash be an Event instance instead?
event = {
waf_result: result,
trace: trace,
span: span,
response: response,
actions: result.actions
}

span.set_tag('appsec.event', 'true') if span

waf_context.events << event
end
end

_result, block = Rack::Reactive::Response.publish(op, response)
end

_result, block = Rack::Reactive::Response.publish(op, response)
end
next [nil, [[:block, event]]] if block

next [nil, [[:block, event]]] if block
ret, res = stack.call(response)

ret, res = stack.call(response)
if event
res ||= []
res << [:monitor, event]
end

if event
res ||= []
res << [:monitor, event]
[ret, res]
end

[ret, res]
end

Instrumentation.gateway.watch('rack.request.body', :appsec) do |stack, request|
block = false
event = nil
waf_context = request.env['datadog.waf.context']

AppSec::Reactive::Operation.new('rack.request.body') do |op|
trace = active_trace
span = active_span

Rack::Reactive::RequestBody.subscribe(op, waf_context) do |result, _block|
if result.status == :match
# TODO: should this hash be an Event instance instead?
event = {
waf_result: result,
trace: trace,
span: span,
request: request,
actions: result.actions
}

span.set_tag('appsec.event', 'true') if span

waf_context.events << event
def watch_request_body(gateway = Instrumentation.gateway)
gateway.watch('rack.request.body', :appsec) do |stack, request|
block = false
event = nil
waf_context = request.env['datadog.waf.context']

AppSec::Reactive::Operation.new('rack.request.body') do |op|
trace = active_trace
span = active_span

Rack::Reactive::RequestBody.subscribe(op, waf_context) do |result, _block|
if result.status == :match
# TODO: should this hash be an Event instance instead?
event = {
waf_result: result,
trace: trace,
span: span,
request: request,
actions: result.actions
}

span.set_tag('appsec.event', 'true') if span

waf_context.events << event
end
end

_result, block = Rack::Reactive::RequestBody.publish(op, request)
end

_result, block = Rack::Reactive::RequestBody.publish(op, request)
end
next [nil, [[:block, event]]] if block

next [nil, [[:block, event]]] if block
ret, res = stack.call(request)

ret, res = stack.call(request)
if event
res ||= []
res << [:monitor, event]
end

if event
res ||= []
res << [:monitor, event]
[ret, res]
end

[ret, res]
end
end
# rubocop:enable Metrics/CyclomaticComplexity
# rubocop:enable Metrics/PerceivedComplexity
# rubocop:enable Metrics/MethodLength
# rubocop:enable Metrics/AbcSize

class << self
private

def active_trace
Expand Down
2 changes: 2 additions & 0 deletions lib/datadog/appsec/contrib/rack/patcher.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# typed: ignore

require_relative '../patcher'
require_relative '../../monitor'
require_relative 'gateway/watcher'

module Datadog
Expand All @@ -22,6 +23,7 @@ def target_version
end

def patch
Monitor::Gateway::Watcher.watch
Gateway::Watcher.watch
Patcher.instance_variable_set(:@patched, true)
end
Expand Down
4 changes: 2 additions & 2 deletions lib/datadog/appsec/contrib/rack/request_middleware.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ def call(env)

# TODO: handle exceptions, except for @app.call

context = processor.new_context
context = processor.activate_context
env['datadog.waf.context'] = context

request = ::Rack::Request.new(env)
Expand Down Expand Up @@ -68,7 +68,7 @@ def call(env)
ensure
if context
add_waf_runtime_tags(active_trace, context)
context.finalize
processor.deactivate_context
end
end

Expand Down
Loading