-
Notifications
You must be signed in to change notification settings - Fork 30
Commits on Sep 30, 2020
-
Fixes - (#158) PII: make the PII scrubbing of In-App WAF attack events case-insensitive in order to correctly scrub transformed request parameters. - (#159) Monitoring: fix the content type and length monitoring of HTTP responses. - (#157) Gin middleware: use the request Go context instead of Gin's so that the agent can properly manage the request execution context, but also to correctly propagate values stored in the Go context before the middleware function.
Julio Guerra committedSep 30, 2020 Configuration menu - View commit details
-
Copy full SHA for 378102d - Browse repository at this point
Copy the full SHA 378102dView commit details
Commits on Oct 1, 2020
-
Julio Guerra committed
Oct 1, 2020 Configuration menu - View commit details
-
Copy full SHA for e530ae8 - Browse repository at this point
Copy the full SHA e530ae8View commit details -
ci: add github's codeql analysis
Julio Guerra authoredOct 1, 2020 Configuration menu - View commit details
-
Copy full SHA for 9451a76 - Browse repository at this point
Copy the full SHA 9451a76View commit details
Commits on Oct 14, 2020
-
agent/metrics: add binning metrics
Add a new type of metrics for binning metrics and explicitely rename the previous sum metrics store types to make the API clearer.
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for e688a38 - Browse repository at this point
Copy the full SHA e688a38View commit details -
agent/metrics: poc version integrating time and performance histograms
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for e74b9a0 - Browse repository at this point
Copy the full SHA e74b9a0View commit details -
agent: performance histogram integration
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for bc3cd5d - Browse repository at this point
Copy the full SHA bc3cd5dView commit details -
sqlib/sqtime: add a shared stopwatch
Add a thread-safe shared stopwatch implementation accouting time duration between the first goroutine starting it and the last stopping. It will be used to compute sqreen's execution time per request for the simplest preformance monitoring level 1. It indeed allows to read time less frequenttl than multiple detailed timers.
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for c57c89a - Browse repository at this point
Copy the full SHA c57c89aView commit details -
agent/rule/callback/waf: refactor the callbacks to the new native cal…
…lback api
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for 220fd2c - Browse repository at this point
Copy the full SHA 220fd2cView commit details -
agent/rule/callback: callback framework
Introduce a complete callback framework providing a fully managed and abstraced callback-implementation API. Two new interfaces were introduced for that: 1. The RuleContext is the interface with the security rule the callback is serving. It hides implementation details such as the rule name, the blocking mode, etc. This context is provided at callback instantiation. 2. The CallbackContext is the result of a RuleContext within a given ProtectionContext. It is obtained by calling the RuleContext's Pre() and Post() methods, both expecting a function closure having the CallbackContext as argument. And this is how the callback gets wrapped with rule- and protection-specific features. The closure also allows to do the bridge with the hooked function for example to set its return values when blocking. This patch ports every callback to this new architecture.
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for c11f415 - Browse repository at this point
Copy the full SHA c11f415View commit details -
Julio Guerra committed
Oct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for d71cf72 - Browse repository at this point
Copy the full SHA d71cf72View commit details -
go: update go-sdk to v1.2.0 including binning metrics
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for 5934786 - Browse repository at this point
Copy the full SHA 5934786View commit details -
agent: refactoring protection/agent interfaces
Some improvements - still not perfect - simplifying the protection API and expectations regarding the agent.
Julio Guerra committedOct 14, 2020 Configuration menu - View commit details
-
Copy full SHA for a794b40 - Browse repository at this point
Copy the full SHA a794b40View commit details
Commits on Oct 15, 2020
-
agent/rule/callback: fix the callcount callback wrapper
Julio Guerra committedOct 15, 2020 Configuration menu - View commit details
-
Copy full SHA for 3e01632 - Browse repository at this point
Copy the full SHA 3e01632View commit details -
sdk/middleware/sqecho/v4: update the middleware with the recent prote…
…ction context changes
Julio Guerra committedOct 15, 2020 Configuration menu - View commit details
-
Copy full SHA for 2876050 - Browse repository at this point
Copy the full SHA 2876050View commit details -
agent/protection/http: fix tests
Julio Guerra committedOct 15, 2020 Configuration menu - View commit details
-
Copy full SHA for 15ca756 - Browse repository at this point
Copy the full SHA 15ca756View commit details
Commits on Oct 19, 2020
-
sdk/middleware: fix broken tests
Julio Guerra committedOct 19, 2020 Configuration menu - View commit details
-
Copy full SHA for 7583241 - Browse repository at this point
Copy the full SHA 7583241View commit details -
agent/protection/http: fix http protection tests
Julio Guerra committedOct 19, 2020 Configuration menu - View commit details
-
Copy full SHA for 6f337da - Browse repository at this point
Copy the full SHA 6f337daView commit details -
Julio Guerra committed
Oct 19, 2020 Configuration menu - View commit details
-
Copy full SHA for 229e82c - Browse repository at this point
Copy the full SHA 229e82cView commit details -
agent/backend/api: disable flaky fuzzing test
Julio Guerra committedOct 19, 2020 Configuration menu - View commit details
-
Copy full SHA for 70c0426 - Browse repository at this point
Copy the full SHA 70c0426View commit details -
agent/rule/callback: fix callback tests
Julio Guerra committedOct 19, 2020 Configuration menu - View commit details
-
Copy full SHA for deb4bda - Browse repository at this point
Copy the full SHA deb4bdaView commit details
Commits on Oct 20, 2020
-
sqlib/sqtime: fix first/last start/stop race
Julio Guerra committedOct 20, 2020 Configuration menu - View commit details
-
Copy full SHA for d264bb0 - Browse repository at this point
Copy the full SHA d264bb0View commit details -
agent/rule/callback/security-responses: fix blocking attack api
Julio Guerra committedOct 20, 2020 Configuration menu - View commit details
-
Copy full SHA for f7414d1 - Browse repository at this point
Copy the full SHA f7414d1View commit details -
agent/metrics: fix ongoing values handling
Julio Guerra committedOct 20, 2020 Configuration menu - View commit details
-
Copy full SHA for 1caf4d2 - Browse repository at this point
Copy the full SHA 1caf4d2View commit details -
agent/metrics: update metrics documentation
Julio Guerra committedOct 20, 2020 Configuration menu - View commit details
-
Copy full SHA for 4d95c3a - Browse repository at this point
Copy the full SHA 4d95c3aView commit details
Commits on Oct 21, 2020
-
agent/rule/callback: ignore the perf cap on critical callbacks
Julio Guerra committedOct 21, 2020 Configuration menu - View commit details
-
Copy full SHA for 0869c28 - Browse repository at this point
Copy the full SHA 0869c28View commit details
Commits on Oct 22, 2020
-
agent/rule/callback: add tests of callback middlewares
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 0df2594 - Browse repository at this point
Copy the full SHA 0df2594View commit details -
agent/rule/callback: fix go1.12 compilation error
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for fedf3fa - Browse repository at this point
Copy the full SHA fedf3faView commit details -
agent/rule/callback: try fix flaky windows test
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 6e0c42a - Browse repository at this point
Copy the full SHA 6e0c42aView commit details -
agent/rule/callback: try fix flaky windows test
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 1072066 - Browse repository at this point
Copy the full SHA 1072066View commit details -
agent/metrics: add an extra test to try to find windows issue
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 66d6f39 - Browse repository at this point
Copy the full SHA 66d6f39View commit details -
agent/metrics: try debug windows panic
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 8b3df8e - Browse repository at this point
Copy the full SHA 8b3df8eView commit details -
agent/actors: try fix a flaky windows test by using the monotonic clock
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for bdf2d9b - Browse repository at this point
Copy the full SHA bdf2d9bView commit details -
agent/metrics: fix max value when ongoing
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 2b73910 - Browse repository at this point
Copy the full SHA 2b73910View commit details -
agent/rule/waf: add some deadline checks to abort the waf when the de…
…adline exceeded
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 41c8af8 - Browse repository at this point
Copy the full SHA 41c8af8View commit details -
agent/metrics: remove debug block of code
Julio Guerra committedOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for 7c8604f - Browse repository at this point
Copy the full SHA 7c8604fView commit details -
Monitor the execution time of requests protected by Sqreen. Optionally, it is now possible to set the maximum amount of time Sqreen is allowed to run per request: Sqreen's monitoring and protections will only run for the given amount of time. This option is disabled by default and should be used with caution as it can lead to partially protected requests. The resulting performance monitoring diagrams and setting are available at <https://my.sqreen.com/application/goto/settings/performance>. Note that the execution time monitoring diagram cannot be used as a strict Application Performance Monitoring diagram as it is based on a lossy exponential time-interval representation.
Julio Guerra authoredOct 22, 2020 Configuration menu - View commit details
-
Copy full SHA for e4f8db2 - Browse repository at this point
Copy the full SHA e4f8db2View commit details
Commits on Oct 28, 2020
-
agent/rule/callback/monitor-http-status-code: add not_found attack
Log a `not_found` attack when the response is the NotFound status code 404. This attack is used by an internal backend playbook to detect vulnerability security scanners. The `HandleAttack()` API has been adapted so that callbacks can have more options and finer-grained control on the attack event creation. In order to avoid the cost of having multiple callbacks on the HTTP status code monitoring hookpoint (which would require the use of Go reflection), the not_found attack is added to the existing monitoring callback.
Julio Guerra committedOct 28, 2020 Configuration menu - View commit details
-
Copy full SHA for a542038 - Browse repository at this point
Copy the full SHA a542038View commit details -
agent/rule: adapt to the new HandleAttack API
Julio Guerra committedOct 28, 2020 Configuration menu - View commit details
-
Copy full SHA for 94a43cf - Browse repository at this point
Copy the full SHA 94a43cfView commit details -
agent/rule/callback/monitor-http-status-code: add tests
Julio Guerra committedOct 28, 2020 Configuration menu - View commit details
-
Copy full SHA for 65f548a - Browse repository at this point
Copy the full SHA 65f548aView commit details -
agent: enhance the event manager event throughtput
When dogfooding the not_found attack, we suddenly started having a high-frequency event. We therefore scaled the event queue to 10000 events by default, and the number of goroutines sending them to Sqreen can now scale up to the number of available CPUs. This allows to increase the overall throughput and handle more events rather than dropping them.
Julio Guerra committedOct 28, 2020 Configuration menu - View commit details
-
Copy full SHA for 6e98daa - Browse repository at this point
Copy the full SHA 6e98daaView commit details
Commits on Oct 29, 2020
-
agent/rule/callback/monitor-http-status-code: add not_found events
Log a `not_found` attack when the response is the NotFound status code 404. This attack is used by an internal backend playbook to detect vulnerability security scanners. The `HandleAttack()` API has been adapted so that callbacks can have more options and finer-grained control on the attack event creation. In order to avoid the cost of having multiple callbacks on the HTTP status code monitoring hookpoint (which would require the use of Go reflection), the not_found attack is added to the existing monitoring callback. When dogfooding the not_found attack, we suddenly started having a high-frequency event. We therefore scaled the event queue to 10000 events by default, and the number of goroutines sending them to Sqreen can now scale up to the number of available CPUs. This allows to increase the overall throughput and handle more events rather than dropping them.
Julio Guerra authoredOct 29, 2020 Configuration menu - View commit details
-
Copy full SHA for 856c0a5 - Browse repository at this point
Copy the full SHA 856c0a5View commit details
Commits on Oct 30, 2020
-
sdk/middleware/echo: fix status code monitoring of handler errors
Monitor the return error of Echo request handlers to see if they are Echo's HTTPErrors, so that we can properly capture the HTTP status code that will be later used by Echo. The HTTP response is indded out of the middleware scope in that case.
Julio Guerra committedOct 30, 2020 Configuration menu - View commit details
-
Copy full SHA for 7c8c77b - Browse repository at this point
Copy the full SHA 7c8c77bView commit details -
sdk/middleware/echo: fix status code monitoring of handler errors
Monitor the return error of Echo request handlers to see if they are Echo's HTTPErrors, so that we can properly capture the HTTP status code that will be later used by Echo. The HTTP response is otherwise out of the middleware scope in that case.
Julio Guerra authoredOct 30, 2020 Configuration menu - View commit details
-
Copy full SHA for 344793d - Browse repository at this point
Copy the full SHA 344793dView commit details
Commits on Nov 2, 2020
-
Add backoff error logging for better error management in the request hotpaths. A backoff error logger is created using `plog.WithBackoff()`. It manages error counters and logs the error every time the counter value is a power of two. Error can be either individually counted when wrapped by `sqerrors.WithKey()`, or simply share the common counter of the backoff error logger object.
Julio Guerra committedNov 2, 2020 Configuration menu - View commit details
-
Copy full SHA for bb740d2 - Browse repository at this point
Copy the full SHA bb740d2View commit details -
agent: backoff error logger integration
Simple backoff error logger integration in the agent and its callbacks. The error indexing should be further defined over time as it is currently oversampling due to shared keys (eg. avoiding using the same key for two distinct binding accessor expresssions).
Julio Guerra committedNov 2, 2020 Configuration menu - View commit details
-
Copy full SHA for 0b0339d - Browse repository at this point
Copy the full SHA 0b0339dView commit details
Commits on Nov 3, 2020
-
agent: general fixes after overall dogfooding
Julio Guerra committedNov 3, 2020 Configuration menu - View commit details
-
Copy full SHA for 33c9148 - Browse repository at this point
Copy the full SHA 33c9148View commit details -
agent: hot path error management
Critical execution errors happening in the request hot path are now logged according to exponential backoff counters of their occurrences. This avoids slowing down request handlers but also spamming the internal agent queue. Such error will now go through the agent error logging facility every time the backoff counter is a power of two (1, 2, 4, 8, etc.). This is disabled when in debug log level.
Julio Guerra authoredNov 3, 2020 Configuration menu - View commit details
-
Copy full SHA for f6e3046 - Browse repository at this point
Copy the full SHA f6e3046View commit details -
sdk/middleware/sqgin: fix response's content-lenght monitoring
Correctly handle the default response case.
Julio Guerra committedNov 3, 2020 Configuration menu - View commit details
-
Copy full SHA for be1a2c6 - Browse repository at this point
Copy the full SHA be1a2c6View commit details -
sdk/middleware: add response monitoring tests
Julio Guerra committedNov 3, 2020 Configuration menu - View commit details
-
Copy full SHA for 9a1e81d - Browse repository at this point
Copy the full SHA 9a1e81dView commit details -
sdk/middleware/sqgin: response's content length monitoring
Fix the response's content-length monitoring with the default response (ie. the handler does nothing).
Julio Guerra authoredNov 3, 2020 Configuration menu - View commit details
-
Copy full SHA for ce914a0 - Browse repository at this point
Copy the full SHA ce914a0View commit details
Commits on Nov 5, 2020
-
agent/rule/callback/security-response: fix the event name and restore…
… the tests Use the track event name the backend expects. Restoring the tests highlighted some inconsistent blocking behavior between user/ip blocking/redirecting that are now fixed.
Julio Guerra committedNov 5, 2020 Configuration menu - View commit details
-
Copy full SHA for f6ddb94 - Browse repository at this point
Copy the full SHA f6ddb94View commit details -
agent/rule/callback/security-responses: fix the event name of securit…
…y response actions Enable the correct event display on the dashboard by fixing the track event name with the one the backend actually expects. Also, restoring the security response tests highlighted some inconsistent blocking behavior between user/ip blocking/redirecting that are now fixed.
Julio Guerra authoredNov 5, 2020 Configuration menu - View commit details
-
Copy full SHA for ecb633a - Browse repository at this point
Copy the full SHA ecb633aView commit details -
sdk: move to public interface API
Moving to an interface-only public API has the benefit of hiding the type-implementation details that may change over time in the future. The returned SDK values are now interface values and no longer pointer values. This may break integrations having explicit types. We recommend to instead use type-inference when possible. From now, this change will allow us to transparently change the underlying values returned by SDK calls. For example, as of today, the underlying SDK values are small event collection wrappers that shouldn't be returned by reference in order to avoid useless allocations, for the benefit of the overall application performance. If the underlying object size increase in the future, we will be able to return allocated structures instead without breaking existing SDK integrations.
Julio Guerra committedNov 5, 2020 Configuration menu - View commit details
-
Copy full SHA for 7067c0b - Browse repository at this point
Copy the full SHA 7067c0bView commit details
Commits on Nov 6, 2020
-
sdk: interface-only public sdk api
Moving to an interface-only public API has the benefit of hiding the type-implementation details that may change over time in the future. The returned SDK values are now interface values and no longer pointer values. This may break integrations having explicit types. We recommend to instead use type-inference when possible. From now, this change will allow us to transparently change the underlying values returned by SDK calls. For example, as of today, the underlying SDK values are small event collection wrappers that shouldn't be returned by reference in order to avoid useless allocations, for the benefit of the overall application performance. If the underlying object size increase in the future, we will be able to return allocated structures instead without breaking existing SDK integrations.
Julio Guerra authoredNov 6, 2020 Configuration menu - View commit details
-
Copy full SHA for a2837c6 - Browse repository at this point
Copy the full SHA a2837c6View commit details
Commits on Nov 10, 2020
-
agent/rule/callback/binding-accessors/sql: unwrap the SQL driver befo…
…re reading its package path Following up on elastic/apm-agent-go#848 and the addition of the `Unwrap()` method to Elastic's SQL driver tracer, it is now possible to unwrap it and properly read the package path of the underlying driver as expected.
Julio Guerra committedNov 10, 2020 Configuration menu - View commit details
-
Copy full SHA for 9ff0c80 - Browse repository at this point
Copy the full SHA 9ff0c80View commit details -
agent/rule/callback/binding-accessors/sql: finer-grained SQL dialect …
…method errors Add finer-grained error keys that can be returned by the SQL dialect getter method. This also highlighted the fact errors can be assigned multiple keys while bubbling up the call stack. Therefore, the quick solution to get an error key is to make `sqerrors.Key()` return the deepest key rather than the top one. Ideally, it should rather combine multiple keys together in the future.
Julio Guerra committedNov 10, 2020 Configuration menu - View commit details
-
Copy full SHA for 204c448 - Browse repository at this point
Copy the full SHA 204c448View commit details
Commits on Nov 12, 2020
-
agent/rule/callback/binding-accessors/sql: unwrap the SQL driver befo…
…re reading its package path Following up on elastic/apm-agent-go#848 and the addition of the `Unwrap()` method to Elastic's SQL driver tracer, it is now possible to unwrap it and properly read the package path of the underlying driver as expected.
Julio Guerra authoredNov 12, 2020 Configuration menu - View commit details
-
Copy full SHA for f87a508 - Browse repository at this point
Copy the full SHA f87a508View commit details -
sdk/middleware/sqhttp: transparent response writer wrapper
Make the HTTP response writer wrapper transparent by implementing the same *known* interfaces as the underlying HTTP response writer it implements. The list of interfaces is currently every optional `net/http` interfaces, and some from `io` when relevant: - `http.Flusher`: to allow flushing any buffered to the client. This enables support for streaming handlers. - `http.Hijacker`: to allow handlers to takeover the HTTP connection. This should enable the support for websocket servers, which are not officially supported by Sqreen, but is now experimentally allowed. - `http.Pusher`: for HTTP2 server push. - `http.CloseNotifier`: the deprecated closed connection notifier. - `io.ReaderFrom`: for optimized copies (eg. `io.Copy(file, w)`) - `io.WriteString`: for optimized string write (which avoids a temporary string copy into a byte slice) The transparent wrapper implementation has been generated from a tool that will be released in the Go agent repository in the future. Fixes #162 and #134
Julio Guerra committedNov 12, 2020 Configuration menu - View commit details
-
Copy full SHA for b8d555a - Browse repository at this point
Copy the full SHA b8d555aView commit details -
sdk/middleware/{sqgin,sqecho}: response writer wrapper cleanup
Remove the useless response writer wrapper as Echo and Gin already monitor them. Note that Gin actually doesn't allow changing the writter and all its helpers use the internal in-memory writer. Therefore, we are for now accepting multiple responses to keep the original response writer behaviour.
Julio Guerra committedNov 12, 2020 Configuration menu - View commit details
-
Copy full SHA for 6f3abd2 - Browse repository at this point
Copy the full SHA 6f3abd2View commit details
Commits on Nov 17, 2020
-
agent/protection/http/binding-accessor: unalias url.Values to fix the…
… js conversion
Julio Guerra committedNov 17, 2020 Configuration menu - View commit details
-
Copy full SHA for 58eeeae - Browse repository at this point
Copy the full SHA 58eeeaeView commit details -
sdk/middleware/sqhttp: transparent http response writer wrapper
Make the HTTP response writer wrapper transparent by implementing the same *known* interfaces as the underlying HTTP response writer. The list of interfaces is currently every optional `net/http` interfaces, and some from `io` when relevant: - `http.Flusher`: to allow flushing any buffered to the client. This enables support for streaming handlers. - `http.Hijacker`: to allow handlers to takeover the HTTP connection. This should enable the support for websocket servers, which are not officially supported by Sqreen, but is now experimentally allowed. - `http.Pusher`: for HTTP2 server push. - `http.CloseNotifier`: the deprecated closed connection notifier. - `io.ReaderFrom`: for optimized copies (eg. `io.Copy(file, w)`) - `io.WriteString`: for optimized string write (which avoids a temporary string copy into a byte slice) The transparent wrapper implementation has been generated with a tool that will be released in the Go agent repository in the future. Fixes #162 and #134
Julio Guerra authoredNov 17, 2020 Configuration menu - View commit details
-
Copy full SHA for b4824cc - Browse repository at this point
Copy the full SHA b4824ccView commit details
Commits on Nov 18, 2020
-
agent/metrics: fix metrics regressions
- Restore max length checks - Update max length error checks
Julio Guerra authoredNov 18, 2020 Configuration menu - View commit details
-
Copy full SHA for 9b0856e - Browse repository at this point
Copy the full SHA 9b0856eView commit details -
agent/path-passlist: fix the lookup by prefix rather than exact match
Julio Guerra committedNov 18, 2020 Configuration menu - View commit details
-
Copy full SHA for d4c64f0 - Browse repository at this point
Copy the full SHA d4c64f0View commit details -
agent: fix the early logger log level
Julio Guerra committedNov 18, 2020 Configuration menu - View commit details
-
Copy full SHA for a0d8208 - Browse repository at this point
Copy the full SHA a0d8208View commit details -
sdk: add FromRequest() function helper
Julio Guerra committedNov 18, 2020 Configuration menu - View commit details
-
Copy full SHA for 402e454 - Browse repository at this point
Copy the full SHA 402e454View commit details -
Add `sdk.FromRequest()` helper function to retrieve Sqreen's SDK handle from a request. It is a shortcut to `sdk.FromContext(r.Context())`.
Julio Guerra authoredNov 18, 2020 Configuration menu - View commit details
-
Copy full SHA for ded1fbb - Browse repository at this point
Copy the full SHA ded1fbbView commit details
Commits on Nov 19, 2020
-
Julio Guerra committed
Nov 19, 2020 Configuration menu - View commit details
-
Copy full SHA for 31ce2bb - Browse repository at this point
Copy the full SHA 31ce2bbView commit details