Skip to content
This repository has been archived by the owner on Aug 14, 2024. It is now read-only.

Specification: Better scrubbing of sensitive data #773

Merged
merged 9 commits into from
Jan 11, 2023
58 changes: 47 additions & 11 deletions src/docs/sdk/data-handling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,19 @@ and is **disabled by default**. That means that data that is naturally sensitive

Some examples of data guarded by this flag:

- When attaching HTTP requests to events
- Request Body: "raw" bodies (bodies which cannot be parsed as JSON or formdata) are removed
- HTTP Headers: known sensitive headers such as `Authorization` or `Cookies` are removed too.
- *Note* that if a user explicitly sets a request on the scope, nothing is stripped from that request. The above rules only apply to integrations that come with the SDK.
- User-specific information (e.g. the current user ID according to the used web-framework) is not sent at all.
- On desktop applications
- The username logged in the device is not included. This is often a person's name.
- The machine name is not included, for example `Bruno's laptop`
- SDKs don't set `{{auto}}` as `user.ip`. This instructs the server to keep the connection's IP address.*
- When attaching HTTP requests to events
antonpirker marked this conversation as resolved.
Show resolved Hide resolved
- Request Body: "raw" bodies (bodies which cannot be parsed as JSON or formdata) are removed
- HTTP Headers: known sensitive headers such as `Authorization` or `Cookies` are removed too.
- _Note_ that if a user explicitly sets a request on the scope, nothing is stripped from that request. The above rules only apply to integrations that come with the SDK.
antonpirker marked this conversation as resolved.
Show resolved Hide resolved
- User-specific information (e.g. the current user ID according to the used web-framework) is not sent at all.
- On desktop applications
- The username logged in the device is not included. This is often a person's name.
- The machine name is not included, for example `Bruno's laptop`
- SDKs don't set `{{auto}}` as `user.ip`. This instructs the server to keep the connection's IP address.\*
antonpirker marked this conversation as resolved.
Show resolved Hide resolved

* Specifically about IP address, it's important to note that it's standard to log IP address of incoming connecting in services on the Internet.
This not only allows security tools and operations to understand abuse coming from a single IP, like spam bots and other issues.
But also developers to understand if issues in their application are being triggered by a single malicious source.
This not only allows security tools and operations to understand abuse coming from a single IP, like spam bots and other issues.
But also developers to understand if issues in their application are being triggered by a single malicious source.

Sentry server is always aware of the connecting IP address and can use it for logging in some platforms. Namely JavaScript and iOS/macOS/tvOS.
All other platforms require the event to include `user.ip={{auto}}` which happens if `sendDefaultPii` is set to true.
Expand All @@ -51,6 +51,42 @@ Some examples of auto instrumentation that could attach sensitive data:
- Desktop apps including window title.
- A Web framework routing instrumentation attaching route `to` and `from`.

## Structuring Data

For better data scrubbing on the server side, SDKs should save data in a strucutured way, when possible. Starting point of the discussion was [RFC-0038](https://github.com/getsentry/rfcs/blob/main/text/0038-scrubbing-sensitive-data.md)
antonpirker marked this conversation as resolved.
Show resolved Hide resolved

### Spans

This helps Relay to know what kind of data it receives and this helps with scrubbing sensitive data.

- `http` spans containing urls:

The description of spans with `op` set to `http` must follow the format `HTTP_METHOD scheme://host/path` (ex. `GET https://example.com/foo`).
If an authority is present in the URL (`https://username:[email protected]`), the authority must be omitted completely.
If query strings or fragments are present in the URL, both are set into the data attribute of the span.
gggritso marked this conversation as resolved.
Show resolved Hide resolved

```js
span.setData({
'http.query': url.getQuery(),
antonpirker marked this conversation as resolved.
Show resolved Hide resolved
'http.fragment: url.getFragment(),
})
```

Additionally all semantic conventions of OpenTelementry for http spans should be set in the `span.data` if applicable:
https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/http/

- `db` spans containing database queries: (sql, graphql, elasticsearch, mongodb, ...)
antonpirker marked this conversation as resolved.
Show resolved Hide resolved

The `description` fields should include the saniticed database command. All sensitive data should be removed and replaced with a placeholder.
antonpirker marked this conversation as resolved.
Show resolved Hide resolved
gggritso marked this conversation as resolved.
Show resolved Hide resolved

Additionally all semantic conventions of OpenTelementry for database spans should be set in the `span.data` if applicable:
https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/database/

### Breadcrumbs

If the `message` in a breadcrumb contains an URL it should be formatted the same way as in `http` spans (see above).
antonpirker marked this conversation as resolved.
Show resolved Hide resolved
The query and the fragment should also be set in the data attribute like with `http` spans.

## Variable Size

Fields in the event payload that allow user-specified or dynamic values are restricted in size. This applies to most meta data fields, such as variables in a stack trace, as well as contexts, tags and extra data:
Expand Down