Skip to content

Commit

Permalink
Add support for regular expression matching and sanitizing of headers…
Browse files Browse the repository at this point in the history 
… in FastAPI.
  • Loading branch information
Daniel Rogers committed Nov 7, 2022
1 parent 40e4e2e commit 73e863f
Show file tree
Hide file tree
Showing 3 changed files with 128 additions and 51 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- `opentelemetry-instrumentation-pymysql` Add tests for commit() and rollback().
([#1424](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1424))
- `opentelemetry-instrumentation-fastapi` Add support for regular expression matching and sanitization of HTTP headers.
([#1403](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1403))

### Fixed

Expand Down
95 changes: 69 additions & 26 deletions ...entelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ async def foobar():
Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FASTAPI_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from tracking, set the environment variable ``OTEL_PYTHON_FASTAPI_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` to cover all instrumentations) to a string of comma delimited regexes that match the
URLs.
For example,
Expand All @@ -45,7 +46,7 @@ async def foobar():
will exclude requests such as ``https://site/client/123/info`` and ``https://site/xyz/healthcheck``.
You can also pass the comma delimited regexes to the ``instrument_app`` method directly:
You can also pass comma delimited regexes directly to the ``instrument_app`` method:
.. code-block:: python
Expand All @@ -54,9 +55,12 @@ async def foobar():
Request/Response hooks
**********************
Utilize request/response hooks to execute custom logic to be performed before/after performing a request. The server request hook takes in a server span and ASGI
scope object for every incoming request. The client request hook is called with the internal span and an ASGI scope which is sent as a dictionary for when the method receive is called.
The client response hook is called with the internal span and an ASGI event which is sent as a dictionary for when the method send is called.
This instrumentation supports request and response hooks. These are functions that get called
right after a span is created for a request and right before the span is finished for the response.
- The server request hook is passed a server span and ASGI scope object for every incoming request.
- The client request hook is called with the internal span and an ASGI scope when the method ``receive`` is called.
- The client response hook is called with the internal span and an ASGI event when the method ``send`` is called.
.. code-block:: python
Expand All @@ -76,54 +80,93 @@ def client_response_hook(span: Span, message: dict):
Capture HTTP request and response headers
*****************************************
You can configure the agent to capture predefined HTTP headers as span attributes, according to the `semantic convention <https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#http-request-and-response-headers>`_.
You can configure the agent to capture specified HTTP headers as span attributes, according to the
`semantic convention <https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#http-request-and-response-headers>`_.
Request headers
***************
To capture predefined HTTP request headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST``
to a comma-separated list of HTTP header names.
To capture HTTP request headers as span attributes, set the environment variable
``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to a comma delimited list of HTTP header names.
For example,
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="content-type,custom_request_header"
will extract ``content-type`` and ``custom_request_header`` from request headers and add them as span attributes.
will extract ``content-type`` and ``custom_request_header`` from the request headers and add them as span attributes.
Request header names in FastAPI are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment
variable will capture the header named ``custom-header``.
Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example:
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="Accept.*,X-.*"
It is recommended that you should give the correct names of the headers to be captured in the environment variable.
Request header names in fastapi are case insensitive. So, giving header name as ``CUStom-Header`` in environment variable will be able capture header with name ``custom-header``.
Would match all request headers that start with ``Accept`` and ``X-``.
The name of the added span attribute will follow the format ``http.request.header.<header_name>`` where ``<header_name>`` being the normalized HTTP header name (lowercase, with - characters replaced by _ ).
The value of the attribute will be single item list containing all the header values.
To capture all request headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to ``".*"``.
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=".*"
The name of the added span attribute will follow the format ``http.request.header.<header_name>`` where ``<header_name>``
is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a
single item list containing all the header values.
Example of the added span attribute,
For example:
``http.request.header.custom_request_header = ["<value1>,<value2>"]``
Response headers
****************
To capture predefined HTTP response headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE``
to a comma-separated list of HTTP header names.
To capture HTTP response headers as span attributes, set the environment variable
``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to a comma delimited list of HTTP header names.
For example,
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="content-type,custom_response_header"
will extract ``content-type`` and ``custom_response_header`` from response headers and add them as span attributes.
will extract ``content-type`` and ``custom_response_header`` from the response headers and add them as span attributes.
Response header names in FastAPI are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment
variable will capture the header named ``custom-header``.
It is recommended that you should give the correct names of the headers to be captured in the environment variable.
Response header names captured in fastapi are case insensitive. So, giving header name as ``CUStomHeader`` in environment variable will be able capture header with name ``customheader``.
Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example:
::
The name of the added span attribute will follow the format ``http.response.header.<header_name>`` where ``<header_name>`` being the normalized HTTP header name (lowercase, with - characters replaced by _ ).
The value of the attribute will be single item list containing all the header values.
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="Content.*,X-.*"
Example of the added span attribute,
Would match all response headers that start with ``Content`` and ``X-``.
To capture all response headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to ``".*"``.
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE=".*"
The name of the added span attribute will follow the format ``http.response.header.<header_name>`` where ``<header_name>``
is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a
single item list containing all the header values.
For example:
``http.response.header.custom_response_header = ["<value1>,<value2>"]``
Sanitizing headers
******************
In order to prevent storing sensitive data such as personally identifiable information (PII), session keys, passwords,
etc, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS``
to a comma delimited list of HTTP header names to be sanitized. Regexes may be used, and all header names will be
matched in a case-insensitive manner.
For example,
::
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS=".*session.*,set-cookie"
will replace the value of headers such as ``session-id`` and ``set-cookie`` with ``[REDACTED]`` in the span.
Note:
Environment variable names to capture http headers are still experimental, and thus are subject to change.
The environment variable names used to capture HTTP headers are still experimental, and thus are subject to change.
API
---
Expand Down
82 changes: 57 additions & 25 deletions instrumentation/opentelemetry-instrumentation-fastapi/tests/test_fastapi_instrumentation.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
from opentelemetry.test.globals_test import reset_trace_globals
from opentelemetry.test.test_base import TestBase
from opentelemetry.util.http import (
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS,
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST,
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE,
_active_requests_count_attrs,
Expand Down Expand Up @@ -529,24 +530,23 @@ def test_mark_span_internal_in_presence_of_span_from_other_framework(self):
)


@patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*",
},
)
class TestHTTPAppWithCustomHeaders(TestBase):
def setUp(self):
super().setUp()
self.env_patch = patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
},
)
self.env_patch.start()
self.app = self._create_app()
otel_fastapi.FastAPIInstrumentor().instrument_app(self.app)
self.client = TestClient(self.app)

def tearDown(self) -> None:
super().tearDown()
self.env_patch.stop()
with self.disable_logging():
otel_fastapi.FastAPIInstrumentor().uninstrument_app(self.app)

Expand All @@ -559,6 +559,9 @@ async def _():
headers = {
"custom-test-header-1": "test-header-value-1",
"custom-test-header-2": "test-header-value-2",
"my-custom-regex-header-1": "my-custom-regex-value-1,my-custom-regex-value-2",
"My-Custom-Regex-Header-2": "my-custom-regex-value-3,my-custom-regex-value-4",
"My-Secret-Header": "My Secret Value",
}
content = {"message": "hello world"}
return JSONResponse(content=content, headers=headers)
Expand All @@ -573,12 +576,20 @@ def test_http_custom_request_headers_in_span_attributes(self):
"http.request.header.custom_test_header_2": (
"test-header-value-2",
),
"http.request.header.regex_test_header_1": ("Regex Test Value 1",),
"http.request.header.regex_test_header_2": (
"RegexTestValue2,RegexTestValue3",
),
"http.request.header.my_secret_header": ("[REDACTED]",),
}
resp = self.client.get(
"/foobar",
headers={
"custom-test-header-1": "test-header-value-1",
"custom-test-header-2": "test-header-value-2",
"Regex-Test-Header-1": "Regex Test Value 1",
"regex-test-header-2": "RegexTestValue2,RegexTestValue3",
"My-Secret-Header": "My Secret Value",
},
)
self.assertEqual(200, resp.status_code)
Expand All @@ -602,6 +613,9 @@ def test_http_custom_request_headers_not_in_span_attributes(self):
headers={
"custom-test-header-1": "test-header-value-1",
"custom-test-header-2": "test-header-value-2",
"Regex-Test-Header-1": "Regex Test Value 1",
"regex-test-header-2": "RegexTestValue2,RegexTestValue3",
"My-Secret-Header": "My Secret Value",
},
)
self.assertEqual(200, resp.status_code)
Expand All @@ -623,6 +637,13 @@ def test_http_custom_response_headers_in_span_attributes(self):
"http.response.header.custom_test_header_2": (
"test-header-value-2",
),
"http.response.header.my_custom_regex_header_1": (
"my-custom-regex-value-1,my-custom-regex-value-2",
),
"http.response.header.my_custom_regex_header_2": (
"my-custom-regex-value-3,my-custom-regex-value-4",
),
"http.response.header.my_secret_header": ("[REDACTED]",),
}
resp = self.client.get("/foobar")
self.assertEqual(200, resp.status_code)
Expand Down Expand Up @@ -653,24 +674,23 @@ def test_http_custom_response_headers_not_in_span_attributes(self):
self.assertNotIn(key, server_span.attributes)


@patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*",
},
)
class TestWebSocketAppWithCustomHeaders(TestBase):
def setUp(self):
super().setUp()
self.env_patch = patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
},
)
self.env_patch.start()
self.app = self._create_app()
otel_fastapi.FastAPIInstrumentor().instrument_app(self.app)
self.client = TestClient(self.app)

def tearDown(self) -> None:
super().tearDown()
self.env_patch.stop()
with self.disable_logging():
otel_fastapi.FastAPIInstrumentor().uninstrument_app(self.app)

Expand All @@ -688,6 +708,12 @@ async def _(websocket: fastapi.WebSocket):
"headers": [
(b"custom-test-header-1", b"test-header-value-1"),
(b"custom-test-header-2", b"test-header-value-2"),
(b"Regex-Test-Header-1", b"Regex Test Value 1"),
(
b"regex-test-header-2",
b"RegexTestValue2,RegexTestValue3",
),
(b"My-Secret-Header", b"My Secret Value"),
],
}
)
Expand Down Expand Up @@ -727,6 +753,13 @@ def test_web_socket_custom_request_headers_in_span_attributes(self):

self.assertSpanHasAttributes(server_span, expected)

@patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
},
)
def test_web_socket_custom_request_headers_not_in_span_attributes(self):
not_expected = {
"http.request.header.custom_test_header_3": (
Expand Down Expand Up @@ -799,16 +832,15 @@ def test_web_socket_custom_response_headers_not_in_span_attributes(self):
self.assertNotIn(key, server_span.attributes)


@patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
},
)
class TestNonRecordingSpanWithCustomHeaders(TestBase):
def setUp(self):
super().setUp()
self.env_patch = patch.dict(
"os.environ",
{
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
},
)
self.env_patch.start()
self.app = fastapi.FastAPI()

@self.app.get("/foobar")
Expand Down

0 comments on commit 73e863f

Please sign in to comment.