From f9e723334b5262d9a0fb6998b8ffcffa08e71861 Mon Sep 17 00:00:00 2001 From: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> Date: Tue, 20 Dec 2022 11:26:06 -0500 Subject: [PATCH 01/20] chore(ci): evenly distribute tox scenarios on test runners (#4808) ## Description Work on the alternative method for https://github.com/DataDog/dd-trace-py/pull/4807 to evenly distribute test suites across test runners when running tox scenarios. Previously, our method of distributing tox scenarios across test runners is to divide the number of tox variants by the parallelism (in profile tests case, 15). All test runners run the same number of test scenarios except for the last one, which will additionally run the remainder, which will act as a bottleneck if the remainder is large. For example, the profiler test suite runs 37 tests across 15 runners, meaning every testrunner runs 2 test suites and the last one runs an additional 7. This PR changes it so that we evenly distribute test scenarios across all of the workers, as well as the remainder. This should make it so that each test runner executes either `(num_tests / num_runners)` or `(num_tests / num_runners) + 1` tests. For example, the profiler test suite (most severe example of the former problem) now runs at about 13 minutes compared to 30 previously. ## Checklist - [ ] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [ ] Add additional sections for `feat` and `fix` pull requests. - [ ] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Motivation ## Design ## Testing strategy ## Relevant issue(s) ## Testing strategy ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. --- scripts/run-tox-scenario | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/run-tox-scenario b/scripts/run-tox-scenario index e5eeaf1cc40..24442cbed42 100755 --- a/scripts/run-tox-scenario +++ b/scripts/run-tox-scenario @@ -12,5 +12,5 @@ ENVLIST=$(tox -l | grep "$PATTERN") if [[ -z "${CIRCLE_NODE_TOTAL}" && -z "${CIRCLE_NODE_INDEX}" ]]; then exec echo "$ENVLIST" | tr '\n' ',' | xargs -I ARGS tox -e ARGS -- $@ else - exec echo "$ENVLIST" | sort | python -c "import os, sys; t, i = int(os.getenv('CIRCLE_NODE_TOTAL')), int(os.getenv('CIRCLE_NODE_INDEX')); inp = sys.stdin.readlines(); s = len(inp)//t; print(''.join(inp[i*s:(i+1)*s] if i+1 Date: Tue, 20 Dec 2022 11:58:09 -0500 Subject: [PATCH 02/20] chore(ci): move vertica tests to riot (#4809) ## Description This PR moves the vertica test suite to be run in riot instead of tox. This PR also cleans up the `tox.ini` file by removing any accidentally leftover dependencies/commands/references to test suites that have already been moved over to riot. ## Checklist - [ ] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [ ] Add additional sections for `feat` and `fix` pull requests. - [ ] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Motivation ## Design ## Testing strategy ## Relevant issue(s) ## Testing strategy ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. --- .circleci/config.yml | 26 ++------------------------ riotfile.py | 8 ++++++++ tox.ini | 28 ---------------------------- 3 files changed, 10 insertions(+), 52 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 0384c4dfa18..7a0afd32c8c 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -191,20 +191,9 @@ commands: parameters: pattern: type: string - wait: - type: string - default: "" steps: - checkout - restore_tox_cache - - when: - condition: - << parameters.wait >> - steps: - - setup_riot - - run: - name: "Waiting for << parameters.wait >>" - command: riot -v run 'wait' << parameters.wait >> - start_docker_services: env: SNAPSHOT_CI=1 services: memcached redis testagent @@ -220,9 +209,6 @@ commands: parameters: pattern: type: string - wait: - type: string - default: "" store_coverage: type: boolean default: true @@ -230,14 +216,6 @@ commands: - checkout - setup_tox - restore_tox_cache - - when: - condition: - << parameters.wait >> - steps: - - setup_riot - - run: - name: "Waiting for << parameters.wait >>" - command: riot -v run 'wait' << parameters.wait >> - run: name: "Run scripts/run-tox-scenario" command: scripts/run-tox-scenario '<< parameters.pattern >>' @@ -1076,9 +1054,9 @@ jobs: - VP_TEST_PASSWORD=abc123 - VP_TEST_DATABASE=docker steps: - - run_tox_scenario: + - run_test: wait: vertica - pattern: '^vertica_contrib-' + pattern: 'vertica' wsgi: <<: *machine_executor diff --git a/riotfile.py b/riotfile.py index f954658975d..2a368975ca4 100644 --- a/riotfile.py +++ b/riotfile.py @@ -477,6 +477,14 @@ def select_pys(min_version=MIN_PYTHON_VERSION, max_version=MAX_PYTHON_VERSION): "msgpack": ["~=1.0.0", latest], }, ), + Venv( + name="vertica", + command="pytest {cmdargs} tests/contrib/vertica/", + pys=select_pys(max_version="3.9"), + pkgs={ + "vertica-python": [">=0.6.0,<0.7.0", ">=0.7.0,<0.8.0"], + }, + ), Venv( name="wait", command="python tests/wait-for-services.py {cmdargs}", diff --git a/tox.ini b/tox.ini index 9273cdff171..961e4b3e17d 100644 --- a/tox.ini +++ b/tox.ini @@ -24,14 +24,6 @@ envlist = # FIXME[riot-3.11]: Riot venvs break with Py 3.11 importlib, specifically with hypothesis (test_http.py). # We'll skip the test_http.py tests in riot and run them separately here through tox in CI. py{27,35,36,37,38,39,310,311}-tracer_test_http -# Integrations environments - pymongo_contrib-py{27,35,36,37}-pymongo{30,31,32,33,34,35,36,37,38,39,310,}-mongoengine -# pymongo does not yet support Python 3.8: https://github.com/pymssql/pymssql/issues/586 -# but these tests still work. - pymongo_contrib-py{38,39,310,311}-pymongo{30,31,32,33,35,36,37,38,39,310,}-mongoengine - pynamodb_contrib-py{27,35,36,37,38,39,310,311}-pynamodb{40,41,42,43,}-moto1 - pyodbc_contrib-py{27,35,36,37,38,39}-pyodbc{3,4} - vertica_contrib-py{27,35,36,37,38,39}-vertica{060,070} isolated_build = true @@ -100,11 +92,6 @@ deps = # backports py27: enum34 # integrations - blinker: blinker - futures: futures - futures30: futures>=3.0,<3.1 - futures31: futures>=3.1,<3.2 - futures32: futures>=3.2,<3.3 gevent11: gevent>=1.1,<1.2 gevent12: gevent>=1.2,<1.3 gevent13: gevent>=1.3,<1.4 @@ -117,19 +104,6 @@ deps = # Note - gevent<20.12 does not set a maximum supported version. # To test with gevent<20.12 we need to manually install greenlet<2. greenlet1: greenlet>=1,<2 - memcached: python-memcached - moto: moto - moto1: moto>=1,<2 - mongoengine: mongoengine - pytest: pytest>=3 - pytest3: pytest>=3.0,<4.0 - redis: redis - redis210: redis>=2.10,<2.11 - sqlalchemy: sqlalchemy - vertica060: vertica-python>=0.6.0,<0.7.0 - vertica070: vertica-python>=0.7.0,<0.8.0 - webtest: WebTest - aiobotocore_contrib: pytest-asyncio # pass along test env variables passenv= @@ -148,5 +122,3 @@ commands = py27-profile: python -m tests.profiling.run pytest --capture=no --verbosity=2 --benchmark-disable --ignore-glob="*asyncio*" {posargs} tests/profiling # Coverage is excluded from profile because of an issue with Python 3.5. py3{5,6,7,8,9,10,11}-profile: python -m tests.profiling.run pytest --no-cov --capture=no --verbosity=2 --benchmark-disable {posargs} tests/profiling -# Contribs - vertica_contrib: python -m pytest {posargs} tests/contrib/vertica/ From 0dfd930ea5d9fb7ea1600b35ab0cbbb8052396f4 Mon Sep 17 00:00:00 2001 From: Federico Mon Date: Wed, 21 Dec 2022 07:55:46 +0000 Subject: [PATCH 03/20] chore(iast): remote config thread workaround for gevent (#4749) ## Description Workaround for remote config to work properly with gevent. This PR ensures the Remote config worker thread is launched at the very end, not interrupting the gevent patching mechanism. Without this PR, the gevent patching is interrupted so if gevent is used, it won't work as expected. As a side effect, if gevent is used and Django is installed, our Django patching mechanism collides with gevent raising a Django ImproperlyConfigured error. ## Checklist - [ ] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [ ] Add additional sections for `feat` and `fix` pull requests. - [ ] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. --- ddtrace/appsec/_remoteconfiguration.py | 7 +++++-- ddtrace/internal/writer.py | 5 +++++ ddtrace/tracer.py | 3 --- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/ddtrace/appsec/_remoteconfiguration.py b/ddtrace/appsec/_remoteconfiguration.py index dc2bfab1275..d89c5b8fb4e 100644 --- a/ddtrace/appsec/_remoteconfiguration.py +++ b/ddtrace/appsec/_remoteconfiguration.py @@ -21,8 +21,11 @@ log = get_logger(__name__) -def enable_appsec_rc(tracer): - # type: (Tracer) -> None +def enable_appsec_rc(): + # type: () -> None + # Import tracer here to avoid a circular import + from ddtrace import tracer + if _appsec_rc_features_is_enabled(): RemoteConfig.register(ASM_FEATURES_PRODUCT, appsec_rc_reload_features(tracer)) diff --git a/ddtrace/internal/writer.py b/ddtrace/internal/writer.py index ad11e13a7c9..9c4bb983585 100644 --- a/ddtrace/internal/writer.py +++ b/ddtrace/internal/writer.py @@ -16,6 +16,7 @@ import tenacity import ddtrace +from ddtrace.appsec._remoteconfiguration import enable_appsec_rc from ddtrace.vendor.dogstatsd import DogStatsd from . import agent @@ -509,10 +510,14 @@ def write(self, spans=None): try: if self.status != service.ServiceStatus.RUNNING: self.start() + # instrumentation telemetry writer should be enabled/started after the global tracer and configs # are initialized if asbool(os.getenv("DD_INSTRUMENTATION_TELEMETRY_ENABLED", True)): telemetry_writer.enable() + # appsec remote config should be enabled/started after the global tracer and configs + # are initialized + enable_appsec_rc() except service.ServiceStatusError: pass diff --git a/ddtrace/tracer.py b/ddtrace/tracer.py index 80d135af8cc..309752b717b 100644 --- a/ddtrace/tracer.py +++ b/ddtrace/tracer.py @@ -16,7 +16,6 @@ from typing import Union from ddtrace import config -from ddtrace.appsec._remoteconfiguration import enable_appsec_rc from ddtrace.filters import TraceFilter from ddtrace.internal.sampling import SpanSamplingRule from ddtrace.internal.sampling import get_span_sampling_rules @@ -238,7 +237,6 @@ def __init__( self._single_span_sampling_rules, self._agent_url, ) - enable_appsec_rc(self) self._hooks = _hooks.Hooks() atexit.register(self._atexit) @@ -520,7 +518,6 @@ def _child_after_fork(self): self._single_span_sampling_rules, self._agent_url, ) - enable_appsec_rc(self) self._new_process = True From 61aa2763081e0023718aa73f92ad5563fb688d6e Mon Sep 17 00:00:00 2001 From: Brett Langdon Date: Wed, 21 Dec 2022 12:17:13 -0500 Subject: [PATCH 04/20] chore(ci): remove test_latest workflow (#4792) ## Description Right now the `test_latest` workflow runs on every single PR. We only want to run this workflow either manually or via the nightly job. I tried to get pipeline parameters working to inject the environment variable for `riotfile.py` but it did weird things where it would run a larger set of tests than were being asked for (e.g. running pymysql tests when we were in the bottle suite). Instead of still trying to get that working, just disable entirely running `test_latest` until we can either implement a hold/manual approval to run or actually implement the pipeline parameter approach. Doing this now saves us $$$ on CI costs, when we generally don't need these additional test runs. ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. --- .circleci/config.yml | 4 ---- riotfile.py | 29 +---------------------------- 2 files changed, 1 insertion(+), 32 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 7a0afd32c8c..d0178faa916 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -1314,7 +1314,3 @@ workflows: # - profile-windows-311: *requires_pre_check # Final reports - coverage_report: *requires_tests - - test_latest: - <<: *workflow_test - diff --git a/riotfile.py b/riotfile.py index 2a368975ca4..1542efa6584 100644 --- a/riotfile.py +++ b/riotfile.py @@ -1,12 +1,9 @@ # type: ignore -import json import logging import os import sys from typing import List from typing import Tuple -from urllib.error import HTTPError -from urllib.request import urlopen from riot import Venv from riot import latest as latest_riot @@ -16,32 +13,8 @@ latest = object() # sentinel value -def find_workflow(workflow_id: str) -> str: - """Use CircleCI API to retrieve current workflow name""" - try: - url = f"https://circleci.com/api/v2/workflow/{workflow_id}" - res = urlopen(url) - body = res.read().decode() - return json.loads(body)["name"] - except HTTPError: - logger.error("Error loading workflow information from CircleC: %s", url) - raise - - -# Set up PY_Latest to True if the current workflow is test_latest -PY_Latest = False -if "CIRCLE_WORKFLOW_ID" not in os.environ: - if "DD_USE_LATEST_VERSION" in os.environ: - PY_Latest = True - else: - logger.warning("not in CircleCI. Use fixed dependencies versions.") -else: - # CircleCI has no environment variable for the workflow name and no possibility - # to set it at the workflow level. We use the CircleCI API to do it. - PY_Latest = find_workflow(os.environ["CIRCLE_WORKFLOW_ID"]) == "test_latest" - logger.info("Set latest versions of packages: %s", ["FIXED", "LATEST"][PY_Latest]) - # Import fixed version if needed +PY_Latest = os.environ.get("DD_USE_LATEST_VERSION") == "true" if not PY_Latest: try: sys.path.extend([".", ".circleci"]) From 8d0e0d19fa6582676bd1b768f0d15911c96ada09 Mon Sep 17 00:00:00 2001 From: Harvinder Ghotra Date: Wed, 21 Dec 2022 12:18:58 -0500 Subject: [PATCH 05/20] fix(botocore): keep newlines in json data for kinesis records (#4700) ## Description During our serialization & de-serialization of kinesis records, any `\n` at the end of the record is stripped by the json library. Customer pipelines rely on this `\n` delimiter to distinguish between different records. This change appends the line break to the end of the record string if it was originally there. ## Checklist - [ ] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [ ] Add additional sections for `feat` and `fix` pull requests. - [ ] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Motivation Add support for Databricks which relies on the `\n` delimiter to be present at the end of the record. Look at the related issue for more details. ## Design If the incoming record has a `\n` at the end of the string, we add the `\n` into the string post serialization since it was stripped during the de-serialization process by the json library. ## Testing strategy - two unit tests have been added ``` pytest -k test_kinesis_put_records_newline_base64_trace_injection tests/contrib/botocore/test.py pytest -k test_kinesis_put_records_newline_json_trace_injection tests/contrib/botocore/test.py ``` ## Relevant issue(s) Fixes #4317 ## Testing strategy ## Reviewer Checklist - [x] Title is accurate. - [x] Description motivates each change. - [x] No unnecessary changes were introduced in this PR. - [x] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [x] Tests provided or description of manual testing performed is included in the code or PR. - [x] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [x] All relevant GitHub issues are correctly linked. - [x] Backports are identified and tagged with Mergifyio. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Brett Langdon Co-authored-by: Tahir H. Butt Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> Co-authored-by: Munir Abdinur --- ddtrace/contrib/botocore/patch.py | 25 +++- ...-newline-strip-issue-3a2419b6f29fcab8.yaml | 4 + tests/contrib/botocore/test.py | 118 ++++++++++++++++++ 3 files changed, 143 insertions(+), 4 deletions(-) create mode 100644 releasenotes/notes/fix-kinesis-newline-strip-issue-3a2419b6f29fcab8.yaml diff --git a/ddtrace/contrib/botocore/patch.py b/ddtrace/contrib/botocore/patch.py index d0074db0875..5586f5402db 100644 --- a/ddtrace/contrib/botocore/patch.py +++ b/ddtrace/contrib/botocore/patch.py @@ -10,6 +10,7 @@ from typing import Dict from typing import List from typing import Optional +from typing import Tuple import botocore.client import botocore.exceptions @@ -45,6 +46,8 @@ MAX_KINESIS_DATA_SIZE = 1 << 20 # 1MB MAX_EVENTBRIDGE_DETAIL_SIZE = 1 << 18 # 256KB +LINE_BREAK = "\n" + log = get_logger(__name__) @@ -176,8 +179,17 @@ def inject_trace_to_eventbridge_detail(params, span): entry["Detail"] = detail_json +def get_json_from_str(data_str): + # type: (str) -> Tuple[str, Optional[Dict[str, Any]]] + data_obj = json.loads(data_str) + + if data_str.endswith(LINE_BREAK): + return LINE_BREAK, data_obj + return "", data_obj + + def get_kinesis_data_object(data): - # type: (str) -> Optional[Dict[str, Any]] + # type: (str) -> Tuple[str, Optional[Dict[str, Any]]] """ :data: the data from a kinesis stream @@ -190,13 +202,14 @@ def get_kinesis_data_object(data): # check if data is a json string try: - return json.loads(data) + return get_json_from_str(data) except ValueError: pass # check if data is a base64 encoded json string try: - return json.loads(base64.b64decode(data).decode("ascii")) + data_str = base64.b64decode(data).decode("ascii") + return get_json_from_str(data_str) except ValueError: raise TraceInjectionDecodingError("Unable to parse kinesis streams data string") @@ -216,11 +229,15 @@ def inject_trace_to_kinesis_stream_data(record, span): return data = record["Data"] - data_obj = get_kinesis_data_object(data) + line_break, data_obj = get_kinesis_data_object(data) data_obj["_datadog"] = {} HTTPPropagator.inject(span.context, data_obj["_datadog"]) data_json = json.dumps(data_obj) + # if original string had a line break, add it back + if line_break: + data_json += line_break + # check if data size will exceed max size with headers data_size = len(data_json) if data_size >= MAX_KINESIS_DATA_SIZE: diff --git a/releasenotes/notes/fix-kinesis-newline-strip-issue-3a2419b6f29fcab8.yaml b/releasenotes/notes/fix-kinesis-newline-strip-issue-3a2419b6f29fcab8.yaml new file mode 100644 index 00000000000..db5a811b567 --- /dev/null +++ b/releasenotes/notes/fix-kinesis-newline-strip-issue-3a2419b6f29fcab8.yaml @@ -0,0 +1,4 @@ +--- +fixes: + - | + botocore: Before this change, the botocore integration stripped newlines from the JSON string encoded in the data blob of Amazon Kinesis records. This change includes a terminating newline if it is present in the decoded data. diff --git a/tests/contrib/botocore/test.py b/tests/contrib/botocore/test.py index 41fb67825ec..7210e948f95 100644 --- a/tests/contrib/botocore/test.py +++ b/tests/contrib/botocore/test.py @@ -1795,6 +1795,124 @@ def test_kinesis_put_records_base64_trace_injection(self): client.delete_stream(StreamName=stream_name) + @mock_kinesis + def test_kinesis_put_records_newline_json_trace_injection(self): + client = self.session.create_client("kinesis", region_name="us-east-1") + + stream_name = "test" + client.create_stream(StreamName=stream_name, ShardCount=1) + stream = client.describe_stream(StreamName=stream_name)["StreamDescription"] + shard_id = stream["Shards"][0]["ShardId"] + + partition_key = "1234" + data = [ + {"Data": json.dumps({"Hello": "World"}) + "\n", "PartitionKey": partition_key}, + {"Data": json.dumps({"foo": "bar"}) + "\n", "PartitionKey": partition_key}, + ] + + Pin(service=self.TEST_SERVICE, tracer=self.tracer).onto(client) + client.put_records(StreamName=stream_name, Records=data) + + # check if the appropriate span was generated + spans = self.get_spans() + assert spans + span = spans[0] + assert len(spans) == 1 + assert span.get_tag("aws.region") == "us-east-1" + assert span.get_tag("aws.operation") == "PutRecords" + assert span.get_tag("params.MessageBody") is None + assert_is_measured(span) + assert_span_http_status_code(span, 200) + assert span.service == "test-botocore-tracing.kinesis" + assert span.resource == "kinesis.putrecords" + records = span.get_tag("params.Records") + assert records is None + + resp = client.get_shard_iterator(StreamName=stream_name, ShardId=shard_id, ShardIteratorType="TRIM_HORIZON") + shard_iterator = resp["ShardIterator"] + + # ensure headers are present in received message + resp = client.get_records(ShardIterator=shard_iterator) + assert len(resp["Records"]) == 2 + records = resp["Records"] + record = records[0] + data_str = record["Data"].decode("ascii") + assert data_str.endswith("\n") + data = json.loads(data_str) + headers = data["_datadog"] + assert headers is not None + assert headers[HTTP_HEADER_TRACE_ID] == str(span.trace_id) + assert headers[HTTP_HEADER_PARENT_ID] == str(span.span_id) + + record = records[1] + data_str = record["Data"].decode("ascii") + assert data_str.endswith("\n") + data = json.loads(data_str) + assert "_datadog" not in data + + client.delete_stream(StreamName=stream_name) + + @mock_kinesis + def test_kinesis_put_records_newline_base64_trace_injection(self): + client = self.session.create_client("kinesis", region_name="us-east-1") + + stream_name = "test" + client.create_stream(StreamName=stream_name, ShardCount=1) + stream = client.describe_stream(StreamName=stream_name)["StreamDescription"] + shard_id = stream["Shards"][0]["ShardId"] + + partition_key = "1234" + sample_string = json.dumps({"Hello": "World"}) + "\n" + sample_string_bytes = sample_string.encode("ascii") + base64_bytes = base64.b64encode(sample_string_bytes) + data_str = base64_bytes.decode("ascii") + data = [ + {"Data": data_str, "PartitionKey": partition_key}, + {"Data": data_str, "PartitionKey": partition_key}, + ] + + Pin(service=self.TEST_SERVICE, tracer=self.tracer).onto(client) + client.put_records(StreamName=stream_name, Records=data) + + # check if the appropriate span was generated + spans = self.get_spans() + assert spans + span = spans[0] + assert len(spans) == 1 + assert span.get_tag("aws.region") == "us-east-1" + assert span.get_tag("aws.operation") == "PutRecords" + assert span.get_tag("params.MessageBody") is None + assert_is_measured(span) + assert_span_http_status_code(span, 200) + assert span.service == "test-botocore-tracing.kinesis" + assert span.resource == "kinesis.putrecords" + records = span.get_tag("params.Records") + assert records is None + + resp = client.get_shard_iterator(StreamName=stream_name, ShardId=shard_id, ShardIteratorType="TRIM_HORIZON") + shard_iterator = resp["ShardIterator"] + + # ensure headers are present in received message + resp = client.get_records(ShardIterator=shard_iterator) + assert len(resp["Records"]) == 2 + records = resp["Records"] + record = records[0] + data_str = record["Data"].decode("ascii") + assert data_str.endswith("\n") + data = json.loads(data_str) + headers = data["_datadog"] + assert headers is not None + assert headers[HTTP_HEADER_TRACE_ID] == str(span.trace_id) + assert headers[HTTP_HEADER_PARENT_ID] == str(span.span_id) + + record = records[1] + data_str = base64.b64decode(record["Data"]).decode("ascii") + assert data_str.endswith("\n") + data = json.loads(data_str) + assert "_datadog" not in data + + client.delete_stream(StreamName=stream_name) + @unittest.skipIf(PY2, "Skipping for Python 2.7 since older moto doesn't support secretsmanager") def test_secretsmanager(self): from moto import mock_secretsmanager From ee87f81dde9d14839e8e6146f3ddfae8e8d5c84a Mon Sep 17 00:00:00 2001 From: Brett Langdon Date: Wed, 21 Dec 2022 14:55:03 -0500 Subject: [PATCH 06/20] fix(tests): pin attrs<22.2.0 for Python 3.6 in integration tests (#4816) ## Description `attrs==22.2.0` removed support for Python 3.5 and made Python 3.6 support deprecated. This causes deprecation warnings to be logged and causes our integration tests to fail. This change makes sure we install a version of `attrs<22.2.0` only for these failing integration tests. https://www.attrs.org/en/22.2.0/changelog.html#id1 ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> Co-authored-by: Yun Kim --- riotfile.py | 17 ++++++++++++++++- tests/integration/test_integration.py | 25 +++++++++++++++---------- 2 files changed, 31 insertions(+), 11 deletions(-) diff --git a/riotfile.py b/riotfile.py index 1542efa6584..c7fb8f0be14 100644 --- a/riotfile.py +++ b/riotfile.py @@ -306,7 +306,6 @@ def select_pys(min_version=MIN_PYTHON_VERSION, max_version=MAX_PYTHON_VERSION): ), Venv( name="integration", - pys=select_pys(), command="pytest --no-cov {cmdargs} tests/integration/", pkgs={"msgpack": [latest]}, venvs=[ @@ -315,6 +314,14 @@ def select_pys(min_version=MIN_PYTHON_VERSION, max_version=MAX_PYTHON_VERSION): env={ "AGENT_VERSION": "latest", }, + venvs=[ + Venv(pys=select_pys(max_version="3.5")), + # DEV: attrs marked Python 3.6 as deprecated in 22.2.0, + # this logs a warning and causes these tests to fail + # https://www.attrs.org/en/22.2.0/changelog.html#id1 + Venv(pys=["3.6"], pkgs={"attrs": "<22.2.0"}), + Venv(pys=select_pys(min_version="3.7")), + ], ), Venv( name="integration-snapshot", @@ -322,6 +329,14 @@ def select_pys(min_version=MIN_PYTHON_VERSION, max_version=MAX_PYTHON_VERSION): "DD_TRACE_AGENT_URL": "http://localhost:9126", "AGENT_VERSION": "testagent", }, + venvs=[ + Venv(pys=select_pys(max_version="3.5")), + # DEV: attrs marked Python 3.6 as deprecated in 22.2.0, + # this logs a warning and causes these tests to fail + # https://www.attrs.org/en/22.2.0/changelog.html#id1 + Venv(pys=["3.6"], pkgs={"attrs": "<22.2.0"}), + Venv(pys=select_pys(min_version="3.7")), + ], ), ], ), diff --git a/tests/integration/test_integration.py b/tests/integration/test_integration.py index 3bbf47b5377..7d9949f7a7e 100644 --- a/tests/integration/test_integration.py +++ b/tests/integration/test_integration.py @@ -56,9 +56,12 @@ def test_debug_mode(): assert p.stdout.read() == b"" assert b"DEBUG:ddtrace" not in p.stderr.read() + env = os.environ.copy() + env.update({"DD_TRACE_DEBUG": "true", "DD_CALL_BASIC_CONFIG": "true"}) + p = subprocess.Popen( [sys.executable, "-c", "import ddtrace"], - env=dict(DD_TRACE_DEBUG="true", DD_CALL_BASIC_CONFIG="true"), + env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) @@ -670,16 +673,18 @@ def test_regression_logging_in_context(tmpdir, logs_injection, debug_mode, patch """.lstrip() % str(patch_logging) ) + + env = os.environ.copy() + env.update( + { + "DD_TRACE_LOGS_INJECTION": str(logs_injection).lower(), + "DD_TRACE_DEBUG": str(debug_mode).lower(), + "DD_CALL_BASIC_CONFIG": "true", + } + ) + p = subprocess.Popen( - [sys.executable, "test.py"], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - cwd=str(tmpdir), - env=dict( - DD_TRACE_LOGS_INJECTION=str(logs_injection).lower(), - DD_TRACE_DEBUG=str(debug_mode).lower(), - DD_CALL_BASIC_CONFIG="true", - ), + [sys.executable, "test.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=str(tmpdir), env=env ) try: p.wait(timeout=2) From 07a267fc917768addbd1e600ead1915180c65fc2 Mon Sep 17 00:00:00 2001 From: Nikolay Martynov Date: Thu, 22 Dec 2022 05:59:10 -0500 Subject: [PATCH 07/20] feat(profiling): support v2.4 profile uploads (#4774) ## Description Update profiler to use new version of backend API to upload profiles. ## Checklist - [ ] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [ ] Add additional sections for `feat` and `fix` pull requests. - [ ] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Motivation Using new BE API would allow us to implement new features in the future. ## Design ## Testing strategy Make sure that profiles are uploaded and all profile information (FG, tags) is correct. ## Reviewer Checklist - [x] Title is accurate. - [x] Description motivates each change. - [x] No unnecessary changes were introduced in this PR. - [x] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [x] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. Co-authored-by: Brett Langdon Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> --- ddtrace/profiling/exporter/http.py | 111 ++++---- ddtrace/profiling/profiler.py | 6 +- .../profile-upload-2.4-8277b229fda6ff53.yaml | 3 + tests/profiling/exporter/test_http.py | 241 +++++++++--------- tests/profiling/test_profiler.py | 8 +- 5 files changed, 181 insertions(+), 188 deletions(-) create mode 100755 releasenotes/notes/profile-upload-2.4-8277b229fda6ff53.yaml diff --git a/ddtrace/profiling/exporter/http.py b/ddtrace/profiling/exporter/http.py index 59af75d11df..484054a38a6 100644 --- a/ddtrace/profiling/exporter/http.py +++ b/ddtrace/profiling/exporter/http.py @@ -25,8 +25,8 @@ HOSTNAME = platform.node() -PYTHON_IMPLEMENTATION = platform.python_implementation().encode() -PYTHON_VERSION = platform.python_version().encode() +PYTHON_IMPLEMENTATION = platform.python_implementation() +PYTHON_VERSION = platform.python_version() class UploadFailed(tenacity.RetryError, exporter.ExportError): @@ -54,7 +54,7 @@ class PprofHTTPExporter(pprof.PprofExporter): service = attr.ib(default=None, type=typing.Optional[str]) env = attr.ib(default=None, type=typing.Optional[str]) version = attr.ib(default=None, type=typing.Optional[str]) - tags = attr.ib(factory=dict, type=typing.Dict[str, bytes]) + tags = attr.ib(factory=dict, type=typing.Dict[str, str]) max_retry_delay = attr.ib(default=None) _container_info = attr.ib(factory=container.get_container_info, repr=False) _retry_upload = attr.ib(init=False, eq=False) @@ -71,67 +71,54 @@ def __attrs_post_init__(self): retry=tenacity.retry_if_exception_type((http_client.HTTPException, OSError, IOError)), ) tags = { - k: six.ensure_binary(v) + k: six.ensure_str(v, "utf-8") for k, v in itertools.chain( parse_tags_str(os.environ.get("DD_TAGS")).items(), parse_tags_str(os.environ.get("DD_PROFILING_TAGS")).items(), ) } - tags.update({k: six.ensure_binary(v) for k, v in self.tags.items()}) + tags.update(self.tags) tags.update( { - "host": HOSTNAME.encode("utf-8"), - "language": b"python", + "host": HOSTNAME, + "language": "python", "runtime": PYTHON_IMPLEMENTATION, "runtime_version": PYTHON_VERSION, - "profiler_version": ddtrace.__version__.encode("ascii"), + "profiler_version": ddtrace.__version__, } ) if self.version: - tags["version"] = self.version.encode("utf-8") + tags["version"] = self.version if self.env: - tags["env"] = self.env.encode("utf-8") + tags["env"] = self.env self.tags = tags @staticmethod def _encode_multipart_formdata( - fields, # type: typing.Dict[str, bytes] - tags, # type: typing.Dict[str, bytes] - data, # type: typing.Dict[bytes, bytes] + event, # type: bytes + data, # type: typing.List[typing.Dict[str, bytes]] ): # type: (...) -> typing.Tuple[bytes, bytes] boundary = binascii.hexlify(os.urandom(16)) # The body that is generated is very sensitive and must perfectly match what the server expects. body = ( - b"".join( - b"--%s\r\n" - b'Content-Disposition: form-data; name="%s"\r\n' - b"\r\n" - b"%s\r\n" % (boundary, field.encode(), value) - for field, value in fields.items() - if field != "chunk-data" - ) - + b"".join( - b"--%s\r\n" - b'Content-Disposition: form-data; name="tags[]"\r\n' - b"\r\n" - b"%s:%s\r\n" % (boundary, tag.encode(), value) - for tag, value in tags.items() - ) + (b"--%s\r\n" % boundary) + + b'Content-Disposition: form-data; name="event"; filename="event.json"\r\n' + + b"Content-Type: application/json\r\n\r\n" + + event + + b"\r\n" + b"".join( - ( - b'--%s\r\nContent-Disposition: form-data; name="data[%s]"; filename="%s"\r\n' - % (boundary, field_name, field_name) - ) - + b"Content-Type: application/octet-stream\r\n\r\n" - + field_data + (b"--%s\r\n" % boundary) + + (b'Content-Disposition: form-data; name="%s"; filename="%s"\r\n' % (item["name"], item["filename"])) + + (b"Content-Type: %s\r\n\r\n" % (item["content-type"])) + + item["data"] + b"\r\n" - for field_name, field_data in data.items() + for item in data ) - + b"--%s--" % boundary + + b"--%s--\r\n" % boundary ) content_type = b"multipart/form-data; boundary=%s" % boundary @@ -141,15 +128,15 @@ def _encode_multipart_formdata( def _get_tags( self, service # type: str ): - # type: (...) -> typing.Dict[str, bytes] + # type: (...) -> str tags = { - "service": service.encode("utf-8"), - "runtime-id": runtime.get_runtime_id().encode("ascii"), + "service": service, + "runtime-id": runtime.get_runtime_id(), } tags.update(self.tags) - return tags + return ",".join(tag + ":" + value for tag, value in tags.items()) def export( self, @@ -178,21 +165,15 @@ def export( pprof = six.BytesIO() with gzip.GzipFile(fileobj=pprof, mode="wb") as gz: gz.write(profile.SerializeToString()) - fields = { - "version": b"3", - "family": b"python", - "runtime-id": runtime.get_runtime_id().encode("ascii"), - "start": ( - datetime.datetime.utcfromtimestamp(start_time_ns / 1e9).replace(microsecond=0).isoformat() + "Z" - ).encode(), - "end": ( - datetime.datetime.utcfromtimestamp(end_time_ns / 1e9).replace(microsecond=0).isoformat() + "Z" - ).encode(), - } - - service = self.service or os.path.basename(profile.string_table[profile.mapping[0].filename]) - data = {b"auto.pprof": pprof.getvalue()} + data = [ + { + "name": b"auto", + "filename": b"auto.pprof", + "content-type": b"application/octet-stream", + "data": pprof.getvalue(), + } + ] if self.enable_code_provenance: code_provenance = six.BytesIO() @@ -204,11 +185,27 @@ def export( } ).encode("utf-8") ) - data[b"code-provenance.json"] = code_provenance.getvalue() + data.append( + { + "name": b"code-provenance", + "filename": b"code-provenance.json", + "content-type": b"application/json", + "data": code_provenance.getvalue(), + } + ) + + service = self.service or os.path.basename(profile.string_table[profile.mapping[0].filename]) + event = { + "version": "4", + "family": "python", + "attachments": [item["filename"].decode("utf-8") for item in data], + "tags_profiler": self._get_tags(service), + "start": (datetime.datetime.utcfromtimestamp(start_time_ns / 1e9).replace(microsecond=0).isoformat() + "Z"), + "end": (datetime.datetime.utcfromtimestamp(end_time_ns / 1e9).replace(microsecond=0).isoformat() + "Z"), + } content_type, body = self._encode_multipart_formdata( - fields, - tags=self._get_tags(service), + event=json.dumps(event).encode("utf-8"), data=data, ) headers["Content-Type"] = content_type diff --git a/ddtrace/profiling/profiler.py b/ddtrace/profiling/profiler.py index 2ba5f2e113c..8d832624d66 100644 --- a/ddtrace/profiling/profiler.py +++ b/ddtrace/profiling/profiler.py @@ -109,7 +109,7 @@ class _ProfilerInstance(service.Service): # User-supplied values url = attr.ib(default=None) service = attr.ib(factory=lambda: os.environ.get("DD_SERVICE")) - tags = attr.ib(factory=dict, type=typing.Dict[str, bytes]) + tags = attr.ib(factory=dict, type=typing.Dict[str, str]) env = attr.ib(factory=lambda: os.environ.get("DD_ENV")) version = attr.ib(factory=lambda: os.environ.get("DD_VERSION")) tracer = attr.ib(default=ddtrace.tracer) @@ -160,7 +160,7 @@ def _build_default_exporters(self): endpoint = agent.get_trace_url() if self.agentless: - endpoint_path = "/v1/input" + endpoint_path = "/api/v2/profile" else: # Agent mode # path is relative because it is appended @@ -168,7 +168,7 @@ def _build_default_exporters(self): endpoint_path = "profiling/v1/input" if self._lambda_function_name is not None: - self.tags.update({"functionname": self._lambda_function_name.encode("utf-8")}) + self.tags.update({"functionname": self._lambda_function_name}) return [ http.PprofHTTPExporter( diff --git a/releasenotes/notes/profile-upload-2.4-8277b229fda6ff53.yaml b/releasenotes/notes/profile-upload-2.4-8277b229fda6ff53.yaml new file mode 100755 index 00000000000..669d65e953c --- /dev/null +++ b/releasenotes/notes/profile-upload-2.4-8277b229fda6ff53.yaml @@ -0,0 +1,3 @@ +upgrade: + - | + profiling: upgrades the profiler to support the ``v2.4`` backend API for profile uploads, using a new request format. diff --git a/tests/profiling/exporter/test_http.py b/tests/profiling/exporter/test_http.py index fa92e112e5a..328bcca00f0 100644 --- a/tests/profiling/exporter/test_http.py +++ b/tests/profiling/exporter/test_http.py @@ -1,6 +1,7 @@ # -*- encoding: utf-8 -*- import collections import email.parser +import json import platform import socket import sys @@ -14,6 +15,7 @@ import ddtrace from ddtrace.internal import compat +from ddtrace.internal.utils.formats import parse_tags_str from ddtrace.profiling import exporter from ddtrace.profiling.exporter import http @@ -41,18 +43,31 @@ def log_message(format, *args): # noqa: A002 @staticmethod def _check_tags(tags): - tags.sort() + tags.split(",").sort() return ( len(tags) == 6 - and tags[0].startswith(b"host:") - and tags[1] == b"language:python" - and tags[2] == ("profiler_version:%s" % ddtrace.__version__).encode("utf-8") - and tags[3].startswith(b"runtime-id:") - and tags[4] == b"runtime:CPython" - and tags[5].startswith(b"service:") - and tags[6] == platform.python_version().encode(), + and tags[0].startswith("host:") + and tags[1] == "language:python" + and tags[2] == ("profiler_version:%s" % ddtrace.__version__) + and tags[3].startswith("runtime-id:") + and tags[4] == "runtime:CPython" + and tags[5].startswith("service:") + and tags[6] == platform.python_version(), ) + def _check_event(self, event_json): + event = json.loads(event_json.decode()) + for key, check in { + "start": lambda x: x == "1970-01-01T00:00:00Z", + "end": lambda x: x.startswith("20"), + "family": lambda x: x == "python", + "version": lambda x: x == "4", + "tags_profiler": self._check_tags, + }.items(): + if not check(event[key]): + return False + return True + def do_POST(self): assert self.path.startswith(self.path_prefix) api_key = self.headers["DD-API-KEY"] @@ -73,17 +88,14 @@ def do_POST(self): for part in msg.get_payload(): items[part.get_param("name", header="content-disposition")].append(part.get_payload(decode=True)) for key, check in { - "start": lambda x: x[0] == b"1970-01-01T00:00:00Z", - "end": lambda x: x[0].startswith(b"20"), - "family": lambda x: x[0] == b"python", - "version": lambda x: x[0] == b"3", - "tags[]": self._check_tags, - "data[auto.pprof]": lambda x: x[0].startswith(b"\x1f\x8b\x08\x00"), - "data[code-provenance.json]": lambda x: x[0].startswith(b"\x1f\x8b\x08\x00"), + "event": lambda x: self._check_event(x[0]), + "auto": lambda x: x[0].startswith(b"\x1f\x8b\x08\x00"), + "code-provenance": lambda x: x[0].startswith(b"\x1f\x8b\x08\x00"), }.items(): if not check(items[key]): self.send_error(400, "Wrong value for %s: %r" % (key, items[key])) return + self.send_error(200, "OK") @@ -238,191 +250,172 @@ def test_export_tracer_base_path_agent_less(endpoint_test_server): exp.export(test_pprof.TEST_EVENTS, 0, compat.time_ns()) -def _check_tags_types(tags): - for k, v in tags.items(): - assert isinstance(k, str) - assert isinstance(v, bytes) - - def test_get_tags(): - tags = http.PprofHTTPExporter(env="foobar", endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(env="foobar", endpoint="")._get_tags("foobar")) assert len(tags) == 8 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["env"] == b"foobar" - assert tags["runtime"] == b"CPython" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["env"] == "foobar" + assert tags["runtime"] == "CPython" + assert tags["profiler_version"] == ddtrace.__version__ assert "version" not in tags def test_get_malformed(monkeypatch): monkeypatch.setenv("DD_TAGS", "mytagfoobar") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 7 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["profiler_version"] == ddtrace.__version__ monkeypatch.setenv("DD_TAGS", "mytagfoobar,") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 7 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["profiler_version"] == ddtrace.__version__ monkeypatch.setenv("DD_TAGS", ",") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 7 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["profiler_version"] == ddtrace.__version__ monkeypatch.setenv("DD_TAGS", "foo:bar,") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 8 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["foo"] == b"bar" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["foo"] == "bar" + assert tags["profiler_version"] == ddtrace.__version__ def test_get_tags_override(monkeypatch): monkeypatch.setenv("DD_TAGS", "mytag:foobar") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 8 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["mytag"] == b"foobar" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["mytag"] == "foobar" + assert tags["profiler_version"] == ddtrace.__version__ assert "version" not in tags monkeypatch.setenv("DD_TAGS", "mytag:foobar,author:jd") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 9 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["mytag"] == b"foobar" - assert tags["author"] == b"jd" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["mytag"] == "foobar" + assert tags["author"] == "jd" + assert tags["profiler_version"] == ddtrace.__version__ assert "version" not in tags monkeypatch.setenv("DD_TAGS", "") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 7 - assert tags["service"] == b"foobar" + assert tags["service"] == "foobar" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["profiler_version"] == ddtrace.__version__ assert "version" not in tags monkeypatch.setenv("DD_TAGS", "foobar:baz,service:mycustomservice") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 8 - assert tags["service"] == b"mycustomservice" + assert tags["service"] == "mycustomservice" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["foobar"] == b"baz" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["foobar"] == "baz" + assert tags["profiler_version"] == ddtrace.__version__ assert "version" not in tags monkeypatch.setenv("DD_TAGS", "foobar:baz,service:🤣") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) assert len(tags) == 8 - assert tags["service"] == u"🤣".encode("utf-8") + assert tags["service"] == "🤣" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["foobar"] == b"baz" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["foobar"] == "baz" + assert tags["profiler_version"] == ddtrace.__version__ assert "version" not in tags - tags = http.PprofHTTPExporter(endpoint="", version="123")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="", version="123")._get_tags("foobar")) assert len(tags) == 9 - assert tags["service"] == u"🤣".encode("utf-8") + assert tags["service"] == "🤣" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["foobar"] == b"baz" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") - assert tags["version"] == b"123" + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["foobar"] == "baz" + assert tags["profiler_version"] == ddtrace.__version__ + assert tags["version"] == "123" assert "env" not in tags - tags = http.PprofHTTPExporter(endpoint="", version="123", env="prod")._get_tags("foobar") - _check_tags_types(tags) + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="", version="123", env="prod")._get_tags("foobar")) assert len(tags) == 10 - assert tags["service"] == u"🤣".encode("utf-8") + assert tags["service"] == "🤣" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["foobar"] == b"baz" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") - assert tags["version"] == b"123" - assert tags["env"] == b"prod" - - tags = http.PprofHTTPExporter(endpoint="", version="123", env="prod", tags={"mytag": "123"})._get_tags("foobar") - _check_tags_types(tags) + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["foobar"] == "baz" + assert tags["profiler_version"] == ddtrace.__version__ + assert tags["version"] == "123" + assert tags["env"] == "prod" + + tags = parse_tags_str( + http.PprofHTTPExporter(endpoint="", version="123", env="prod", tags={"mytag": "123"})._get_tags("foobar") + ) assert len(tags) == 11 - assert tags["service"] == u"🤣".encode("utf-8") + assert tags["service"] == "🤣" assert len(tags["host"]) assert len(tags["runtime-id"]) - assert tags["language"] == b"python" - assert tags["runtime"] == b"CPython" - assert tags["foobar"] == b"baz" - assert tags["profiler_version"] == ddtrace.__version__.encode("utf-8") - assert tags["version"] == b"123" - assert tags["env"] == b"prod" - assert tags["mytag"] == b"123" + assert tags["language"] == "python" + assert tags["runtime"] == "CPython" + assert tags["foobar"] == "baz" + assert tags["profiler_version"] == ddtrace.__version__ + assert tags["version"] == "123" + assert tags["env"] == "prod" + assert tags["mytag"] == "123" def test_get_tags_legacy(monkeypatch): monkeypatch.setenv("DD_PROFILING_TAGS", "mytag:baz") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) - assert tags["mytag"] == b"baz" + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) + assert tags["mytag"] == "baz" # precedence monkeypatch.setenv("DD_TAGS", "mytag:val1,ddtag:hi") monkeypatch.setenv("DD_PROFILING_TAGS", "mytag:val2,ddptag:lo") - tags = http.PprofHTTPExporter(endpoint="")._get_tags("foobar") - _check_tags_types(tags) - assert tags["mytag"] == b"val2" - assert tags["ddtag"] == b"hi" - assert tags["ddptag"] == b"lo" + tags = parse_tags_str(http.PprofHTTPExporter(endpoint="")._get_tags("foobar")) + assert tags["mytag"] == "val2" + assert tags["ddtag"] == "hi" + assert tags["ddptag"] == "lo" diff --git a/tests/profiling/test_profiler.py b/tests/profiling/test_profiler.py index 36787eda9af..68413fe73c8 100644 --- a/tests/profiling/test_profiler.py +++ b/tests/profiling/test_profiler.py @@ -151,7 +151,7 @@ def test_tags_api(): if isinstance(exp, http.PprofHTTPExporter): assert exp.env == "staging" assert exp.version == "123" - assert exp.tags["foo"] == b"bar" + assert exp.tags["foo"] == "bar" break else: pytest.fail("Unable to find HTTP exporter") @@ -184,7 +184,7 @@ def test_env_agentless(monkeypatch): monkeypatch.setenv("DD_PROFILING_AGENTLESS", "true") monkeypatch.setenv("DD_API_KEY", "foobar") prof = profiler.Profiler() - _check_url(prof, "https://intake.profile.datadoghq.com", "foobar", endpoint_path="/v1/input") + _check_url(prof, "https://intake.profile.datadoghq.com", "foobar", endpoint_path="/api/v2/profile") def test_env_agentless_site(monkeypatch): @@ -192,7 +192,7 @@ def test_env_agentless_site(monkeypatch): monkeypatch.setenv("DD_PROFILING_AGENTLESS", "true") monkeypatch.setenv("DD_API_KEY", "foobar") prof = profiler.Profiler() - _check_url(prof, "https://intake.profile.datadoghq.eu", "foobar", endpoint_path="/v1/input") + _check_url(prof, "https://intake.profile.datadoghq.eu", "foobar", endpoint_path="/api/v2/profile") def test_env_no_agentless(monkeypatch): @@ -387,4 +387,4 @@ def test_profiler_serverless(monkeypatch): monkeypatch.setenv("AWS_LAMBDA_FUNCTION_NAME", "foobar") p = profiler.Profiler() assert isinstance(p._scheduler, scheduler.ServerlessScheduler) - assert p.tags["functionname"] == b"foobar" + assert p.tags["functionname"] == "foobar" From 4b9b59e84f945575f85d7d4a1a9820c58078d40d Mon Sep 17 00:00:00 2001 From: Federico Mon Date: Thu, 22 Dec 2022 12:45:02 +0000 Subject: [PATCH 08/20] chore(asm): change default value for remote config env var (#4813) ## Description Change the value for DD_REMOTE_CONFIGURATION_ENABLED to True by default, so AppSec 1-click-activation feature is enabled by default, unless this environment variable is disabled. ## Checklist - [ ] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [ ] Add additional sections for `feat` and `fix` pull requests. - [ ] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. --- ddtrace/appsec/utils.py | 2 +- ...click-activation-with-RCM-by-default-92233ac7f60292e0.yaml | 4 ++++ tests/appsec/test_remoteconfiguration.py | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) create mode 100644 releasenotes/notes/asm-1-click-activation-with-RCM-by-default-92233ac7f60292e0.yaml diff --git a/ddtrace/appsec/utils.py b/ddtrace/appsec/utils.py index 38a009ff15b..9c33224f5b6 100644 --- a/ddtrace/appsec/utils.py +++ b/ddtrace/appsec/utils.py @@ -6,7 +6,7 @@ def _appsec_rc_features_is_enabled(): # type: () -> bool - if asbool(os.environ.get("DD_REMOTE_CONFIGURATION_ENABLED", "false")): + if asbool(os.environ.get("DD_REMOTE_CONFIGURATION_ENABLED", "true")): return APPSEC_ENV not in os.environ return False diff --git a/releasenotes/notes/asm-1-click-activation-with-RCM-by-default-92233ac7f60292e0.yaml b/releasenotes/notes/asm-1-click-activation-with-RCM-by-default-92233ac7f60292e0.yaml new file mode 100644 index 00000000000..1978aac3910 --- /dev/null +++ b/releasenotes/notes/asm-1-click-activation-with-RCM-by-default-92233ac7f60292e0.yaml @@ -0,0 +1,4 @@ +--- +features: + - | + ASM: one click activation enabled by default using Remote Configuration Management (RCM). Set ``DD_REMOTE_CONFIGURATION_ENABLED=false`` to disable this feature. diff --git a/tests/appsec/test_remoteconfiguration.py b/tests/appsec/test_remoteconfiguration.py index 485567f8b4d..9ed94620f2a 100644 --- a/tests/appsec/test_remoteconfiguration.py +++ b/tests/appsec/test_remoteconfiguration.py @@ -26,7 +26,7 @@ def _set_and_get_appsec_tags(tracer): def test_rc_enabled_by_default(tracer): result = _set_and_get_appsec_tags(tracer) assert result is None - assert not _appsec_rc_features_is_enabled() + assert _appsec_rc_features_is_enabled() def test_rc_activate_is_active_and_get_processor_tags(tracer): From 0fbb60c6d42a12a55a871ec89c4b26d5c66ef3cc Mon Sep 17 00:00:00 2001 From: Brett Langdon Date: Thu, 22 Dec 2022 10:32:35 -0500 Subject: [PATCH 09/20] fix(internal): catch possible OSError when calling platform.libc_ver() on an unsupported machine (#4828) ## Description When calling `platform.libc_ver()` on an unsupported system it can raise an `OSError`. This change adds exception handling for `OSError` and regression test. ## Reviewer Checklist - [x] Title is accurate. - [x] Description motivates each change. - [x] No unnecessary changes were introduced in this PR. - [x] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [x] Tests provided or description of manual testing performed is included in the code or PR. - [x] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [x] All relevant GitHub issues are correctly linked. - [x] Backports are identified and tagged with Mergifyio. --- ddtrace/internal/telemetry/data.py | 25 +++++++++++--- ...atform-libc-ver-call-8872bde91bcc8765.yaml | 4 +++ tests/telemetry/test_data.py | 34 ++++++++++++++++--- 3 files changed, 55 insertions(+), 8 deletions(-) create mode 100644 releasenotes/notes/fix-telemetry-platform-libc-ver-call-8872bde91bcc8765.yaml diff --git a/ddtrace/internal/telemetry/data.py b/ddtrace/internal/telemetry/data.py index 00f66c9dbb6..249766e1356 100644 --- a/ddtrace/internal/telemetry/data.py +++ b/ddtrace/internal/telemetry/data.py @@ -33,11 +33,28 @@ def _get_container_id(): def _get_os_version(): # type: () -> str """Returns the os version for applications running on Unix, Mac or Windows 32-bit""" - mver, _, _ = platform.mac_ver() - _, wver, _, _ = platform.win32_ver() - _, lver = platform.libc_ver() + try: + mver, _, _ = platform.mac_ver() + if mver: + return mver + + _, wver, _, _ = platform.win32_ver() + if wver: + return wver + + # This is the call which is more likely to fail + # + # https://docs.python.org/3/library/platform.html#unix-platforms + # Note that this function has intimate knowledge of how different libc versions add symbols + # to the executable is probably only usable for executables compiled using gcc. + _, lver = platform.libc_ver() + if lver: + return lver + except OSError: + # We were unable to lookup the proper version + pass - return mver or wver or lver or "" + return "" @cached() diff --git a/releasenotes/notes/fix-telemetry-platform-libc-ver-call-8872bde91bcc8765.yaml b/releasenotes/notes/fix-telemetry-platform-libc-ver-call-8872bde91bcc8765.yaml new file mode 100644 index 00000000000..59923209a2d --- /dev/null +++ b/releasenotes/notes/fix-telemetry-platform-libc-ver-call-8872bde91bcc8765.yaml @@ -0,0 +1,4 @@ +--- +fixes: + - | + telemetry: This fix resolves an issue when we try to fetch ``platform.libc_ver()`` on an unsupported system. diff --git a/tests/telemetry/test_data.py b/tests/telemetry/test_data.py index 1afc69d654d..06f3bddff07 100644 --- a/tests/telemetry/test_data.py +++ b/tests/telemetry/test_data.py @@ -108,10 +108,10 @@ def test_get_host_info(): @pytest.mark.parametrize( "mac_ver,win32_ver,libc_ver,expected", [ - ((None, None, None), (None, "4.1.6", None, None), (None, None), "4.1.6"), - (("3.5.6", None, None), (None, "", None, None), (None, None), "3.5.6"), - ((None, None, None), (None, None, None, None), (None, "1.2.7"), "1.2.7"), - ((None, None, None), (None, None, None, None), (None, None), ""), + (("", "", ""), ("", "4.1.6", "", ""), ("", ""), "4.1.6"), + (("3.5.6", "", ""), ("", "", "", ""), ("", ""), "3.5.6"), + (("", "", ""), ("", "", "", ""), ("", "1.2.7"), "1.2.7"), + (("", "", ""), ("", "", "", ""), ("", ""), ""), ], ) def test_get_os_version(mac_ver, win32_ver, libc_ver, expected): @@ -125,6 +125,32 @@ def test_get_os_version(mac_ver, win32_ver, libc_ver, expected): assert _get_os_version() == expected +@pytest.mark.parametrize( + "mac_ver,win32_ver,expected", + [ + # We are on a unix system that raised an OSError + (("", "", ""), ("", "", "", ""), ""), + # We are on macOS, we never call platform.libc_ver(), so success + (("3.5.6", "", ""), ("", "", "", ""), "3.5.6"), + # We are on a Windows machine, we never call platform.libc_ver(), so success + (("", "", ""), ("", "4.1.6", "", ""), "4.1.6"), + ], +) +def test_get_os_version_os_error(mac_ver, win32_ver, expected): + """regression test for platform.libc_ver() raising an OSError on Windows/unsupported machine""" + with mock.patch("platform.mac_ver") as macos: + macos.return_value = mac_ver + + with mock.patch("platform.win32_ver") as win32: + win32.return_value = win32_ver + + # Cause platform.libc_ver() to raise an OSError + with mock.patch("platform.libc_ver") as libc: + libc.side_effect = OSError + + assert _get_os_version() == expected + + def test_get_container_id_when_container_exists(): """ validates the return value of _get_container_id when get_container_info() From 6ecad6d924af108d7348891df7637f52367fb967 Mon Sep 17 00:00:00 2001 From: Phil Chen Date: Thu, 22 Dec 2022 08:06:58 -0800 Subject: [PATCH 10/20] feat(agent): add IPv6 support for hostnames (#4740) ## Description Adds support to for using IPv6 hostnames like "2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF" by adding brackets around them so they can be parsed properly by `urllib`. ## Checklist - [x] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [x] Add additional sections for `feat` and `fix` pull requests. [N/A] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Relevant issue(s) https://github.com/DataDog/dd-trace-py/issues/4739 ## Testing strategy Tested that setting the environment variable `DD_AGENT_HOST` to IPv6 addresses results in brackets around the addresses and that such URLs can be properly verified. ## Reviewer Checklist - [x] Title is accurate. - [x] Description motivates each change. - [x] No unnecessary changes were introduced in this PR. - [x] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [x] Tests provided or description of manual testing performed is included in the code or PR. - [x] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [x] All relevant GitHub issues are correctly linked. - [x] Backports are identified and tagged with Mergifyio. Co-authored-by: Kyle Verhoog --- ddtrace/internal/agent.py | 19 +++++++++++++-- docs/configuration.rst | 18 +++++++++++++++ docs/spelling_wordlist.txt | 1 + ...upport-for-hostnames-0fb700a1ef7ed57e.yaml | 4 ++++ tests/tracer/test_agent.py | 23 +++++++++++++++++++ 5 files changed, 63 insertions(+), 2 deletions(-) create mode 100644 releasenotes/notes/add-ipv6-support-for-hostnames-0fb700a1ef7ed57e.yaml diff --git a/ddtrace/internal/agent.py b/ddtrace/internal/agent.py index a366249a25c..b97d0b8c15e 100644 --- a/ddtrace/internal/agent.py +++ b/ddtrace/internal/agent.py @@ -1,4 +1,5 @@ import os +import socket from typing import TypeVar from typing import Union @@ -22,14 +23,28 @@ T = TypeVar("T") +# This method returns if a hostname is an IPv6 address +def is_ipv6_hostname(hostname): + # type: (Union[T, str]) -> bool + if not isinstance(hostname, str): + return False + try: + socket.inet_pton(socket.AF_INET6, hostname) + return True + except socket.error: # not a valid address + return False + + def get_trace_hostname(default=DEFAULT_HOSTNAME): # type: (Union[T, str]) -> Union[T, str] - return os.environ.get("DD_AGENT_HOST", os.environ.get("DD_TRACE_AGENT_HOSTNAME", default)) + hostname = os.environ.get("DD_AGENT_HOST", os.environ.get("DD_TRACE_AGENT_HOSTNAME", default)) + return "[{}]".format(hostname) if is_ipv6_hostname(hostname) else hostname def get_stats_hostname(default=DEFAULT_HOSTNAME): # type: (Union[T, str]) -> Union[T, str] - return os.environ.get("DD_AGENT_HOST", os.environ.get("DD_DOGSTATSD_HOST", default)) + hostname = os.environ.get("DD_AGENT_HOST", os.environ.get("DD_DOGSTATSD_HOST", default)) + return "[{}]".format(hostname) if is_ipv6_hostname(hostname) else hostname def get_trace_port(default=DEFAULT_TRACE_PORT): diff --git a/docs/configuration.rst b/docs/configuration.rst index db9f1609150..246d70af223 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -121,6 +121,24 @@ below: default: False description: Controls whether ``logging.basicConfig`` is called in ``ddtrace-run`` or when debug mode is enabled. + DD_AGENT_HOST: + type: String + default: | + ``localhost`` + description: | + The host name to use to connect the Datadog agent for traces. The host name + can be IPv4, IPv6, or a domain name. If ``DD_TRACE_AGENT_URL`` is specified, the + value of ``DD_AGENT_HOST`` is ignored. + + Example for IPv4: ``DD_AGENT_HOST=192.168.10.1`` + + Example for IPv6: ``DD_AGENT_HOST=2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF`` + + Example for domain name: ``DD_AGENT_HOST=host`` + version_added: + v0.17.0: + v1.7.0: + DD_TRACE_AGENT_URL: type: URL default: | diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 7d412f3f7af..99ea798e416 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -88,6 +88,7 @@ greenlets grpc gunicorn hostname +hostnames http httplib https diff --git a/releasenotes/notes/add-ipv6-support-for-hostnames-0fb700a1ef7ed57e.yaml b/releasenotes/notes/add-ipv6-support-for-hostnames-0fb700a1ef7ed57e.yaml new file mode 100644 index 00000000000..f7ee6f4b107 --- /dev/null +++ b/releasenotes/notes/add-ipv6-support-for-hostnames-0fb700a1ef7ed57e.yaml @@ -0,0 +1,4 @@ +--- +features: + - | + tracing: Adds support for IPv6 agent hostnames for `DD_AGENT_HOST`. diff --git a/tests/tracer/test_agent.py b/tests/tracer/test_agent.py index 8fed70f1fad..fce909f420d 100644 --- a/tests/tracer/test_agent.py +++ b/tests/tracer/test_agent.py @@ -3,16 +3,37 @@ from ddtrace.internal import agent +@pytest.mark.parametrize( + "hostname,expected", + [ + (None, False), + ("10.0.0.1", False), + ("192.168.1.1", False), + ("https://www.datadog.com", False), + ("2001:db8:3333:4444:5555:6666:7777:8888", True), + ("2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF", True), + ("[2001:db8:3333:4444:5555:6666:7777:8888]", False), + ("::", True), + ], +) +def test_is_ipv6_hostname(hostname, expected): + assert agent.is_ipv6_hostname(hostname) == expected + + def test_trace_hostname(monkeypatch): assert agent.get_trace_hostname() == "localhost" monkeypatch.setenv("DD_AGENT_HOST", "host") assert agent.get_trace_hostname() == "host" + monkeypatch.setenv("DD_AGENT_HOST", "2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF") + assert agent.get_trace_hostname() == "[2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF]" def test_stats_hostname(monkeypatch): assert agent.get_stats_hostname() == "localhost" monkeypatch.setenv("DD_AGENT_HOST", "host") assert agent.get_stats_hostname() == "host" + monkeypatch.setenv("DD_AGENT_HOST", "2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF") + assert agent.get_stats_hostname() == "[2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF]" def test_trace_port(monkeypatch): @@ -193,6 +214,8 @@ def test_verify_url(): agent.verify_url("https://localhost:1234") agent.verify_url("https://localhost") agent.verify_url("http://192.168.1.1") + agent.verify_url("http://[2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF]") + agent.verify_url("http://[2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF]:1234") agent.verify_url("unix:///file.sock") agent.verify_url("unix:///file") From 5b8e5c8d5f9a8fef590162cf4a8c6c72fb3974bf Mon Sep 17 00:00:00 2001 From: "Gabriele N. Tornetta" Date: Thu, 22 Dec 2022 19:11:45 +0000 Subject: [PATCH 11/20] docs(debugging): add dynamic instrumentation API (#4236) ## Description This change adds the dynamic instrumentation public API in the API section of the docs. ## Checklist - [x] Title must conform to [conventional commit](https://github.com/conventional-changelog/commitlint/tree/master/%40commitlint/config-conventional). - [x] Add additional sections for `feat` and `fix` pull requests. - [x] Ensure tests are passing for affected code. - [x] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] PR cannot be broken up into smaller PRs. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [x] Release note has been added for fixes and features, or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. - [ ] Add to milestone. Co-authored-by: Tahir H. Butt --- ddtrace/debugging/_debugger.py | 17 +++++++++++++---- docs/api.rst | 7 +++++++ docs/spelling_wordlist.txt | 1 + 3 files changed, 21 insertions(+), 4 deletions(-) diff --git a/ddtrace/debugging/_debugger.py b/ddtrace/debugging/_debugger.py index 4e22c6c34f0..6a0a2b1b379 100644 --- a/ddtrace/debugging/_debugger.py +++ b/ddtrace/debugging/_debugger.py @@ -151,12 +151,17 @@ class Debugger(Service): @classmethod def enable(cls, run_module=False): # type: (bool) -> None - """Enable the debugger (idempotent).""" + """Enable dynamic instrumentation + + This class method is idempotent. Dynamic instrumentation will be + disabled automatically at exit. + """ if sys.version_info >= (3, 11, 0): raise RuntimeError( "Dynamic Instrumentation is not yet compatible with Python 3.11. " "See tracking issue for more details: https://github.com/DataDog/dd-trace-py/issues/4149" ) + if cls._instance is not None: log.debug("%s already enabled", cls.__name__) return @@ -180,7 +185,11 @@ def enable(cls, run_module=False): @classmethod def disable(cls): # type: () -> None - """Disable the debugger (idempotent).""" + """Disable dynamic instrumentation. + + This class method is idempotent. Called automatically at exit, if + dynamic instrumentation was enabled. + """ if cls._instance is None: log.debug("%s not enabled", cls.__name__) return @@ -211,7 +220,7 @@ def __init__(self, tracer=None): Snapshot: SnapshotJsonEncoder(service_name), str: str, }, - on_full=self.on_encoder_buffer_full, + on_full=self._on_encoder_buffer_full, ) self._probe_registry = ProbeRegistry(self.__logger__(service_name, self._encoder)) self._uploader = self.__uploader__(self._encoder) @@ -233,7 +242,7 @@ def __init__(self, tracer=None): log.debug("%s initialized (service name: %s)", self.__class__.__name__, service_name) - def on_encoder_buffer_full(self, item, encoded): + def _on_encoder_buffer_full(self, item, encoded): # type (Any, bytes) -> None # Send upload request self._uploader.upload() diff --git a/docs/api.rst b/docs/api.rst index 75b3a660c9c..c682191726c 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -39,3 +39,10 @@ Runtime Metrics .. autoclass:: ddtrace.runtime.RuntimeMetrics :members: + + +Dynamic Instrumentation +======================= + +.. automodule:: ddtrace.debugging + :members: diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 99ea798e416..5ffe3c2ec86 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -205,5 +205,6 @@ Serverless serverless cattrs IAST +programmatically DES Blowfish From c25d2ae44a312b8ad23131a93a3c5ed23840760d Mon Sep 17 00:00:00 2001 From: Zachary Groves <32471391+ZStriker19@users.noreply.github.com> Date: Thu, 22 Dec 2022 12:48:25 -0700 Subject: [PATCH 12/20] chore(w3c): update tracestate tag regex (#4827) ## Description Right now `_dd.propagation_error` (and any internal tag that starts with p) will get set as a tracestate tag. With this change, we make sure to only add `_dd.p.*` tags. ## Checklist - [x] Followed the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines) when writing a release note. - [x] Add additional sections for `feat` and `fix` pull requests. - [x] [Library documentation](https://github.com/DataDog/dd-trace-py/tree/1.x/docs) and/or [Datadog's documentation site](https://github.com/DataDog/documentation/) is updated. Link to doc PR in description. ## Motivation ## Design ## Testing strategy ## Relevant issue(s) ## Testing strategy ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. Co-authored-by: Munir Abdinur Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> --- ddtrace/internal/utils/http.py | 2 +- tests/tracer/test_utils.py | 14 ++++++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/ddtrace/internal/utils/http.py b/ddtrace/internal/utils/http.py index 880e266585a..258e5faeda3 100644 --- a/ddtrace/internal/utils/http.py +++ b/ddtrace/internal/utils/http.py @@ -171,7 +171,7 @@ def w3c_get_dd_list_member(context): for k, v in context._meta.items(): if ( isinstance(k, six.string_types) - and k.startswith("_dd.p") + and k.startswith("_dd.p.") # we've already added sampling decision and user id and k not in [SAMPLING_DECISION_TRACE_TAG_KEY, USER_ID_KEY] ): diff --git a/tests/tracer/test_utils.py b/tests/tracer/test_utils.py index 1a98c8669de..256f1151a5e 100644 --- a/tests/tracer/test_utils.py +++ b/tests/tracer/test_utils.py @@ -418,6 +418,19 @@ def _(): ), ["s:2", "o:synthetics", "t.unk:-4", "t.unknown:baz64"], ), + ( + Context( + trace_id=1234, + sampling_priority=2, + dd_origin="synthetics", + meta={ + # we should not propagate _dd.propagation_error, since it does not start with _dd.p. + "_dd.propagation_error": "-4", + "_dd.p.unknown": "baz64", + }, + ), + ["s:2", "o:synthetics", "t.unknown:baz64"], + ), ( Context( trace_id=1234, @@ -473,6 +486,7 @@ def _(): ], ids=[ "basic", + "does_not_add_propagation_error", "does_not_add_non_prefixed_tags", "does_not_add_more_than_256_char", "char_replacement", From 1b8a7f64affbd2ca2b312479545067091fd9e4cb Mon Sep 17 00:00:00 2001 From: Brett Langdon Date: Thu, 22 Dec 2022 17:02:49 -0500 Subject: [PATCH 13/20] fix(tracing): disallow v0.5 api usage for windows machines (#4830) ## Description Using the `v0.5` API endpoint on Windows can cause issues (see #4829 for details). This PR makes it so we default to `v0.4` on Windows and will raise an exception if you try to use `v0.5` on a Windows machine. ## Relevant issue(s) Fixes #4829 ## Testing strategy ## Reviewer Checklist - [ ] Title is accurate. - [ ] Description motivates each change. - [ ] No unnecessary changes were introduced in this PR. - [ ] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [ ] Tests provided or description of manual testing performed is included in the code or PR. - [ ] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [ ] All relevant GitHub issues are correctly linked. - [ ] Backports are identified and tagged with Mergifyio. --- ddtrace/internal/writer.py | 19 +++- .../notes/v05-encoding-eac70f23850a445e.yaml | 5 +- tests/tracer/test_writer.py | 93 +++++++++++++++++++ 3 files changed, 115 insertions(+), 2 deletions(-) diff --git a/ddtrace/internal/writer.py b/ddtrace/internal/writer.py index 9c4bb983585..4300cc7926e 100644 --- a/ddtrace/internal/writer.py +++ b/ddtrace/internal/writer.py @@ -279,9 +279,26 @@ def __init__( if headers: self._headers.update(headers) self._timeout = timeout + + # Default to v0.4 if we are on Windows since there is a known compatibility issue + # https://github.com/DataDog/dd-trace-py/issues/4829 + # DEV: sys.platform on windows should be `win32` or `cygwin`, but using `startswith` + # as a safety precaution. + # https://docs.python.org/3/library/sys.html#sys.platform + is_windows = sys.platform.startswith("win") or sys.platform.startswith("cygwin") + default_api_version = "v0.4" if is_windows else "v0.5" + self._api_version = ( - api_version or os.getenv("DD_TRACE_API_VERSION") or ("v0.5" if priority_sampler is not None else "v0.3") + api_version + or os.getenv("DD_TRACE_API_VERSION") + or (default_api_version if priority_sampler is not None else "v0.3") ) + if is_windows and self._api_version == "v0.5": + raise RuntimeError( + "There is a known compatibiltiy issue with v0.5 API and Windows, " + "please see https://github.com/DataDog/dd-trace-py/issues/4829 for more details." + ) + try: Encoder = MSGPACK_ENCODERS[self._api_version] except KeyError: diff --git a/releasenotes/notes/v05-encoding-eac70f23850a445e.yaml b/releasenotes/notes/v05-encoding-eac70f23850a445e.yaml index bb3f3c164a2..73c92aaf756 100644 --- a/releasenotes/notes/v05-encoding-eac70f23850a445e.yaml +++ b/releasenotes/notes/v05-encoding-eac70f23850a445e.yaml @@ -1,5 +1,8 @@ --- upgrade: - | - tracing: upgrades the default trace API version to ``v0.5``. The ``v0.5`` trace API version generates + tracing: upgrades the default trace API version to ``v0.5`` for non-Windows systems. The ``v0.5`` trace API version generates smaller payloads, thus increasing the throughput to the Datadog agent especially with larger traces. + - | + tracing: configuring the ``v0.5`` trace API version on Windows machines will raise a ``RuntimeError`` + due to known compatibility issues. Please see https://github.com/DataDog/dd-trace-py/issues/4829 for more details. diff --git a/tests/tracer/test_writer.py b/tests/tracer/test_writer.py index a230869d5d2..d5e3ddd4625 100644 --- a/tests/tracer/test_writer.py +++ b/tests/tracer/test_writer.py @@ -1,5 +1,7 @@ +import contextlib import os import socket +import sys import tempfile import threading import time @@ -20,12 +22,23 @@ from ddtrace.internal.writer import LogWriter from ddtrace.internal.writer import Response from ddtrace.internal.writer import _human_size +from ddtrace.sampler import RateByServiceSampler from ddtrace.span import Span from tests.utils import AnyInt from tests.utils import BaseTestCase from tests.utils import override_env +@contextlib.contextmanager +def mock_sys_platform(new_value): + old_value = sys.platform + try: + sys.platform = new_value + yield + finally: + sys.platform = old_value + + class DummyOutput: def __init__(self): self.entries = [] @@ -552,6 +565,86 @@ def test_writer_recreate_api_version(init_api_version, api_version, endpoint, en assert isinstance(writer._encoder, encoder_cls) +@pytest.mark.parametrize( + "sys_platform, api_version, ddtrace_api_version, priority_sampler, raises_error, expected", + [ + # -- win32 + # Defaults on windows + ("win32", None, None, None, False, "v0.3"), + # Default with priority sampler + ("win32", None, None, RateByServiceSampler(), False, "v0.4"), + # Explicitly passed in API version is always used + ("win32", "v0.3", None, RateByServiceSampler(), False, "v0.3"), + ("win32", "v0.3", "v0.4", None, False, "v0.3"), + ("win32", "v0.3", "v0.4", RateByServiceSampler(), False, "v0.3"), + # Env variable is used if explicit value is not given + ("win32", None, "v0.4", None, False, "v0.4"), + ("win32", None, "v0.4", RateByServiceSampler(), False, "v0.4"), + # v0.5 is not supported on windows + ("win32", "v0.5", None, None, True, None), + ("win32", "v0.5", None, RateByServiceSampler(), True, None), + ("win32", "v0.5", "v0.4", RateByServiceSampler(), True, None), + ("win32", None, "v0.5", RateByServiceSampler(), True, None), + # -- cygwin + # Defaults on windows + ("cygwin", None, None, None, False, "v0.3"), + # Default with priority sampler + ("cygwin", None, None, RateByServiceSampler(), False, "v0.4"), + # Explicitly passed in API version is always used + ("cygwin", "v0.3", None, RateByServiceSampler(), False, "v0.3"), + ("cygwin", "v0.3", "v0.4", None, False, "v0.3"), + ("cygwin", "v0.3", "v0.4", RateByServiceSampler(), False, "v0.3"), + # Env variable is used if explicit value is not given + ("cygwin", None, "v0.4", None, False, "v0.4"), + ("cygwin", None, "v0.4", RateByServiceSampler(), False, "v0.4"), + # v0.5 is not supported on windows + ("cygwin", "v0.5", None, None, True, None), + ("cygwin", "v0.5", None, RateByServiceSampler(), True, None), + ("cygwin", "v0.5", "v0.4", RateByServiceSampler(), True, None), + ("cygwin", None, "v0.5", RateByServiceSampler(), True, None), + # -- Non-windows + # defaults + ("darwin", None, None, None, False, "v0.3"), + # Default with priority sample + ("darwin", None, None, RateByServiceSampler(), False, "v0.5"), + # Explicitly setting api version + ("darwin", "v0.4", None, RateByServiceSampler(), False, "v0.4"), + # Explicitly set version takes precedence + ("darwin", "v0.4", "v0.5", RateByServiceSampler(), False, "v0.4"), + # Via env variable + ("darwin", None, "v0.4", RateByServiceSampler(), False, "v0.4"), + ("darwin", None, "v0.5", RateByServiceSampler(), False, "v0.5"), + ], +) +def test_writer_api_version_selection( + sys_platform, api_version, ddtrace_api_version, priority_sampler, raises_error, expected, monkeypatch +): + """test to verify that we are unable to select v0.5 api version when on a windows machine. + + https://docs.python.org/3/library/sys.html#sys.platform + + The possible ``sys.platform`` values when on windows are ``win32`` or ``cygwin``. + """ + + # Mock the value of `sys.platform` to be a specific value + with mock_sys_platform(sys_platform): + + # If desired, set the DD_TRACE_API_VERSION env variable + if ddtrace_api_version is not None: + monkeypatch.setenv("DD_TRACE_API_VERSION", ddtrace_api_version) + + try: + # Create a new writer + writer = AgentWriter( + agent_url="http://dne:1234", api_version=api_version, priority_sampler=priority_sampler + ) + assert writer._api_version == expected + except RuntimeError: + # If we were not expecting a RuntimeError, then cause the test to fail + if not raises_error: + pytest.fail("Raised RuntimeError when it was not expected") + + def test_writer_reuse_connections_envvar(monkeypatch): monkeypatch.setenv("DD_TRACE_WRITER_REUSE_CONNECTIONS", "false") writer = AgentWriter(agent_url="http://localhost:9126") From bdaaac92ada9387a939a38eb8950b02070ddae93 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 23 Dec 2022 08:41:02 +0000 Subject: [PATCH 14/20] chore(deps): bump actions/cache from 3.0.11 to 3.2.0 (#4823) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/cache](https://github.com/actions/cache) from 3.0.11 to 3.2.0.
Release notes

Sourced from actions/cache's releases.

v3.2.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/cache/compare/v3...v3.2.0

v3.2.0-beta.1

What's Changed

v3.1.0-beta.3

What's Changed

  • Bug fixes for bsdtar fallback, if gnutar not available, and gzip fallback, if cache saved using old cache action, on windows.

Full Changelog: https://github.com/actions/cache/compare/v3.1.0-beta.2...v3.1.0-beta.3

v3.1.0-beta.2

What's Changed

Full Changelog: https://github.com/actions/cache/compare/v3.1.0-beta.1...v3.1.0-beta.2

... (truncated)

Changelog

Sourced from actions/cache's changelog.

3.0.11

  • Update toolkit version to 3.0.5 to include @actions/core@^1.10.0
  • Update @actions/cache to use updated saveState and setOutput functions from @actions/core@^1.10.0

3.1.0-beta.1

  • Update @actions/cache on windows to use gnu tar and zstd by default and fallback to bsdtar and zstd if gnu tar is not available. (issue)

3.1.0-beta.2

  • Added support for fallback to gzip to restore old caches on windows.

3.1.0-beta.3

  • Bug fixes for bsdtar fallback if gnutar not available and gzip fallback if cache saved using old cache action on windows.

3.2.0-beta.1

  • Added two new actions - restore and save for granular control on cache.

3.2.0

  • Released the two new actions - restore and save for granular control on cache
Commits
  • c17f4bf GA for granular cache (#1035)
  • ac25611 docs: fix an invalid link in workarounds.md (#929)
  • dc097e3 Update examples.md (#1026)
  • fb86cbf Updated node example (#1008)
  • a57932f Merge pull request #1014 from jongwooo/chore/use-built-in-cache-action
  • 04b13ca chore: Use built-in cache action to cache dependencies
  • 941bc71 Merge pull request #1004 from jongwooo/chore/use-cache-in-check-dist
  • 08d8639 Merge branch 'main' into chore/use-cache-in-check-dist
  • a2f324e Merge pull request #1013 from jongwooo/refactor/use-early-return-pattern-to-a...
  • 35f4702 refactor: Use early return pattern to avoid nested conditions
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/cache&package-manager=github_actions&previous-version=3.0.11&new-version=3.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Brett Langdon --- .github/workflows/test_frameworks.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/test_frameworks.yml b/.github/workflows/test_frameworks.yml index 32d625c418f..8347bf55328 100644 --- a/.github/workflows/test_frameworks.yml +++ b/.github/workflows/test_frameworks.yml @@ -157,7 +157,7 @@ jobs: repository: tiangolo/fastapi ref: 0.75.0 path: fastapi - - uses: actions/cache@v3.0.11 + - uses: actions/cache@v3.2.0 id: cache with: path: ${{ env.pythonLocation }} From a79ef009f1cd801742ccb6c6d0dc3f7b4c56cf9d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 23 Dec 2022 13:03:16 -0500 Subject: [PATCH 15/20] chore(deps): bump actions/stale from 6 to 7 (#4812) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/stale](https://github.com/actions/stale) from 6 to 7.
Release notes

Sourced from actions/stale's releases.

v7.0.0

⚠️ This version contains breaking changes ⚠️

What's Changed

Breaking Changes

  • In this release we prevent this action from managing the stale label on items included in exempt-issue-labels and exempt-pr-labels
  • We decided that this is outside of the scope of this action, and to be left up to the maintainer

New Contributors

Full Changelog: https://github.com/actions/stale/compare/v6...v7.0.0

v6.0.1

Update @​actions/core to 1.10.0 #839

Full Changelog: https://github.com/actions/stale/compare/v6.0.0...v6.0.1

Changelog

Sourced from actions/stale's changelog.

Changelog

[7.0.0]

:warning: Breaking change :warning:

[6.0.1]

Update @​actions/core to v1.10.0 (#839)

[6.0.0]

:warning: Breaking change :warning:

Issues/PRs default close-issue-reason is now not_planned(#789)

[5.1.0]

Don't process stale issues right after they're marked stale [Add close-issue-reason option]#764#772 Various dependabot/dependency updates

4.1.0 (2021-07-14)

Features

4.0.0 (2021-07-14)

Features

Bug Fixes

  • dry-run: forbid mutations in dry-run (#500) (f1017f3), closes #499
  • logs: coloured logs (#465) (5fbbfba)
  • operations: fail fast the current batch to respect the operations limit (#474) (5f6f311), closes #466
  • label comparison: make label comparison case insensitive #517, closes #516
  • filtering comments by actor could have strange behavior: "stale" comments are now detected based on if the message is the stale message not who made the comment(#519), fixes #441, #509, #518

Breaking Changes

... (truncated)

Commits
  • 6f05e42 draft release for v7.0.0 (#888)
  • eed91cb Update how stale handles exempt items (#874)
  • 10dc265 Merge pull request #880 from akv-platform/update-stale-repo
  • 9c1eb3f Update .md files and allign build-test.yml with the current test.yml
  • bc357bd Update .github/workflows/release-new-action-version.yml
  • 690ede5 Update .github/ISSUE_TEMPLATE/bug_report.md
  • afbcabf Merge branch 'main' into update-stale-repo
  • e364411 Update name of codeql.yml file
  • 627cef3 fix print outputs step (#859)
  • 975308f Merge pull request #876 from jongwooo/chore/use-cache-in-check-dist
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/stale&package-manager=github_actions&previous-version=6&new-version=7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/stale.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index 7e3190e48ae..578f0eb87fe 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -12,7 +12,7 @@ jobs: stale: runs-on: ubuntu-latest steps: - - uses: actions/stale@v6 + - uses: actions/stale@v7 with: # Increase the total API operations from 30 to 200 # DEV: GitHub Actions have an API rate limit of 1000 operations per hour per repository From 5cf1588e578924942a1aae2792063521cbbfd499 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 27 Dec 2022 11:15:02 -0700 Subject: [PATCH 16/20] chore(ci): Bump pypa/cibuildwheel from 2.11.3 to 2.11.4 (#4843) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.11.3 to 2.11.4.
Release notes

Sourced from pypa/cibuildwheel's releases.

v2.11.4

  • 🐛 Fix a bug that caused missing wheels on Windows when a test was skipped using CIBW_TEST_SKIP (#1377)
  • 🛠 Updates CPython 3.11 to 3.11.1 (#1371)
  • 🛠 Updates PyPy 3.7 to 3.7.10, except on macOS which remains on 7.3.9 due to a bug. (#1371)
  • 📚 Added a reference to abi3audit to the docs (#1347)
Changelog

Sourced from pypa/cibuildwheel's changelog.

v2.11.4

24 Dec 2022

  • 🐛 Fix a bug that caused missing wheels on Windows when a test was skipped using CIBW_TEST_SKIP (#1377)
  • 🛠 Updates CPython 3.11 to 3.11.1 (#1371)
  • 🛠 Updates PyPy to 7.3.10, except on macOS which remains on 7.3.9 due to a bug on that platform. (#1371)
  • 📚 Added a reference to abi3audit to the docs (#1347)
Commits
  • 27fc88e Bump version: v2.11.4
  • a7e9ece Merge pull request #1371 from pypa/update-dependencies-pr
  • b9a3ed8 Update cibuildwheel/resources/build-platforms.toml
  • 3dcc2ff fix: not skipping the tests stops the copy (Windows ARM) (#1377)
  • 1c9ec76 Merge pull request #1378 from pypa/henryiii-patch-3
  • 22b433d Merge pull request #1379 from pypa/pre-commit-ci-update-config
  • 98fdf8c [pre-commit.ci] pre-commit autoupdate
  • cefc5a5 Update dependencies
  • e53253d ci: move to ubuntu 20
  • e9ecc65 [pre-commit.ci] pre-commit autoupdate (#1374)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pypa/cibuildwheel&package-manager=github_actions&previous-version=2.11.3&new-version=2.11.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/build_deploy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build_deploy.yml b/.github/workflows/build_deploy.yml index 97faa359434..c3020248f20 100644 --- a/.github/workflows/build_deploy.yml +++ b/.github/workflows/build_deploy.yml @@ -106,7 +106,7 @@ jobs: platforms: all - name: Build wheels python 3.6 and above - uses: pypa/cibuildwheel@v2.11.3 + uses: pypa/cibuildwheel@v2.11.4 env: # configure cibuildwheel to build native archs ('auto'), and some # emulated ones From 5471764113c23ccbb3f9d7178839b5f297f8eb0f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 27 Dec 2022 14:39:27 -0500 Subject: [PATCH 17/20] chore(deps): Bump actions/cache from 3.2.0 to 3.2.1 (#4842) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/cache](https://github.com/actions/cache) from 3.2.0 to 3.2.1.
Release notes

Sourced from actions/cache's releases.

v3.2.1

What's Changed

Full Changelog: https://github.com/actions/cache/compare/v3.2.0...v3.2.1

Changelog

Sourced from actions/cache's changelog.

3.2.0

  • Released the two new actions - restore and save for granular control on cache

3.2.1

  • Update @actions/cache on windows to use gnu tar and zstd by default and fallback to bsdtar and zstd if gnu tar is not available. (issue)
  • Added support for fallback to gzip to restore old caches on windows.
  • Added logs for cache version in case of a cache miss.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/cache&package-manager=github_actions&previous-version=3.2.0&new-version=3.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/test_frameworks.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/test_frameworks.yml b/.github/workflows/test_frameworks.yml index 8347bf55328..d4a389b4384 100644 --- a/.github/workflows/test_frameworks.yml +++ b/.github/workflows/test_frameworks.yml @@ -157,7 +157,7 @@ jobs: repository: tiangolo/fastapi ref: 0.75.0 path: fastapi - - uses: actions/cache@v3.2.0 + - uses: actions/cache@v3.2.1 id: cache with: path: ${{ env.pythonLocation }} From 836135b0754f815eee8896711d0e0a88a1bccaf3 Mon Sep 17 00:00:00 2001 From: Kyle Verhoog Date: Tue, 27 Dec 2022 16:05:45 -0500 Subject: [PATCH 18/20] perf(tracer): remove redundant DD_ENV logic (#4778) The DD_ENV logic is already implemented in the config. There is no need to also have it in sitecustomize. The duplicated logic resulted in the tag being unnecessarily set twice for each span. ## Reviewer Checklist - [x] Title is accurate. - [x] Description motivates each change. - [x] No unnecessary changes were introduced in this PR. - [x] Avoid breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes unless absolutely necessary. - [x] Tests provided or description of manual testing performed is included in the code or PR. - [x] Release note has been added and follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html#Release-Note-Guidelines), or else `changelog/no-changelog` label added. - [x] All relevant GitHub issues are correctly linked. - [x] Backports are identified and tagged with Mergifyio. Co-authored-by: Munir Abdinur Co-authored-by: Munir Abdinur --- ddtrace/bootstrap/sitecustomize.py | 5 ---- tests/commands/ddtrace_run_env.py | 6 ---- tests/commands/ddtrace_run_service.py | 6 ---- tests/commands/test_runner.py | 17 ----------- .../integration/test_integration_snapshots.py | 30 +++++++++++++++++++ ...t_integration_snapshots.test_env_vars.json | 26 ++++++++++++++++ 6 files changed, 56 insertions(+), 34 deletions(-) delete mode 100644 tests/commands/ddtrace_run_env.py delete mode 100644 tests/commands/ddtrace_run_service.py create mode 100644 tests/snapshots/tests.integration.test_integration_snapshots.test_env_vars.json diff --git a/ddtrace/bootstrap/sitecustomize.py b/ddtrace/bootstrap/sitecustomize.py index a8429538db9..3834c6f0acf 100644 --- a/ddtrace/bootstrap/sitecustomize.py +++ b/ddtrace/bootstrap/sitecustomize.py @@ -18,7 +18,6 @@ from ddtrace import config # noqa -from ddtrace import constants from ddtrace.debugging._config import config as debugger_config from ddtrace.internal.logger import get_logger # noqa from ddtrace.internal.runtime.runtime_metrics import RuntimeWorker @@ -114,10 +113,6 @@ def update_patched_modules(): patch_all(**EXTRA_PATCHED_MODULES) - dd_env = os.getenv("DD_ENV") - if dd_env: - tracer.set_tags({constants.ENV_KEY: dd_env}) - if "DD_TRACE_GLOBAL_TAGS" in os.environ: env_tags = os.getenv("DD_TRACE_GLOBAL_TAGS") tracer.set_tags(parse_tags_str(env_tags)) diff --git a/tests/commands/ddtrace_run_env.py b/tests/commands/ddtrace_run_env.py deleted file mode 100644 index b752f9f71fc..00000000000 --- a/tests/commands/ddtrace_run_env.py +++ /dev/null @@ -1,6 +0,0 @@ -from ddtrace import tracer - - -if __name__ == "__main__": - assert tracer._tags.get("env") == "test" - print("Test success") diff --git a/tests/commands/ddtrace_run_service.py b/tests/commands/ddtrace_run_service.py deleted file mode 100644 index a0480dc317d..00000000000 --- a/tests/commands/ddtrace_run_service.py +++ /dev/null @@ -1,6 +0,0 @@ -import os - - -if __name__ == "__main__": - assert os.getenv("DD_SERVICE") == "my_test_service" - print("Test success") diff --git a/tests/commands/test_runner.py b/tests/commands/test_runner.py index d3b33117367..f9ea896e705 100644 --- a/tests/commands/test_runner.py +++ b/tests/commands/test_runner.py @@ -39,23 +39,6 @@ def inject_sitecustomize(path): class DdtraceRunTest(BaseTestCase): - def test_service_name_passthrough(self): - """ - $DD_SERVICE gets passed through to the program - """ - with self.override_env(dict(DD_SERVICE="my_test_service")): - out = subprocess.check_output(["ddtrace-run", "python", "tests/commands/ddtrace_run_service.py"]) - assert out.startswith(b"Test success") - - def test_env_name_passthrough(self): - """ - $DD_ENV gets passed through to the global tracer as an 'env' tag - """ - - with self.override_env(dict(DD_ENV="test")): - out = subprocess.check_output(["ddtrace-run", "python", "tests/commands/ddtrace_run_env.py"]) - assert out.startswith(b"Test success") - def test_env_enabling(self): """ DD_TRACE_ENABLED=false allows disabling of the global tracer diff --git a/tests/integration/test_integration_snapshots.py b/tests/integration/test_integration_snapshots.py index 0324789cba0..302d97ae96e 100644 --- a/tests/integration/test_integration_snapshots.py +++ b/tests/integration/test_integration_snapshots.py @@ -1,5 +1,6 @@ # -*- coding: utf-8 -*- import multiprocessing +import os import mock import pytest @@ -292,3 +293,32 @@ def test_tracetagsprocessor_only_adds_new_tags(): span.set_metric(SAMPLING_PRIORITY_KEY, USER_KEEP) tracer.shutdown() + + +# Override the token so that both parameterizations of the test use the same snapshot +# (The snapshots should be equivalent) +@snapshot(token_override="tests.integration.test_integration_snapshots.test_env_vars") +@pytest.mark.parametrize("use_ddtracerun", [True, False]) +def test_env_vars(use_ddtracerun, ddtrace_run_python_code_in_subprocess, run_python_code_in_subprocess): + """Ensure environment variable config is respected by ddtrace-run usages as well as regular.""" + if use_ddtracerun: + fn = ddtrace_run_python_code_in_subprocess + else: + fn = run_python_code_in_subprocess + + env = os.environ.copy() + env.update( + dict( + DD_ENV="prod", + DD_SERVICE="my-svc", + DD_VERSION="1234", + ) + ) + + fn( + """ +from ddtrace import tracer +tracer.trace("test-op").finish() +""", + env=env, + ) diff --git a/tests/snapshots/tests.integration.test_integration_snapshots.test_env_vars.json b/tests/snapshots/tests.integration.test_integration_snapshots.test_env_vars.json new file mode 100644 index 00000000000..e6f21a442b9 --- /dev/null +++ b/tests/snapshots/tests.integration.test_integration_snapshots.test_env_vars.json @@ -0,0 +1,26 @@ +[[ + { + "name": "test-op", + "service": "my-svc", + "resource": "test-op", + "trace_id": 0, + "span_id": 1, + "parent_id": 0, + "type": "", + "error": 0, + "meta": { + "_dd.p.dm": "-0", + "env": "prod", + "runtime-id": "616ec77c69174ae8820935b9a733027c", + "version": "1234" + }, + "metrics": { + "_dd.agent_psr": 1.0, + "_dd.top_level": 1, + "_dd.tracer_kr": 1.0, + "_sampling_priority_v1": 1, + "process_id": 37652 + }, + "duration": 69000, + "start": 1670888766539700000 + }]] From ee038045d5d55514f32531106977baee5acb0575 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 28 Dec 2022 10:07:52 -0700 Subject: [PATCH 19/20] chore(deps): bump actions/cache from 3.2.1 to 3.2.2 (#4844) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/cache](https://github.com/actions/cache) from 3.2.1 to 3.2.2.
Release notes

Sourced from actions/cache's releases.

v3.2.2

What's Changed

New Contributors

Full Changelog: https://github.com/actions/cache/compare/v3.2.1...v3.2.2

Changelog

Sourced from actions/cache's changelog.

3.2.1

  • Update @actions/cache on windows to use gnu tar and zstd by default and fallback to bsdtar and zstd if gnu tar is not available. (issue)
  • Added support for fallback to gzip to restore old caches on windows.
  • Added logs for cache version in case of a cache miss.

3.2.2

  • Reverted the changes made in 3.2.1 to use gnu tar and zstd by default on windows.
Commits
  • 4723a57 Revert compression changes related to windows but keep version logging (#1049)
  • d1507cc Merge pull request #1042 from me-and/correct-readme-re-windows
  • 3337563 Merge branch 'main' into correct-readme-re-windows
  • 60c7666 save/README.md: Fix typo in example (#1040)
  • b053f2b Fix formatting error in restore/README.md (#1044)
  • 501277c README.md: remove outdated Windows cache tip link
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/cache&package-manager=github_actions&previous-version=3.2.1&new-version=3.2.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/test_frameworks.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/test_frameworks.yml b/.github/workflows/test_frameworks.yml index d4a389b4384..5c314d29010 100644 --- a/.github/workflows/test_frameworks.yml +++ b/.github/workflows/test_frameworks.yml @@ -157,7 +157,7 @@ jobs: repository: tiangolo/fastapi ref: 0.75.0 path: fastapi - - uses: actions/cache@v3.2.1 + - uses: actions/cache@v3.2.2 id: cache with: path: ${{ env.pythonLocation }} From 6de9b70d5040b7a8e3448662b841ecd5c7975291 Mon Sep 17 00:00:00 2001 From: Munir Abdinur Date: Fri, 30 Dec 2022 16:21:44 -0500 Subject: [PATCH 20/20] chore(dynamic_instrumentation): remove unnecessary log line (#4847) ## Description --- ddtrace/debugging/_config.py | 2 -- .../fix-dynamic-instrumentation-logging-aef13f091e5feed5.yaml | 4 ++++ 2 files changed, 4 insertions(+), 2 deletions(-) create mode 100644 releasenotes/notes/fix-dynamic-instrumentation-logging-aef13f091e5feed5.yaml diff --git a/ddtrace/debugging/_config.py b/ddtrace/debugging/_config.py index 46d535ca57f..34f47607eab 100644 --- a/ddtrace/debugging/_config.py +++ b/ddtrace/debugging/_config.py @@ -5,5 +5,3 @@ log = get_logger(__name__) config = DynamicInstrumentationConfig() - -log.debug("Dynamic instrumentation configuration: %r", config.__dict__) diff --git a/releasenotes/notes/fix-dynamic-instrumentation-logging-aef13f091e5feed5.yaml b/releasenotes/notes/fix-dynamic-instrumentation-logging-aef13f091e5feed5.yaml new file mode 100644 index 00000000000..0b294ea5db4 --- /dev/null +++ b/releasenotes/notes/fix-dynamic-instrumentation-logging-aef13f091e5feed5.yaml @@ -0,0 +1,4 @@ +--- +fixes: + - | + dynamic instrumentation: remove unnecessary log line from application start up