Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track request body size in XHR and Fetch instrumentations #4706

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
e349fa4
reduce overlap between (variable shadowing) between individual tests
MustafaHaddara May 10, 2024
656cbc4
update xhr test to check attr names directly instead of using key order
MustafaHaddara May 9, 2024
538e712
update fetch test to read attributes directly instead of using key order
MustafaHaddara May 14, 2024
eaf9786
override api request in beforeEach call
MustafaHaddara May 14, 2024
d6149ca
Add getXHRBodyLength and getFetchBodyLength functions to opentelemetr…
MustafaHaddara May 14, 2024
d97b02b
track request body content length in xhr instrumentation
MustafaHaddara May 14, 2024
860557e
track request body content length in fetch instrumentation
MustafaHaddara May 14, 2024
9dfe663
add changelog entry
MustafaHaddara May 14, 2024
a14c1f9
update browser; webworker tests
MustafaHaddara May 14, 2024
499c8ca
account for older platforms where ReadableStream can't tee()
MustafaHaddara May 22, 2024
19890b5
lint fix: add comment after @ts-expect-error
MustafaHaddara May 30, 2024
bee76c8
make response body measurement opt-in
MustafaHaddara Jul 15, 2024
854cc53
update named exports
MustafaHaddara Aug 12, 2024
24e3b46
Merge branch 'main' of github.com:open-telemetry/opentelemetry-js int…
MustafaHaddara Aug 19, 2024
12c42e3
switch to experimental new semantic attribute
MustafaHaddara Aug 19, 2024
91637aa
warn instead of error
MustafaHaddara Aug 19, 2024
08ab734
correctly read string length
MustafaHaddara Aug 19, 2024
8d3c533
consistent return value when we can't calculate length
MustafaHaddara Aug 19, 2024
bf92869
lint fix
MustafaHaddara Aug 19, 2024
f21eb54
update ReadableStream length calculation
MustafaHaddara Aug 23, 2024
102d128
switch to pipeThrough() to reduce memory use
MustafaHaddara Aug 23, 2024
a0a26ea
Merge branch 'main' of github.com:open-telemetry/opentelemetry-js int…
MustafaHaddara Aug 23, 2024
8b94dbc
document the semconv attributes we include
MustafaHaddara Aug 23, 2024
3569096
more correct handling of FormData
MustafaHaddara Aug 23, 2024
d011829
support formData.entries()
MustafaHaddara Aug 26, 2024
178e73b
update stream in fetch tests
MustafaHaddara Aug 26, 2024
74cb0ea
add a warning for old platforms
MustafaHaddara Aug 26, 2024
7c2bfe2
Merge branch 'main' of github.com:open-telemetry/opentelemetry-js int…
MustafaHaddara Aug 26, 2024
51cc125
fix fetch tests
MustafaHaddara Aug 26, 2024
f0dcca9
Merge branch 'main' of github.com:open-telemetry/opentelemetry-js int…
MustafaHaddara Aug 27, 2024
192f440
Merge branch 'main' into request-body-size
JamieDanielson Aug 27, 2024
0007def
Merge branch 'main' into request-body-size
MustafaHaddara Oct 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions packages/opentelemetry-sdk-trace-web/src/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,9 +244,7 @@
}

if (body instanceof FormData) {
// typescript doesn't like it when we pass FormData into URLSearchParams
// even though this is actually totally valid
return getByteLength(new URLSearchParams(body as any).toString());
return getFormDataSize(body);
}

if (body instanceof URLSearchParams) {
Expand All @@ -266,6 +264,19 @@
return TEXT_ENCODER.encode(s).byteLength;
}

function getFormDataSize(formData: FormData): number {
let size = 0;
for (const [key, value] of formData.entries()) {

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / browser-tests

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / build

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / node-tests (14)

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / node-windows-tests

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / node-tests (16)

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / webworker-tests

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / node-tests (18)

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / node-tests (20)

Property 'entries' does not exist on type 'FormData'.

Check failure on line 269 in packages/opentelemetry-sdk-trace-web/src/utils.ts

View workflow job for this annotation

GitHub Actions / node-tests (22)

Property 'entries' does not exist on type 'FormData'.
size += key.length;
if (value instanceof Blob) {
size += value.size;
} else {
size += value.length;
}
Comment on lines +271 to +276
Copy link

@tbrockman tbrockman Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the 👻 here, just been a bit busy with life and quitting #dayjob!

Just noticed, should these also use getByteLength?

Feels silly to suggest it given that FormData size varies (as you mentioned, browser/platform-specific implementation differences from things like boundaries and such, I just checked out Firefox for example), so I understand if it seems unnecessary at this point since it'd just be shaving a tiny bit of hypothetical inaccuracy off an estimate that will inherently be incorrect under the circumstances that it applies to (and feel free to ignore).

Suggested change
size += key.length;
if (value instanceof Blob) {
size += value.size;
} else {
size += value.length;
}
size += getByteLength(key.length);
if (value instanceof Blob) {
size += value.size;
} else {
size += getByteLength(value.length);
}

}
return size;
}
Comment on lines +268 to +279
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an improvement over the old implementation (which was stringifying the FormData, and that wouldn't actually stringify file objects correctly), but it still doesn't calculate the exact FormData size. Browsers will serialize FormData in their own way (ex including a separator for each of the fields, etc.)

I will keep investigating for more accurate approaches.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this seems good enough for now, and maybe if anyone is sufficiently motivated they can contribute the changes to calculate it more precisely for their platform 😬


MustafaHaddara marked this conversation as resolved.
Show resolved Hide resolved
/**
* sort resources by startTime
* @param filteredResources
Expand Down
15 changes: 14 additions & 1 deletion packages/opentelemetry-sdk-trace-web/test/utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -645,9 +645,22 @@ describe('utils', () => {
formData.append('key1', 'true');
formData.append('key2', 'hello world');

assert.strictEqual(getXHRBodyLength(formData), 26);
assert.strictEqual(getXHRBodyLength(formData), 23);
assert.strictEqual(getXHRBodyLength(new FormData()), 0);
});
it('should compute body length for FormData payload with a file', () => {
const formData = new FormData();
const f = new File(
['hello world hello world hello world'],
'test_file.txt'
);
formData.append('file', f);

// length should be:
// 4 for the key of the file in the form data
// 35 for the file contents
assert.strictEqual(getXHRBodyLength(formData), 39);
});
it('should compute body length for URLSearchParams payload', () => {
const search = new URLSearchParams({
key1: 'true',
Expand Down
Loading