-
Notifications
You must be signed in to change notification settings - Fork 741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
subscriber: use of SmallVec
in EnvFilter
makes them quite large
#1567
Labels
kind/bug
Something isn't working
Comments
hawkw
added a commit
that referenced
this issue
Sep 15, 2021
The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. Fixes #1567
hawkw
added a commit
that referenced
this issue
Sep 15, 2021
The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. Fixes #1567
hawkw
added a commit
that referenced
this issue
Sep 15, 2021
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
Closed by #1568. |
Wow, thanks for such a quick fix, that's pretty cool! =) |
davidbarsky
pushed a commit
that referenced
this issue
Nov 29, 2021
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 23, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 23, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
hawkw
added a commit
that referenced
this issue
Mar 24, 2022
## Motivation The `DirectiveSet` type used in `EnvFilter` and `Targets` uses `SmallVec` to store the filtering directives when the `SmallVec` feature is enabled. This is intended to improve the performance of iterating over small sets of directives, by avoiding a heap pointer dereference. PR #1550 changed the directives themselves to also use `SmallVec` for storing _field_ filters. This was intended to make the same optimization for field filters. However, it had unintended consequences: an empty `SmallVec` is an array of `T` items (plus metadata), while an empty `Vec` is just a couple of words. Since _most_ filters don't have field filters, this meant that we were suddenly using a lot more space to store...nothing. This made `EnvFilter`s _much_ larger, causing problems for some users (see #1567). ## Solution This branch undoes the change to `SmallVec` for field name/value filters. This takes the size of an `EnvFilter` from 5420 bytes back down to 1272 bytes. I also added some tests that just print the size of various `EnvFilter` and `Targets` values. These don't make any assertions, but can be run for development purposes when making changes to these types. Fixes #1567 Signed-off-by: Eliza Weisman <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@hawkw I think we are experiencing stack overflow on windows as a result of this update.
The previous size_of
EnvFilter
was200
, but now it is5420
.It was hard, but we captured the backtrace on overflow on windows:
Details
Not sure if this is a bug in particular, because opting out of
smallvec
feature fixes the problem, but still this has broken our builds =(Originally posted by @Veetaha in #1550 (comment)
The text was updated successfully, but these errors were encountered: