Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage/tscache: Pick up andy-kimball/arenaskl fix #35630

Merged
merged 3 commits into from
Mar 12, 2019
Merged

storage/tscache: Pick up andy-kimball/arenaskl fix #35630

merged 3 commits into from
Mar 12, 2019

Conversation

nvanbenschoten
Copy link
Member

Fixes #31624.
Fixes #35557.

This commit picks up andy-kimball/arenaskl#4.

I strongly suspect that the uint32 overflow fixed in that PR was the
cause of the two index out of bounds panics. See that commit for more
details.

The PR also fixes a bug in memory recylcling within the tscache. I confirmed
on adriatic that over 900 64MB arenas had been allocated since it was last
wiped.

@nvanbenschoten
storage/tscache: fix skiplist arena re-use
893e69f 
This was incorrectly assuming that `Arena.Size` returned the full
capacity of the memory arena instead of the currently allocated
size. Since it is rare that the arena ever fills up without some
overflow, it was effectively disabling this optimization.

The size check turns out to not even be needed. I believe it was
added back when the page size was dynamic and hooked up to a cluster
setting. Since it isn't currently, we don't need the check.

Release note: None
@nvanbenschoten
storage/tscache: Pick up andy-kimball/arenaskl fix
2d35cc9 
Fixes cockroachdb#31624.
Fixes cockroachdb#35557.

This commit picks up andy-kimball/arenaskl#4.

I strongly suspect that the uint32 overflow fixed in that PR was the
cause of the two index out of bounds panics. See that commit for more
details.

Release note: None
@nvanbenschoten nvanbenschoten requested review from ajwerner and a team March 12, 2019 03:15
@cockroach-teamcity
Copy link
Member

This change is Reviewable

ajwerner
ajwerner approved these changes Mar 12, 2019
View reviewed changes
Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit:

See that commit for more details.

That commit:

The reason for this is subtle and has to do with failed allocations
slowly building up and eventually overflowing Arena.n. I'll explain
on the PR that vendors this fix.

:lgtm: The other commit has reasonable commentary. You can let us in on further subtlety some other time.

Reviewed 1 of 1 files at r1, 3 of 3 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@nvanbenschoten
storage/tscache: return ErrArenaFull whenever a node cantInit
df0f652 
This change returns `ErrArenaFull` in all cases where `cantInit` is true
for a node in `ratchetValueSet`, instead of just when `cantInit` is true
and `ratchetValueSet` is attempting to initialize the node. We already
know any update to the node is going to fail, so there's no sense in
trying to update it again.

This should avoid a few memory copies and atomic loads for scans over nodes
that get stuck in a `cantInit` state in immutable `sklPages`.

Release note: None
@nvanbenschoten
Copy link
Member Author

The theory was that a full sklPage in intervalSkl with a large number of uninitialized nodes could end up resulting in a large number of failed attempts to allocate from the arena as scanTo attempted to ratchet uninitialized nodes. This could drive up the arena size accounting without bound until it overflowed. This, in turn, could result in the ErrArenaFull check being bypassed and an allocation incorrectly appearing to succeed with an offset that would cause an out-of-bounds panic (after a corresponding underflow).

I added a third commit which should short-circuit the arena allocation attempts a little earlier for nodes that have already failed an arena allocation. This would have made the crash less frequent, but wouldn't have solved it entirely. However, this should improve performance in these cases where we already know an arena allocation is going to fail, which aren't uncommon for scans over immutable skl pages.

tbg
tbg approved these changes Mar 12, 2019
View reviewed changes
Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find.

@nvanbenschoten
Copy link
Member Author

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 12, 2019

Build failed (retrying...)

@craig
Copy link
Contributor

craig bot commented Mar 12, 2019

Canceled (will resume)

craig bot pushed a commit that referenced this pull request Mar 12, 2019
@rytaft @RaduBerinde
@nvanbenschoten @celiala
Merge #35321 #35587 #35630 #35644 #35651
e44f517 
35321: opt: propagate set operation output types to input columns r=rytaft a=rytaft

This commit updates the `optbuilder` logic for set operations in which
the types of the input columns do not match the types of the output
columns. This can happen if a column on one side has type Unknown,
but the corresponding column on the other side has a known type such
as Int. The known type must be propagated to the side with the unknown
type to prevent errors in the execution engine related to decoding
types.

If there are any column types on either side that don't match the output,
then the `optbuilder` propagates the output types of the set operation down
to the input columns by wrapping the side with mismatched types in a
Project operation. The Project operation passes through columns that
already have the correct type, and creates cast expressions for those
that don't.

Fixes #34524

Release note (bug fix): Fixed an error that happened when executing
some set operations containing only nulls in one of the input columns.

35587: opt: add more cost for lookup joins with more ON conditions r=RaduBerinde a=RaduBerinde

This is a very limited fix for #34810.

The core problem is that we don't take into account that if we have an
ON condition, not only there's a cost to evaluate it on each row, but
we are generating more internal rows to get a given number of output
rows.

I attempted to do a more general fix (for all join types), where I
tried to estimate the "internal" number of rows using
`unknownFilterSelectivity` for each ON condition. There were two
problems:
 - in some cases (especially with lookup joins) we have an extra ON
   condition that doesn't actually do anything:
     `ab JOIN xy ON a=x AND a=10` becomes
     `ab JOIN xy ON a=x AND a=10 AND x=10` becomes
   and `a=10` could remain as an ON condition. This results in bad
   query plans in important cases (e.g. TPCC) where it prefers to do
   an extra lookup join (due to a non-covering index) just because of
   that condition.

 - we don't have the equality columns readily available for hash join
   (and didn't want to extract them each time we cost). In the future
   we may split the planning into a logical and physical stage, and we
   should then separate the logical joins from hash join.

For 19.1, we simply simply add a cost for lookup joins that is
proportional to the number of remaining ON conditions. This is the
least disruptive method that still fixes the case observed in #34810.
I will leave the issue open to address this properly in the next
release.

Note that although hash joins and merge joins have the same issue in
principle, in practice we always generate these expressions with
equality on all possible columns.

Release note: None

35630: storage/tscache: Pick up andy-kimball/arenaskl fix r=nvanbenschoten a=nvanbenschoten

Fixes #31624.
Fixes #35557.

This commit picks up andy-kimball/arenaskl#4.

I strongly suspect that the uint32 overflow fixed in that PR was the
cause of the two index out of bounds panics. See that commit for more
details.

The PR also fixes a bug in memory recylcling within the tscache. I confirmed
on adriatic that over 900 64MB arenas had been allocated since it was last
wiped.

35644: opt: use correct ordering for insert input in execbuilder r=RaduBerinde a=RaduBerinde

We were setting up a projection on the Insert's input but we were
accidentally using the parent Insert's ordering instead of that of the
input.

Fixes #35564.

Release note (bug fix): Fixed a "column not in input" crash when
`INSERT ... RETURNING` is used inside a clause that requires an
ordering.

35651: jobs, sql, ui: Create `AutoCreateStats` job type r=celiala a=celiala

With #34279, enabling the cluster setting
`sql.stats.experimental_automatic_collection.enabled` has the potential
to create many CreateStats jobs, which can cause the Jobs view on the
AdminUI to become cluttered.

This commit creates a new `AutoCreateStats` job type for these auto-created
CreateStats jobs, so that users are able to still see their own manual runs
of CREATE STATISTICS, via the pre-existing `CreateStats` type.

cc @danhhz, @piyush-singh, @rolandcrosby 

![jobs-auto-create-stats](https://user-images.githubusercontent.com/3051672/54212467-5cea2c80-44b9-11e9-9c11-db749814f019.gif)

Release note (admin ui change): AutoCreateStats type added to
Jobs page to filter automatic statistics jobs.

Fixes #34377.

Co-authored-by: Rebecca Taft <[email protected]>
Co-authored-by: Radu Berinde <[email protected]>
Co-authored-by: Nathan VanBenschoten <[email protected]>
Co-authored-by: Celia La <[email protected]>
@craig
Copy link
Contributor

craig bot commented Mar 12, 2019

Build succeeded

@craig craig bot merged commit df0f652 into cockroachdb:master Mar 12, 2019
@nvanbenschoten nvanbenschoten deleted the nvanbenschoten/tscacheCrash branch March 12, 2019 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tbg tbg tbg approved these changes

@ajwerner ajwerner ajwerner approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@nvanbenschoten @cockroach-teamcity @ajwerner @tbg