Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: only create new txn for Bind if necessary #74423

Merged
merged 1 commit into from
Feb 16, 2022

Conversation

rafiss
Copy link
Collaborator

@rafiss rafiss commented Jan 4, 2022

01ca1e5 changed Bind so that it always
uses a transaction to resolve the arguments. This caused a performance
hit, leading to this code using 4.2% of the CPU utilization in a
read-only workload. To optimize the code, a new transaction is only
created when necessary, which is when we need to resolve user-defined
types or OID types.

No release note since the optimization only applies to unreleased
versions.

Release note: None

@rafiss rafiss requested review from nvanbenschoten and a team January 4, 2022 20:11
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we also need one for evaluating the various oid types as cases. Like IIRC SELECT $1::regclass when you bind a string will do a catalog lookup. I could be wrong, but I recall this causing bugs in some very delicate edge cases where we fall back to avoiding the lease manager. Even when we don't it only subtly worked because the planner had not been reset and the committed previous transaction was used for the lease or something like that.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten)

@rafiss
Copy link
Collaborator Author

rafiss commented Jan 6, 2022

Yeah, you're right. pgwirebase.DecodeDatum ends up calling planner.ResolveOIDFromOID which needs a transaction. I think the tests in pkg/sql/pgwire/testdata/pgtest/bind_and_resolve were supposed to cover this, but I guess they are getting lucky like you said.

@rafiss rafiss force-pushed the optimize-bind-txn branch from 022426f to 7792cdb Compare January 26, 2022 06:25
@rafiss
Copy link
Collaborator Author

rafiss commented Jan 26, 2022

I added the check for OID types, but I still am not sure how to test it.

01ca1e5 changed Bind so that it always
uses a transaction to resolve the arguments. This caused a performance
hit, leading to this code using 4.2% of the CPU utilization in a
read-only workload. To optimize the code, a new transaction is only
created when necessary, which is when we need to resolve user-defined
types or OID types.

No release note since the optimization only applies to unreleased
versions.

Release note: None
@rafiss rafiss force-pushed the optimize-bind-txn branch from 7792cdb to 8f77b90 Compare January 26, 2022 19:25
@rafiss rafiss requested a review from ajwerner February 1, 2022 18:57
Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason this worked before is pretty insane! It worked before because when we end up resolving the OID, we use the InternalExecutor, which, when it gets a nil txn uses its own. ☹️

results, err := ie.QueryRowEx(ctx, "queryOid", txn,
sessiondata.NoSessionDataOverride, q, toResolve)

Pretty wild. Having the internal executor optionally taking a transaction is a little crazy.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten and @rafiss)


pkg/sql/conn_executor_prepare.go, line 437 at r3 (raw file):

			// Use the existing transaction.
			if err := resolve(ctx, txn); err != nil {
				return retErr(err)

Wait, if we have a txn, doesn't that mean the planner is already in a good state? Do we need to reset it and all that jazz?

Code quote:

			// Use the existing transaction.
			if err := resolve(ctx, txn); err != nil {
				return retErr(err)
			}

Copy link
Collaborator Author

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice find on the internalExecutor! hm...

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner and @nvanbenschoten)


pkg/sql/conn_executor_prepare.go, line 437 at r3 (raw file):

Previously, ajwerner wrote…

Wait, if we have a txn, doesn't that mean the planner is already in a good state? Do we need to reset it and all that jazz?

good point .. i'll try moving the resetPlanner stuff into the branch where a new txn is created

Copy link
Collaborator Author

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner and @nvanbenschoten)


pkg/sql/conn_executor_prepare.go, line 437 at r3 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

good point .. i'll try moving the resetPlanner stuff into the branch where a new txn is created

actually, i think it's fine to call resetPlanner each time. it's already called any time any pgwire command is exec'd. (e,g, we call resetPlanner in execStmtInOpenState even though there already is a txn.)

@rafiss rafiss requested a review from otan February 16, 2022 17:01
@rafiss
Copy link
Collaborator Author

rafiss commented Feb 16, 2022

This is ready for review. Given that this is just an optimization, and no existing tests regressed, I think I feel pretty safe about merging this.

Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @nvanbenschoten and @otan)

@rafiss
Copy link
Collaborator Author

rafiss commented Feb 16, 2022

tftr!

bors r=ajwerner

@craig
Copy link
Contributor

craig bot commented Feb 16, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Feb 16, 2022

Build succeeded:

@craig craig bot merged commit 132ea6c into cockroachdb:master Feb 16, 2022
@rafiss rafiss deleted the optimize-bind-txn branch February 16, 2022 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants