-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
opt: memo cache is not invalidated when a user-defined type's schema is renamed #96674
Comments
Won't this fall out of drew's work on checking whether the memo is stale? If it doesn't the reason it doesn't would be that we're not tracking a reference to the schema. |
This is caused by the fact that we stored UDT OID's in the memo and lookup the types by OID to determine if they have changed. cockroach/pkg/sql/opt/metadata.go Line 335 in e62ac1f
We need to either:
|
This is a problem for tables referenced by IDs, too... I'm going to try adding schemas to the list of objects tracked during memo validation so we can just track function OIDs. I think it should allow us to make a similar simplification for data sources as well. @chengxiong-ruan added a fix for the switching databases problem in functions here; I think we could do something similar for resolving types, schemas, and data sources. |
Yes, we'd better have defense on those OID references as well. Though it's only fun when users know the numeric representation for some reason and spend effort on finding the IDs heh. |
The metadata stores tracks each UDT referenced in the query by its OID. It validates the cached data by checking that the OID still resolves to the expected type and that the type hasn't undergone schema changes since the query was resolved. The problem with tracking only by OID instead of by the name used to reference the type is that the query cache can incorrectly pass validation when the UDT's schema or database has been altered in some way. This previously led to behavior like the following: ``` defaultdb> create type sc1.t as enum ('foo', 'bar'); CREATE TYPE defaultdb> select 'foo'::sc1.t; t ------- foo (1 row) defaultdb> alter schema sc1 rename to sc2; ALTER SCHEMA defaultdb> select 'foo'::sc1.t; t ------- foo (1 row) ``` This patch adds schema tracking to the metadata dependency tracking. For now, this is only done for UDT resolution, but following commits will adapt data source and function resolution to also take advantage of this change. This allows catalog objects to be tracked in the metadata by ID rather than by name, which simplifies the implementation. This ensures a cached query will be properly invalidated if the schema or database of a UDT in the query is altered or dropped. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could cause a query involving user-defined types to ignore schema changes when the same query was invoked before and after the change.
The metadata stores tracks each UDT referenced in the query by its OID. It validates the cached data by checking that the OID still resolves to the expected type and that the type hasn't undergone schema changes since the query was resolved. The problem with tracking only by OID instead of by the name used to reference the type is that the query cache can incorrectly pass validation when the UDT's schema or database has been altered in some way. This previously led to behavior like the following: ``` defaultdb> create type sc1.t as enum ('foo', 'bar'); CREATE TYPE defaultdb> select 'foo'::sc1.t; t ------- foo (1 row) defaultdb> alter schema sc1 rename to sc2; ALTER SCHEMA defaultdb> select 'foo'::sc1.t; t ------- foo (1 row) ``` This patch adds schema tracking to the metadata dependency tracking. For now, this is only done for UDT resolution, but following commits will adapt data source and function resolution to also take advantage of this change. This allows catalog objects to be tracked in the metadata by ID rather than by name, which simplifies the implementation. This ensures a cached query will be properly invalidated if the schema or database of a UDT in the query is altered or dropped. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could cause a query involving user-defined types to ignore schema changes when the same query was invoked before and after the change.
The metadata stores tracks each UDT referenced in the query by its OID. It validates the cached data by checking that the OID still resolves to the expected type and that the type hasn't undergone schema changes since the query was resolved. The problem with tracking only by OID instead of by the name used to reference the type is that the query cache can incorrectly pass validation when the UDT's schema or database has been altered in some way. This previously led to behavior like the following: ``` defaultdb> create type sc1.t as enum ('foo', 'bar'); CREATE TYPE defaultdb> select 'foo'::sc1.t; t ------- foo (1 row) defaultdb> alter schema sc1 rename to sc2; ALTER SCHEMA defaultdb> select 'foo'::sc1.t; t ------- foo (1 row) ``` This patch adds schema tracking to the metadata dependency tracking. For now, this is only done for UDT resolution, but following commits will adapt data source and function resolution to also take advantage of this change. This allows catalog objects to be tracked in the metadata by ID rather than by name, which simplifies the implementation. This ensures a cached query will be properly invalidated if the schema or database of a UDT in the query is altered or dropped. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could cause a query involving user-defined types to ignore schema changes when the same query was invoked before and after the change.
Previously, it was possible for invalid queries to be kept in the cache after a schema change, since user-defined types were tracked only by OID. This missed cases where the search path changed which object a name in the query resolved to (or whether it resolved at all). This patch fixes this behavior by tracking UDT references by name, similar to what was already done for data sources. This ensures that the query staleness check doesn't miss changes to the search path. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could prevent a cached query with a user-defined type reference from being invalidated even after a schema change that should prevent the type from being resolved.
Previously, it was possible for invalid queries to be kept in the cache after a schema change, since user-defined types were tracked only by OID. This missed cases where the search path changed which object a name in the query resolved to (or whether it resolved at all). This patch fixes this behavior by tracking UDT references by name, similar to what was already done for data sources. This ensures that the query staleness check doesn't miss changes to the search path. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could prevent a cached query with a user-defined type reference from being invalidated even after a schema change that should prevent the type from being resolved.
Previously, it was possible for invalid queries to be kept in the cache after a schema change, since user-defined types were tracked only by OID. This missed cases where the search path changed which object a name in the query resolved to (or whether it resolved at all). This patch fixes this behavior by tracking UDT references by name, similar to what was already done for data sources. This ensures that the query staleness check doesn't miss changes to the search path. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could prevent a cached query with a user-defined type reference from being invalidated even after a schema change that should prevent the type from being resolved.
96045: opt: check UDFs and UDTs by name when checking metadata dependencies r=DrewKimball a=DrewKimball #### tree: distinguish UDFs and builtins that use a SQL string body This refactor changes the meaning of the `Overload.IsUDF` field to be true only for user-defined functions - meaning those created using `CREATE FUNCTION`. This is contrasted with builtin functions that are defined with a SQL string set in `Overload.Body`. Logic that cares only about user-defined functions can continue checking `Overload.IsUDF`, while cases that deal with the SQL body should use `Overload.HasSQLBody()`. Note that it is insufficient to check whether `Overload.Body` is empty because it is possible to define a UDF with an empty body. #### opt: refactor metadata dependency tracking This commit performs some refactoring for the way data source objects are tracked in `opt.Metadata` in order to make future changes to UDT and UDF dependency tracking easier. More particularly, UDTs and UDFs will be able to re-resolve any references by name. This is necessary in order to handle cases where changes to the search-path cause names in the query to resolve to different objects. #### opt: track UDT references by name in the Metadata Previously, it was possible for invalid queries to be kept in the cache after a schema change, since user-defined types were tracked only by OID. This missed cases where the search path changed which object a name in the query resolved to (or whether it resolved at all). This patch fixes this behavior by tracking UDT references by name, similar to what was already done for data sources. This ensures that the query staleness check doesn't miss changes to the search path. #### opt: track UDFs in the Metadata This patch adds tracking for user-defined functions in `opt.Metadata`. This ensures that the query cache will be correctly invalidated after a schema change or search-path change that could change the query's semantics, or cause it to error. Fixes #95214 Fixes #93082 Fixes #93321 Fixes #96674 Release note (bug fix): Fixed a bug that could prevent a cached query from being invalidated when a UDF or UDT referenced by that query was altered or dropped, or when the schema or database of a UDF or UDT was altered or dropped. 98319: pkg/server: fix `/demologin` to properly redirect to home page r=dhartunian a=abarganier With the introduction of the server controller, we introduced a layer between the HTTP handler and the HTTP server. When this was introduced, the logic to attempt a login to all tenants forgot to handle a specific case for `/demologin`, where the status code is set to a 302 redirect, instead of a 200 status OK. This broke the redirect piece of the `/demologin` endpoint. This patch updates the `attemptLoginToAllTenants` HTTP handler to properly set the 302 response code in the case where the underlying login function does so on the sessionWriter. Release note: none Epic: CRDB-12100 Fixes: #98253 98696: sql: disallow using cluster_logical_timestamp as column default when backfilling r=Xiang-Gu a=Xiang-Gu Previously, `ADD COLUMN ... DEFAULT cluster_logical_timestamp()` would crash the node and leave the table in a corrupt state. The root cause is a nil pointer dereference. This commit fixed it by returning an unimplemented error and hence disallow using this builtin function as default value when backfilling. Fixes: #98269 Release note (bug fix): fixed a bug as detailed in #98269. Co-authored-by: Drew Kimball <[email protected]> Co-authored-by: Alex Barganier <[email protected]> Co-authored-by: Xiang Gu <[email protected]>
Previously, it was possible for invalid queries to be kept in the cache after a schema change, since user-defined types were tracked only by OID. This missed cases where the search path changed which object a name in the query resolved to (or whether it resolved at all). This patch fixes this behavior by tracking UDT references by name, similar to what was already done for data sources. This ensures that the query staleness check doesn't miss changes to the search path. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could prevent a cached query with a user-defined type reference from being invalidated even after a schema change that should prevent the type from being resolved.
Previously, it was possible for invalid queries to be kept in the cache after a schema change, since user-defined types were tracked only by OID. This missed cases where the search path changed which object a name in the query resolved to (or whether it resolved at all). This patch fixes this behavior by tracking UDT references by name, similar to what was already done for data sources. This ensures that the query staleness check doesn't miss changes to the search path. Fixes cockroachdb#96674 Release note (bug fix): Fixed a bug that could prevent a cached query with a user-defined type reference from being invalidated even after a schema change that should prevent the type from being resolved.
We don't invalidate a cached memo referencing user-defined types when the schema owning the UDT is renamed.
The last query should fail because
sc1
no longer exists.Jira issue: CRDB-24272
The text was updated successfully, but these errors were encountered: