sql: cross reference tracking of udf usage #87699
Labels
A-sql-routine
UDFs and Stored Procedures
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-sql-foundations
SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Comments
chengxiong-ruan
added
C-enhancement
|
#83231 should be done before this |
craig bot
pushed a commit
that referenced
this issue
Feb 6, 2023
95245: sem/tree: avoid heap allocs during pretty-printing r=chengxiong-ruan,rafiss a=knz
Everywhere applicable, change
```go
for i, xxx := range yyy {
ctx.FormatNode(&xxx)
}
```
to:
```go
for i := range yyy {
ctx.FormatNode(&yyy[i])
}
```
Epic: None
96476: sql: validate function volatility r=chengxiong-ruan a=chengxiong-ruan
This commits adds a simple udf volatility check by utilizing
the volatilities collected by the optbuilder when building a
statement. Errors are throw if any statement is not as strict
as the target volatility. Stable function folding is turned off
when building statements in CREATE FUNCTION to catch stable
function's volatility.
Release note (sql change): Previously, UDFs can be created with
any volatility no matter if the function body statements contain
any expression which would violate the target volatility. For
example, an immutable function might use `random()` in it. This
change added validations to guarantee that all statements in the
function body should be as strict as the expected UDF volatility.
Informs: #87699
96515: ui: fix time scale selection r=maryliag a=maryliag
When making a call to retrieve the list of fingerprints per index, it was suppose to use the round dates from the time picker, otherwise it wouldn't get the rounded hour that has the aggregated timestamp.
Epic: None
Release note: None
96651: storage: fix TestPebbleMetricEventListener r=itsbilal a=jbowens
In #96021 handling of disk slow events was made asynchronous. This introduced a race during Close where the goroutine logging could still be inflight.
Fix #96635.
Epic: None
Release note: None
Co-authored-by: Raphael 'kena' Poss <[email protected]>
Co-authored-by: Chengxiong Ruan <[email protected]>
Co-authored-by: maryliag <[email protected]>
Co-authored-by: Jackson Owens <[email protected]>
craig bot
pushed a commit
that referenced
this issue
Feb 21, 2023
97038: sql: use udf in check constraint r=chengxiong-ruan a=chengxiong-ruan Informs #87699 **sql: use UDFs in check constraints** This commit turns on UDF usages in CHECK constraints. It does a few things: 1. Add a concept of `SchemaExprContext`, which used to be a pure string. This commit makes it a type so that we can switch on different context of a expression. This allow fine grain control over in which circumstance that a UDF is allowed. 2. Serialize Function Expression to use OID reference to UDF. This allows objects reference UDFs by ID so that functions can be renamed. 3. Refine cross reference validation in function descriptor and table descriptor for more safety. 4. Fix legacy schema changer, so that cross references between UDF and tables are properly tracked when CHECK constraints are added and dropped. 5. Fix declarative schema changer, so that UDFs are recognized in check constraint elements, and ops are added to add and remove UDF back references. Release note (sql change): previously UDFs are not allowed in tables and any other object. This patch enables UDF usage in CHECK constraints of tables in both legacy schema changer and delcarative schema changer. Dependency circles are not allowed, namely if a UDF depends on a table, then the table can't use that UDF. **sql: version gate udf usage check** This commit adds version gate for udf usage check, so that cross reference won't be broken because there won't be cases like a constraint is added in new version, but dropped in an old version. Release note (sql change): This patch adds version gate so the UDF usage in CHECK constraints are not allowed until cluster is fully upgraded to 23.1. **backupccl: fix backup and restore to work with objects referencing UDFs** Release note (enterprise change): Previously UDFs can't be referenced from other objects, so backup and restore oly needs to read and write UDF descriptors without worrying about missing UDFs when doing `RESTORE TABLE`. Now that we turned UDF's usage in table CHECK constraints. We need to be able to handle all function ID rewrites in CHECK constraint expressions during `RESTORE`. A new `RSTORE` command option `skip_missing_udfs` is added, so that when the option is given, UDF dependencies will be skipped if UDF descriptors are missing. The main use case, as of this patch is that currently when doing `RESTORE TABLE`, we don't restore referenced UDFs together, so UDFs are consider missing, which blocks the table restore unless the `skip_missing_udfs` option is specified, so that those check constraints are dropped to remove the dependency to unblock `RESTORE TABLE`. Co-authored-by: Chengxiong Ruan <[email protected]>
craig bot
pushed a commit
that referenced
this issue
Feb 25, 2023
97501: sql: use udf in column default value in new table r=chengxiong-ruan a=chengxiong-ruan Informs: #87699 **sql: split ColumnDefaultExpr context into more granular ones** Previously we only have one `ColumnDefaultExpr` context. This is not enough for the granularity we want to control over UDF use case allowance because we want to allow UDF in column default for new tables but not existing tables. We also want allow UDF for `SET DEFAULT`. To address, this commit split the context into more granular ones so that we can do better allowance control. **sql: use UDFs in column DEFAULT of new tables and SET DEFAULT** Release note (sql change): Previously, we didn't allow using UDFs in column DEFAULT. This commit adds support for that but only allows UDFs in column DEFAULT when creating a new table or doing SET DEFAULT. **sql: add test cases of renaming a function used in CHECK** Release note: None **sql: fix backup and restore to work with UDFs in column DEFAULT** Release note (enterprise change): previously UDFs can't be referenced from column DEFAULTs. Now we turned on UDF's usage in DEFAULTs, we need to teach BACKUP and RESTORE to rewrite UDF ids referenced in column's DEFAULT expressions. If UDF dependencies are missing and `skip_missing_udfs` flag is given, the DEFAULT expressions would be dropped during the RESTORE. Co-authored-by: Chengxiong Ruan <[email protected]>
exalate-issue-sync
bot
added
T-sql-foundations
|
Currently, UDF are only allowed in COLUMN DEFAULT (new table and SET DEFAULT, both don't require a backfill) and check constraint. UDF usage is gated here. There is limitation in the backfiller that UDF cannot be executed. It need to be fixed before moving on to support UDF usage requiring a backfill (such as adding a column with default to a non-empty table). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently, user-defined functions are not allowed to be used within other objects (table, view, udf and etc..), so that forward references to udf and backward references from udf are not recorded at the moment. To enable udf usages, we need to properly track such cross-references.
Jira issue: CRDB-19488
Epic CRDB-19497
The text was updated successfully, but these errors were encountered: