Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigquery: allow string escaping #27

Merged
merged 3 commits into from
Feb 26, 2024

Conversation

zdenal
Copy link

@zdenal zdenal commented Feb 23, 2024

Why

We want to have string escaping logic also in BigQuery dialect with backslashes (https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#escape_sequences)

bigquery().verified_only_select(r#"SELECT CAST(TRIM(NULLIF(TRIM(JSON_QUERY(json_dump, "$.email_verified")), ''), '\"') AS BOOL) AS is_email_verified FROM foo"#);
bigquery().verified_only_select(r#"SELECT CAST(LTRIM(NULLIF(TRIM(JSON_QUERY(json_dump, "$.email_verified")), ''), '\"') AS BOOL) AS is_email_verified FROM foo"#);
bigquery().verified_only_select(r#"SELECT CAST(RTRIM(NULLIF(TRIM(JSON_QUERY(json_dump, "$.email_verified")), ''), '\"') AS BOOL) AS is_email_verified FROM foo"#);
bigquery_unescaped().verified_only_select(r#"SELECT CAST(TRIM(NULLIF(TRIM(JSON_QUERY(json_dump, "$.email_verified")), ''), '\"') AS BOOL) AS is_email_verified FROM foo"#);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to use unescaped dialect logic to let validation logic check the sql is serialised back to same value.
With escaped (default) behaviour it is not serialised back to exact string as values are escaped and stored in AST.

@@ -845,10 +845,10 @@ fn parse_like() {

// Test with escape char
let sql = &format!(
"SELECT * FROM customers WHERE name {}LIKE '%a' ESCAPE '\\'",
Copy link
Author

@zdenal zdenal Feb 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without r#"... the string was in the end containing .. ESCAPE '\', what I think is invalid syntax see

CleanShot 2024-02-25 at 12 34 14@2x

if negated { "NOT " } else { "" }
);
let select = bigquery().verified_only_select(sql);
let select = bigquery().verified_only_select_with_canonical(sql, "");
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as in AST the ESCAPE is presented as escaped_character: char it can be now only one char generically and it won't be never serialised-deserialsed back to origin '\\'. So using "" as canonical format to compare (skip comparison), BUT keep next Statement check to valid it is as expected.

Also this constraint in AST means the valid escpade format of backslash '\\' won't be never parsed by dialect with option ParserOptions::new().with_unescape(false) as it expect only 1 character by parser rule.

Copy link
Collaborator

@lustefaniak lustefaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@lustefaniak lustefaniak changed the title Allow string escaping also in BigQuery. bigquery: allow string escaping Feb 26, 2024
@zdenal zdenal merged commit 7c0410b into main Feb 26, 2024
2 checks passed
@zdenal zdenal deleted the zdenko/syn-2817-parse-error-wrong-handling-of-in-bq branch February 27, 2024 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants