You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To Reproduce
Steps to reproduce the behavior:
Run the staging macro on DataBricks using the sha algorithm. The resulting column will be a string rather than binary.
Expected behavior
The hash keys should be binarys as expected by the DV methodology.
Additional context
Appears in the code that there is an existing cast binary implementation for DataBricks which is being used on the MD5 hashing algorithm. This just need to be applied to the SHA version as well.
Hi @UselessAlias Just to let you know, our next release will introduce opt-in (so we don't break things for everyone!) support for what we're calling "Native hashing" for this. For those interested, this will also provide the same resolution for BigQuery which is also handicapped by strings at the moment. Thanks for your patience
Describe the bug
Noticed that the databricks SHA hashing algorithm is not casting the column to a binary type leaving it as a string.
Environment
dbt version: 1.6.0
automate_dv version: 0.10.1
Database/Platform: DataBricks
To Reproduce
Steps to reproduce the behavior:
Run the staging macro on DataBricks using the sha algorithm. The resulting column will be a string rather than binary.
Expected behavior
The hash keys should be binarys as expected by the DV methodology.
Additional context
Appears in the code that there is an existing cast binary implementation for DataBricks which is being used on the MD5 hashing algorithm. This just need to be applied to the SHA version as well.
Screenshots
AB#5354
The text was updated successfully, but these errors were encountered: