-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: AzureSQL masking pipeline #22
base: main
Are you sure you want to change the base?
feat: AzureSQL masking pipeline #22
Conversation
3b83aab
to
c11bc1f
Compare
c11bc1f
to
7573540
Compare
dcsazure_AzureSQL_to_AzureSQL_mask_pl/dcsazure_AzureSQL_to_AzureSQL_mask_pl.json
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Set this in pre/post script for both sink (filtered and non filtered)
In pre SQL script
concat('SET IDENTITY_INSERT', ' ', $DF_SINK_SCHEMA, '.', $DF_SINK_TABLE, ' ', 'ON')
In post SQL script
concat('SET IDENTITY_INSERT', ' ', $DF_SINK_SCHEMA, '.', $DF_SINK_TABLE, ' ', 'OFF')
General Syntax is
SET IDENTITY_INSERT <SCHEMA>.<TABLE_NAME> <ON|OFF>
PS: I've tested this with table that has identity enabled (Auto incrementing primary key). Request you to test your flow with this enabled.
You might need to add directly in the SINK settings and reexport the template.
6adebfc
to
8179bda
Compare
6272164
to
40be998
Compare
b204213
to
0da24e0
Compare
Problem
We want to make a DCS for Azure template that mask data available in Azure SQL DB.
Solution
Add templates for DCS for Azure Azure SQL to Azure SQL for masking.
Testing Done
Followed the build instructions and imported the template into an existing data factory.
Successfully masked a table with all supported data types and an identity column and inserted the masked data into a sink schema table using the Azure SQL masking pipeline.
Note:
Currently facing an issue masking a table without identity column. Created DAAS-9463 ticket for the same.This issue is fixed in the latest commit.