Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include all _sdc_ meta columns on all synced tables #759

Open
paulhansen-dx opened this issue Apr 20, 2022 · 3 comments
Open

Include all _sdc_ meta columns on all synced tables #759

paulhansen-dx opened this issue Apr 20, 2022 · 3 comments

Comments

@paulhansen-dx
Copy link

Stitch only adds the _sdc_deleted_at meta column when a deleted record is first discovered.
Is there any reason all meta columns cannot be added at the time of the first sync when the destination tables are created by Stitch, regardless of whether or not hard deletes ever occur in the source db? This seems consistent regardless of which database integration is being used.

Summary: Add _sdc_deleted_at column to all destination tables regardless of the existence of a delete event.

@bibli-alex
Copy link

We've just run into this issue too - if the table is using log-based replication, it should always include the column. The fact that they're all NULL is immaterial - our models in our transformations rely on that column existing so that when they become not-null in the future, the code will be there to handle it.

@lewisarmistead
Copy link

This feature would reduce our overhead of detecting and measuring deletes by allowing us to run tests using that _sdc_deleted_at field for new log-based replication sources right after their first sync rather than either waiting for the first deletion or incorporating conditional logic to accommodate the missing field.

@lukegenung
Copy link

This seems like a major oversight, especially since this approach to filtering rows WHERE _sdc_deleted_at IS NULL is how the Stitch docs recommend handling hard deletes. Instead, I just did WHERE COALESCE(created_at, updated_at) IS NOT NULL . This makes Stitch's metadata columns and docs kind of useless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants