Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add created_at indexed column to Lineage events table #2300

Closed
prachim-collab opened this issue Dec 8, 2022 · 0 comments · Fixed by #2299
Closed

Add created_at indexed column to Lineage events table #2300

prachim-collab opened this issue Dec 8, 2022 · 0 comments · Fixed by #2299

Comments

@prachim-collab
Copy link
Contributor

For analytics use case we need to incrementally copy and export lineage_events to DWH. For this use case there is no good way to identify incrementally created events in database. The current event_time in lineage_events table is client generated and can be back dated.
Proposing to alter lineage_events table to add a new created_at timestamp column. Also index this new column.
More discussion to be had regarding backfill and default values of this column.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant