Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Issue #1745] Setup transformation process structure & transform an o…
…pportunity (#1794) ## Summary Fixes #1745 ### Time to review: __10 mins__ ## Changes proposed Setup the transformation process script structure Implement some shared utilities that subsequent PRs will use Implement the transformation logic for opportunities ## Context for reviewers A lot of setup in this PR, a lot that can be reused in the subsequent PRs to add transformations for the other sets of tables. Tried to make sure those would require refactoring or pulling out implementation details by setting up utils like the timestamp conversions + initial query to the DB to fetch the transforming records. As far as the implementation goes, determining what needs to be transformed is pretty simple - the `transformed_at` column is null. There is then a second column that says whether the record should be deleted or not. When we query the staging tables, we also join with the relevant table we'll transform to (opportunity in this case), that way we already have the "source" and "destination" records and just need to modify the destination record (or create it if its an insert).
- Loading branch information