[Issue #2528] Change job schedule ELT process to hourly

[chouinar]

Summary

Fixes #2528

Time to review: 3 mins

Changes proposed

Modified ELT process to run hourly

Removed the load job for the v0 data

Context for reviewers

I can't imagine there is much value in the v0 job now that v1 works. Probably will save us a bit as we won't be running a process near constantly that no one uses.

I changed the schedule expression to a cron so that the schedule is a bit more consistent. The rate(..) configuration seems to be based on when you turn it on, or last did a deploy. This will keep it from varying every time we deploy and instead make it so it's always at the top of the hour.

Additional information

https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-scheduled-rule-pattern.html

[coilysiren]

Approved in concept, but we'll want to solve the terraform lint error 🙏🏽

[chouinar]

Approved in concept, but we'll want to solve the terraform lint error 🙏🏽

I don't know why I always forget terraform formatting. I run the format command like 3x for every API change I make

[mdragon]

LGTM

[coilysiren]

Q for the sake of my own knowledge: the load transform job also copies Oracle data?

[chouinar]

Q for the sake of my own knowledge: the load transform job also copies Oracle data?

Load transform is the full ELT process made up of 3 "tasks"

• Load data from Oracle via the FDW into our staging tables
• Run the transformation logic between the staging and actual opportunity tables
• Set current opportunities

The job I deleted is the old v0 copy job that loads into the "transfer" tables that we never expanded past just the singular opportunity table.