-
Notifications
You must be signed in to change notification settings - Fork 16
Conversation
('[email protected]', '2022-01-07 09:00:00'), | ||
('[email protected]', '2022-01-08 09:00:00'); | ||
|
||
SELECT create_hypertable('onsite_personnel', 'time', migrate_data => true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Creates a hypertable here by first creating a postgres table and then converting it into a hypertable. You have to specify the column that holds the "time" values. In this contrived example, I have unique times that should be the primary key with a "responsible" employee who was on call at that time.
I've added an integration test that demonstrates a basic access and erasure request on this hypertable. Some comments are that we require primary keys to erase. Noting that timescale requires the unique/primary key dimension to contain the time
dimension. You can have composite primary keys including the time dimension in Timescale and fidesops has support for this.
Also worth noting that "datetimes" are not a fully supported date type in Fidesops yet. A lot of our existing examples that deal with dates tend to treat dates as strings. Masking date fields can be problematic outside the null masking strategy. (Although since the date field is going to be part of the primary key, it is less likely that this is a field that is masked by itself).
…ted and populated the database, and a simple access and erasure request.
…inside the network, not outside.
3d2f3b3
to
c98e5d7
Compare
src/fidesops/ops/schemas/connection_configuration/connection_secrets_timescale.py
Show resolved
Hide resolved
Co-authored-by: Paul Sanders <[email protected]>
…e're subclassing the postgresql connector.
Thanks very much for your comments @sanders41! |
❗ Contains a migration; verify downrev before merging
Purpose
Add a connector for Timescale and demonstrate that it can make the standard access and erasure requests.
Changes
Added a Timescale connector that is built entirely off of our existing PostgreSQL connector. This has the same limitations of our current PostgreSQL connector in that we don't yet support complex types, among other things.
The code added here is largely boilerplate very similar to our postgres connector. We should explore ways in the future to create factories that could populate similar information for different connectors. This was a lot of copy/paste/adjusting the relevant variables. The main things that are different are a slightly different docker-compose and the introduction of the hypertable construct.
Checklist
CHANGELOG.md
fileCHANGELOG.md
file is being appended toUnreleased
section in an appropriate category. Add a new category from the list at the top of the file if the needed one isn't already there.Run Unsafe PR Checks
label has been applied, and checks have passed, if this PR touches any external servicesTicket
Fixes #1311