Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add experimental support for Citus #3437

Closed
punktilious opened this issue Mar 7, 2022 · 1 comment
Closed

Add experimental support for Citus #3437

punktilious opened this issue Mar 7, 2022 · 1 comment
Assignees
@punktilious
Labels
enhancement New feature or request

Comments

@punktilious
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Citus is a distributed database extension for PostgreSQL.

Add distribution/replication information to the schema so that this metadata can be applied when the tables are created. Filter the creation of foreign key relationships according to the distribution rules.

Describe the solution you'd like
The schema tool should contain support for Citus databases.

Describe alternatives you've considered
Apply the Citus logic manually after the schema is created.

Acceptance Criteria

  1. GIVEN [a precondition]
    AND [another precondition]
    WHEN [test step]
    AND [test step]
    THEN [verification step]
    AND [verification step]

Additional context
Does not support migration from an earlier version. The first Citus install must be FHIR schema version V0026.
@punktilious punktilious added the enhancement New feature or request label Mar 7, 2022
punktilious added a commit that referenced this issue Mar 7, 2022
@punktilious
issue #3437 experimental support for Citus database
c4a11be 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Mar 9, 2022
@punktilious
issue #3437 schema build changes to support Citus distribution
7244dce 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Mar 14, 2022
@punktilious
issue #3437 citus requires some custom ingestion logic
a7f2379 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 5, 2022
@punktilious
issue #3437 distribute add_any_resource function in Citus
4398119 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 5, 2022
@punktilious
issue #3437 merge latest main
d0eaa36 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 5, 2022
@punktilious
issue #3437 distributed schema using shard_key
68819ab 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 10, 2022
@punktilious
issue #3437 remote index kafka consumer
0752bbb 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 12, 2022
@punktilious
issue #3437 remote index support for profile, tags and security
6343ab9 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 15, 2022
@punktilious
issue #3437 remote index wait until server transaction commits
a8acdb8 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 20, 2022
@punktilious
issue #3437 add logical_resource_ident and ref param support
7125f67 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 27, 2022
@punktilious
issue #3437 fix reference unit tests for new ref_values table
f705e5a 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 27, 2022
@punktilious
issue #3437 use logical_resource_ident for absolute references
0c1702b 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 27, 2022
@punktilious
issue #3437 compilation issues from shard update
8c5cc75 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 30, 2022
@punktilious
issue #3437 fix remote index processing for new logical_resource_iden…
ca0bfa0 
…t recs

Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue May 30, 2022
@punktilious
issue #3437 tidied javadoc
246897f 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 1, 2022
@punktilious
issue #3437 fixed issues using Citus
24bb883 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 1, 2022
@punktilious
issue #3437 use identity cache for more ids
3e6699d 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 2, 2022
@punktilious
issue #3437 merge with main for R4B
19daa4c 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 2, 2022
@punktilious
issue #3437 do not apply permissions to tables not created
fcf2c1e 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 3, 2022
@punktilious
issue #3437 initial derby unit test for fhir-remote-index
f2687cc 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 4, 2022
@punktilious
issue #3437 more unit test coverage for fhir-remote-index
f44b4cb 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 6, 2022
@punktilious
issue #3437 delete parameters from new xx_ref_values
4de35a5 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 8, 2022
@punktilious
issue #3437 allow reindex to be forced after database migration
40e2454 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 8, 2022
@punktilious
issue #3437 use logical_resource_ident for read and vread with citus
c3ab574 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 8, 2022
@punktilious
issue #3437 fixed postgres and db2 add_any_resource to use lr ident
fe03611 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 8, 2022
@punktilious
issue #3437 db-type arg now required when running schema tool
1644408 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 9, 2022
@punktilious
issue #3437 fixed schema migration for Db2
7ca7633 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 9, 2022
@punktilious
issue #3437 fixed missing grant for Db2
89e14ad 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 9, 2022
@punktilious
issue #3437 new inserts need mt_id for Db2 multitenant
fde6f80 
Signed-off-by: Robin Arnold <[email protected]>
@lmsurpre lmsurpre mentioned this issue Jun 10, 2022
Closed
punktilious added a commit that referenced this issue Jun 14, 2022
@punktilious
issue #3437 addressed review comments. Support whole system token val…
7a9221a 
…ues. Remove ref_version_id

Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 14, 2022
@punktilious
issue #3437 fixing drop column for ref_version_id
dec8d3a 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 15, 2022
@punktilious
issue #3437 tab to space and fixed comments
dd08e87 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 15, 2022
@punktilious
issue #3437 fixed fhir-bucket and corrected DropView comment
f37233b 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 15, 2022
@punktilious
issue #3437 corrected DropView comment
5f7e25d 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 15, 2022
@punktilious
issue #3437 javadoc review comments
150e3c3 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 17, 2022
@punktilious
issue #3437 fix usage of logicalResourceId and resourceId for remote …
f85055f 
…index

Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 17, 2022
@punktilious
issue #3437 skip parameter storage for remote index with derby and db2
5aa9325 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 20, 2022
@punktilious
issue #3437 add instanceIdentifier to remote index messages and other…
c446a15 
… review fixes

Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 21, 2022
@punktilious
issue #3713 use distributed add_any_resource function for citus (#3718)
583903a 
* issue #3713 use distributed add_any_resource function for citus

Signed-off-by: Robin Arnold <[email protected]>

* issue #3437 fix usage of logicalResourceId and resourceId for remote index

Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 21, 2022
@punktilious
issue #3437 inject instanceIdentifier when sending remote index payload
6c8d712 
Signed-off-by: Robin Arnold <[email protected]>
punktilious added a commit that referenced this issue Jun 21, 2022
@punktilious
issue #3437 more schema design documentation
59e3c24 
Signed-off-by: Robin Arnold <[email protected]>
@lmsurpre
Copy link
Member

lmsurpre commented Aug 4, 2022 •
edited
Loading

Robin ran the fhir-server-test integration tests against a server configured with the citus backend and all the tests were passed.

Interestingly, this was using single-node PostgreSQL that is configured to work like Citus by specifying dataSourceType=citus in the fhir-server-config.
On a real citus, there remain some limitations on the search API including _include, _revinclude, chaining, and reverse chaining (_has)

@lmsurpre lmsurpre closed this as completed Aug 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@punktilious punktilious

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lmsurpre @punktilious