point-in-time endpoint for column-level lineage

[pawel-big-lebowski]

Signed-off-by: Pawel Leszczynski [email protected]

Problem

Point in time column lineage endpoint - implementation.

Closes: #2262

Solution

• Dataset or job version from a request is used to determine date time when a version was created.
• column_lineage table is filtered for rows earlier than a determined date time.
• Column-lineage rows, dataset rows and dataset_version rows make use of the same date time.
• The combination of point-in-time and withDownstream features does not suit well and deserves a separate PR if needed.

Note: All database schema changes require discussion. Please link the issue for context.

Checklist

• You've signed-off your work
• Your changes are accompanied by tests (if relevant)
• Your change contains a small diff and is self-contained
• You've updated any relevant documentation (if relevant)
• You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
• You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
• You've included a header in any source code files (if relevant)

[codecov]

Codecov Report

Merging #2265 (888a258) into main (f3b1cbd) will increase coverage by 0.27%.
The diff coverage is 94.73%.

@@             Coverage Diff              @@
##               main    #2265      +/-   ##
============================================
+ Coverage     76.47%   76.74%   +0.27%     
- Complexity     1113     1150      +37     
============================================
  Files           216      219       +3     
  Lines          5203     5260      +57     
  Branches        421      423       +2     
============================================
+ Hits           3979     4037      +58     
+ Misses          752      748       -4     
- Partials        472      475       +3     

Impacted Files	Coverage Δ
api/src/main/java/marquez/db/ColumnLineageDao.java	100.00% <ø> (ø)
api/src/main/java/marquez/db/DatasetFieldDao.java	100.00% <ø> (ø)
...c/main/java/marquez/api/ColumnLineageResource.java	75.00% <75.00%> (-8.34%)	⬇️
...java/marquez/db/mappers/PairUuidInstantMapper.java	75.00% <75.00%> (ø)
...i/src/main/java/marquez/service/models/NodeId.java	72.13% <90.00%> (+7.64%)	⬆️
...ain/java/marquez/service/ColumnLineageService.java	97.20% <98.36%> (-0.12%)	⬇️
...a/marquez/common/models/DatasetFieldVersionId.java	100.00% <100.00%> (ø)
.../main/java/marquez/common/models/JobVersionId.java	100.00% <100.00%> (ø)
...arquez/db/mappers/ColumnLineageNodeDataMapper.java	90.47% <100.00%> (+0.47%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more