Skip to content

Releases: climatepolicyradar/navigator-data-ingest

v2.0.5-beta

13 Apr 16:53
0e4eaaa
Compare
Choose a tag to compare

Removing the glob statements to find files for an id and replacing this with well know document paths created using f-strings. This is a less dynamic solution but cloudpathlib takes a very long time to search the entire s3 prefix for the file.

v2.0.4-beta

13 Apr 15:00
0a68e1a
Compare
Choose a tag to compare

Updating the fields of the update type from csv_value and db_value to db_value and s3_value.

v2.0.3-beta

13 Apr 11:06
9a16c0a
Compare
Choose a tag to compare

Update the ingest stage to trust the db_state.json provided value and overwrite any data in s3 even if it is not as expected. The data mismatch is instead logged.

v2.0.2-beta

13 Apr 08:42
1af1b05
Compare
Choose a tag to compare

Updating of the parse action to archive all instances of the document in the pipeline as this is the required functionality now.

v2.0.1-beta

12 Apr 10:16
e070ce1
Compare
Choose a tag to compare
  • Refactoring code
  • Updating the type of the Document that is passed from the backend in the db_state.json file to no longer contain document_status

v2.0.0-beta

11 Apr 13:44
305e998
Compare
Choose a tag to compare

Updating the ingest stage to read in document updates and deletes. This is a document that contains the information on new documents and the updates to be made to existing documents. The ingest stage then contains functionality to facilitate updates to the updated documents.

v1.0.2-beta

20 Mar 14:21
eb5b7be
Compare
Choose a tag to compare

Hardcoding the s3 path for the pdf document upload to s3 to use the navigator/ prefix.

v1.0.1-beta

10 Jan 14:03
bf90dc2
Compare
Choose a tag to compare

Updating of the logs for document upload to include the import ID.

v1.0.0-beta

06 Jan 11:06
6ccf4ac
Compare
Choose a tag to compare

First release for new infra