Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove s3 postgres #1264

Closed
2 tasks done
simonhir opened this issue Feb 2, 2024 · 3 comments
Closed
2 tasks done

Remove s3 postgres #1264

simonhir opened this issue Feb 2, 2024 · 3 comments

Comments

@simonhir
Copy link
Member

simonhir commented Feb 2, 2024

Currently the S3-integration has a PostgreSQL db for cleaning up files. This function is directly supported from S3 via lifetime.
Therefore the separate db functionality should be removed and be replaced by the S3 lifetime.

The lifetime can i.e. be set via S3-Browser

Acceptance criteria

  • The S3-integration does not use a database anymore
  • The S3-integration passes the HexagonalArchitectureTest (needs to be added)

Alternative solution for cleanup

  • New method for tagging folders with (deprecation) label
  • Configure S3 lifecycle to cleanup files having the deprecation label
  • Tagging could be triggered through engine process end event

References

@markostreich
Copy link
Contributor

Bitte prüfen, ob die DB irgendwo für direkte Abfragen genutzt wird. Das wäre noch ein Hinderungsgrund für den Abbau.
Usecase wäre: Ordner-Abfrage (evtl. im Kontext Erstellen der Presigned-URL)

@markostreich
Copy link
Contributor

markostreich commented Jul 1, 2024

Bitte diese Properties im ops-Repo entfernen/korrigieren:

  IO_MUENCHENDIGITAL_DIGIWF_S3_CRONJOB_CLEANUP_EXPIREDFOLDERS: "0 15 10 15 * ?"
  IO_MUENCHENDIGITAL_DIGIWF_S3_CRONJOB_CLEANUP_UNUSEDFOLDERS: "0 15 10 16 * ?"

Wird in #1706 offen gelassen.

@simonhir simonhir self-assigned this Jul 2, 2024
@markostreich
Copy link
Contributor

Das gibt es in application-itest.yaml und application-test.yaml vom digiwf-s3-integration-service.

io.muenchendigital.digiwf.s3:
  cleanupcronjob: "0 15 10 15 * ?"

Sieht so aus, als ob das nicht mehr genutzt wird. Kann wohl auch weg.

This was referenced Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants