Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Infra: Properly setup the Database without re-running ingestion all the time #102

Closed
fwilhe opened this issue Sep 16, 2024 · 1 comment · Fixed by #111
Closed

CI Infra: Properly setup the Database without re-running ingestion all the time #102

fwilhe opened this issue Sep 16, 2024 · 1 comment · Fixed by #111
Assignees
Milestone

Comments

@fwilhe
Copy link
Member

fwilhe commented Sep 16, 2024

Currently there are 2 ways how glvd can be setup in the CI infra

  • Use the container image without any data/schema und run the ingestion process (this takes a long time and may fail depending on the cluster location)
  • Use the container image with a data snapshot (this is obviously not a way to run it in production)

We should look into proper backup/restore concepts for databases, for example with this helm chart

@fwilhe fwilhe self-assigned this Sep 16, 2024
@fwilhe fwilhe changed the title CI Infra: Properly setup the Database CI Infra: Properly setup the Database without re-running ingestion all the time Sep 16, 2024
fwilhe added a commit to gardenlinux/glvd-contrib that referenced this issue Sep 16, 2024
@pnpavlov
Copy link
Member

I would recommend that in the context of this task and covering our development needs (no production usage):

  1. Adopt a helm chart based on the standard ones available online without HA (e.g. Bitnami prepped this one).
  2. Deploy clean DB instance in our development cluster that is available only within the context of the namespace and not exposed to the public internet in any way.
  3. Setup the cron job and let it run first time for a full data load (long).
  4. Take a database SQL backup for local needs and document how the database dump could be downloaded and re-imported in local database.
  5. Let the cron job continue to run daily for delta loads (short).
  6. Occasionally (e.g. monthly?) update the database dump file backup for local development work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants