[EPIC] - Nebari Backup & Restore

[viniciusdc]

This meta issue tracks the progress of implementing backup and restore functionality for our services. The original Request for Discussion (RFD) can be found here. The primary goal of this initiative is to provide an easily accessible solution for users to migrate cluster states during Nebari upgrades.

The first iteration focuses on services including Keycloak, Conda-store, and NFS data.

• [ENH] Backup & Restore initial architecture development #2650

Within the above architecture, each service has its separate service class with some pre-defined methods to be exposed (e.g backup and restore). Bellow are tracking issues for each component in terms of their design (methods, authentication, serialization etc) and inclusion in the main service:

Design

• [ENH] - Design import/export connector for Keycloak API #2519
• [ENH] - Design import/export connector for Conda-Store API #2655
• [ENH] - Design import/export connector for EFS Snapshots #2656

Implementation

• [ENH] - Expose Backup and Restore for Keycloak API #2657
• [ENH] - Expose Backup and Restore for NFS snapshots #2653
• [ENH] - Expose Backup and Restore for Conda-Store API #2520

Note: The development work will be done in a separate repository called nebari-dev/backup-restore, which will be first developed in Python for fast iteration and later will be moved to Go for easier shipping and launching within nebari. (This allows the code to be executed outside the nebari environment if needed)

[viniciusdc]

Later, after the first iteration, the main code will merge with Nebari's to reduce the attrition of painting to sources. Also, it's expected that some schema requirements will be available to new services in the future for dynamically including them as part of the backup process.

Though this heavily depends on merging our docker images, code back to Nebari's main repo.