Backend Storage using CARTO

[ana-md]

General

Is your feature request related to a problem? Please describe.
Kepler.gl has a 'Save/Share your map' feature using Dropbox but it doesn't have a way to privately manage your maps and data after its creation, something that can be frustrating after spending time working on the creation of your maps.

Describe the solution you'd like
Having an integrated and private system to easily persist, load and manage your maps and data from an external provider.

Describe alternatives you've considered
We propose a generic approach, in which any cloud provider with an OAuth API can include a predefined set of methods to implement the backend storage feature. This feature will allow the user to persist maps and datasets, with privacy control, and to get them back from the provider.

As the first backend storage provider, we want to add a set of functionalities that include different possibilities working with your maps and data coming from Kepler (e.g., sharing via URL, saving privately at CARTO, loading back in Kepler, management from your CARTO Dashboard,...).

Implementation Plan

We propose dividing the implementation of this feature into 3 different development phases to make team collaboration between CARTO and Kepler easier and faster.

Each of these development phases will come with both a descriptive issue and a PR associated.
This way, progress for each phase is clearer and can be reviewed in parallel by Kepler’s team.

The development phases (with their current status) are the following:

Working with Maps:

1. Phase 1: Sharing map via URL using CARTO

• Related Issue: Sharing map via URL using CARTO
• PR for this Issue: Sharing map via URL using CARTO

2. Phase 2: Backend Storage (I): Persisting MAPS (privately)

• Related issue: Backend Storage (I): Persisting MAPS (privately)
• PR not there yet, on-going implementation from our side.
• Progress:
  Find here our latest prototype including all the feedback we already got from you in previous conversations: https://projects.invisionapp.com/share/TPTSVDX2KF5#/screens/389126759
  (Note: if there is any other question/feedback from your side, please share it directly in the issue created for this phase)

Working with Datasets:

3. Phase 3: Backend Storage (II): Persisting DATA (privately)

• Issue 3 - Under definition
• PR not there yet
• Progress:
  We are working on a proper definition of data management that provides a great user experience, as there are many possibilities and different challenges due to the different approach while working with data that Kepler and CARTO have.
  In the meantime, we have found the following blockers that we have already reported in your repo and would be great if you could take a look at them:
  - [Bug] Importing data from a URL that's not a file no longer works
  - [Enhancement] Let users rename datasets

Communication

Please, keep in mind that nothing should be merged until all PRs are ready and it can be publicly released as one single feature (Backend Storage).

[ana-md]

@heshan0131 find here a short update on the project status from our side:

Phase 1:

• The PR for Phase 1 is ready, I have just linked it to this issue (editing directly the description above).
• You can try it out signing up from Kepler into CARTO’S trial account and we can extend the account for you if you need it for further testing.
• It would be great if you could give us feedback but please don’t merge it yet until we agree on that later on when the project is ready enough to be released as a complete feature.

Phase 2:
Our team is currently working on Phase 2 and we will keep this feedback from you in mind.
It would be great that, if you have any additional comments to the design or the definition of this phase, you could let us know as soon as possible so that we can implement it accordingly. Thank you!

[heshan0131]

FYI. We are going to provide saving UI from kepler.gl core. started a PR on it. Aim to land it be EOW