Load data and kepler.gl file using URLs #260

macrigiuseppe · 2018-10-22T02:45:58Z

Content:

Added ability in demo website to load map/data using URLs
Refactored the way we load samples using promises
This diff will add the ability to load data and kepelr.gl files using urls. It is currently being implemented only in the demo app. once we agree upon UI and implementation we can move forward and embed the code into Kepler.gl core framework.
The implementation takes advantage of the existing load file action defined in kepler.gl.
I refactored the way we load config and data for samples using promises.

New modal dialog for load from URLS

This work resolves #256

heshan0131 · 2018-11-14T21:09:25Z

package.json

    "cover": "nyc --reporter html --reporter cobertura --reporter json-summary npm test && nyc report",
    "start": "npm run install-and-start -- examples/demo-app start-local",
+    "start:": "npm run install-and-start -- examples/open-modal start-local",


2 start scripts?

hmm, both scripts were already present in package.json. It was added in this commit: 25e18b0

I placed them close to each other to be more consistent.

heshan0131 · 2018-11-14T21:10:40Z

yarn.lock

@@ -5,6 +5,7 @@
 "@deck.gl/core@^5.3.3":
  version "5.3.3"
  resolved "https://registry.yarnpkg.com/@deck.gl/core/-/core-5.3.3.tgz#a13c07e5fa3e22297fd450d6da8ab9aac334b1f0"
+  integrity sha512-1a4me1QRM3/wzEVzZRAflX/Uh67gE9/3fWjl35iVY7lvNrCP0wl2PUfaQkmeThTa4Sp/Qq24ZTpHx1QvkQYH8g==


remove this integrity entries

these entries are generated by the new yarn. I will revert them

heshan0131 · 2018-11-14T21:29:37Z

examples/demo-app/src/actions.js

+// The following methods are only used to load SAMPLES
+/**
+ *
+ * @param options {


not valid JSDoc

- Refactored the way we load samples using promises

heshan0131

LGTM, Some minor comments, maybe add a TODO to cover them later

heshan0131 · 2018-11-16T23:35:01Z

examples/demo-app/src/actions.js

@@ -111,21 +242,19 @@ function loadRemoteMap(sample) {
 * @returns {function(*)}
 */
 export function loadSampleConfigurations(sampleMapId = null) {
-  return (dispatch) => {
+  return dispatch => {
    requestJson(MAP_CONFIG_URL, (error, samples) => {
      if (error) {
        Console.warn(`Error loading sample configuration file ${MAP_CONFIG_URL}`);


shouldn't this error also be caught and displayed in the UI?

I need to update the sample UI in order to show the error. I will do that

heshan0131 · 2018-11-16T23:43:00Z

examples/demo-app/src/actions.js

+  return new Promise((resolve, reject) => {
+    requestMethod(url, (error, result) => {
+      // TODO: we need to let users know about CORS issues
+      error ? reject(error) : resolve(result)


Error will not be presented in all failed request, statusCode 304 for example will not send an error back.

An example of over engineered (but catch all) error handling can be found here:

https://code.uberinternal.com/diffusion/VIVOY/browse/master/src/shared/api/request-utils.js$30-37

heshan0131 · 2018-11-16T23:44:29Z

examples/demo-app/src/components/load-data-modal/load-remote-map.js

+        <InputForm>
+          <StyledDescription>Load your map using your custom URL</StyledDescription>
+          <StyledInputLabel>
+            You can use the following formats: CSV | JSON | KEPLERGL.JSON. Make sure the url contains the file extension.


is KEPLERGL.JSON a required file extension to load the config json? if not, just mention, the json format can also be a kepler.gl config json. Otherwise it's a little confusing

KEPLERGL.JSON is not required. I only used it because that's the file name we generate with kepler.
I am going to update the content of the description

heshan0131 · 2018-11-16T23:46:49Z

examples/demo-app/src/actions.js

+ * @param config
+ */
+function loadMapCallback(dispatch, error, result, options, config) {
+  if (error) {


you can return a dispatch => {} function, instead of passing the diaptch around

return dispatch => { if (error) {...} else { dispatch(...) } }

I will review this one. I think before this diff we were using loadMapCallback in multiple locations but now we may not need to encapsulate it anymore

heshan0131 · 2018-11-16T23:47:37Z

examples/demo-app/src/actions.js

+ */
+function loadMapCallback(dispatch, error, result, options, config) {
+  if (error) {
+    Console.warn(`Error loading datafile ${options.dataUrl}`);


if result in error, we only going to warn them? instead of displaying it in the UI?

leftover from previous implementation that was shuffled around. I will updated it

MosheMalin · 2019-06-27T15:54:25Z

Hi,
This is great feature.
Can it be integrated as is into Kepler itself? (Personally I think the UI is just fine :-) )
Thanks.

macrigiuseppe requested a review from heshan0131 October 22, 2018 15:00

macrigiuseppe self-assigned this Oct 22, 2018

macrigiuseppe added feature A new feature roadmap item Item that's on our current roadmap labels Oct 22, 2018

macrigiuseppe mentioned this pull request Oct 22, 2018

Dynamically loading kepler json from the web #256

Closed

heshan0131 reviewed Nov 14, 2018

View reviewed changes

macrigiuseppe force-pushed the load_remote_url branch from a707c66 to 07a5f96 Compare November 16, 2018 02:06

macrigiuseppe added 13 commits November 16, 2018 11:31

- Added ability in demo website to load map/data using URLs

5014c2a

- Refactored the way we load samples using promises

Updated path

b905b8d

Added ability to pass map url in the URL as query paramenter

9493225

Fixed lint errors

3f0aff1

Added back base path

a6701f3

Removed route name detection

412e5a4

Fixed prop requirement

cda227f

Update yarn.lock and jsdoc

a9cfbe6

Added error handler for retrieving data using url

13efc44

Adding option to load samples using keplergl file

22bf5de

Temporary fix for anyalitcs. It's throwing an error when call loadFiles

80a558f

Filtering out usdot example

7ac3d0f

Fixed analytics tracking for load files

cd92db5

macrigiuseppe force-pushed the load_remote_url branch from 7312b7d to cd92db5 Compare November 16, 2018 21:56

Added style to error message

f386b1f

heshan0131 reviewed Nov 16, 2018

View reviewed changes

heshan0131 approved these changes Nov 17, 2018

View reviewed changes

Handling response errors and displaying them in the UI

8bd1ff8

macrigiuseppe merged commit 1fe81d2 into master Nov 20, 2018

macrigiuseppe deleted the load_remote_url branch November 20, 2018 10:06

macrigiuseppe mentioned this pull request Nov 23, 2018

dynamically add to dataset #176

Open

heshan0131 added the rfc label Jan 23, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Load data and kepler.gl file using URLs #260

Load data and kepler.gl file using URLs #260

macrigiuseppe commented Oct 22, 2018 •

edited

Loading

heshan0131 Nov 14, 2018

macrigiuseppe Nov 16, 2018

heshan0131 Nov 14, 2018

macrigiuseppe Nov 16, 2018 •

edited

Loading

heshan0131 Nov 14, 2018

heshan0131 left a comment

heshan0131 Nov 16, 2018

macrigiuseppe Nov 19, 2018

heshan0131 Nov 16, 2018

heshan0131 Nov 16, 2018

macrigiuseppe Nov 19, 2018

heshan0131 Nov 16, 2018

macrigiuseppe Nov 19, 2018

heshan0131 Nov 16, 2018

macrigiuseppe Nov 19, 2018

MosheMalin commented Jun 27, 2019

Load data and kepler.gl file using URLs #260

Load data and kepler.gl file using URLs #260

Conversation

macrigiuseppe commented Oct 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

macrigiuseppe Nov 16, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heshan0131 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MosheMalin commented Jun 27, 2019

macrigiuseppe commented Oct 22, 2018 •

edited

Loading

macrigiuseppe Nov 16, 2018 •

edited

Loading