Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/better external data #887

Merged
merged 11 commits into from
Apr 17, 2019
Merged

Feature/better external data #887

merged 11 commits into from
Apr 17, 2019

Conversation

jthrilly
Copy link
Member

Resolves #885

  • Allows external data that uses variable names rather than UUIDs to be imported, and makes a "best guess" at converting them to UUIDs on import.
  • Variables that cannot be mapped to the codebook are left unchanged
  • Allows CSV files to be specified as external network data, and converts them to network JSON when loaded.

@jthrilly
Copy link
Member Author

This needs to be tested on cordova before merging. Load development protocol from settings to do this.

@wwqrd
Copy link
Contributor

wwqrd commented Apr 12, 2019

Already acknowledged with @jthrilly, csv isn't working for cordova (ios + android).

@jthrilly
Copy link
Member Author

In response to the poor performance of this, I made a stab at moving it to a web worker. The long term idea would be to move all blocking code to workers.

It has definitely helped, but not as much as I had hoped.

I think this means that the long, long term solution is to calculate this data once at a non performance critical moment (such as starting an interview session, or when the protocol is installed), and then store the decoded and restructured data. Then, stream it in as we do with media assets.

@wwqrd
Copy link
Contributor

wwqrd commented Apr 15, 2019

I think it would be great to have this generate at interview start.

Do we have any way of determining which bits are slow? If it's due to having large files/objects in memory then we may need to think of more alternative approaches, like a restful api, since we can't stream json.

@jthrilly jthrilly merged commit 6aff60b into master Apr 17, 2019
@jthrilly jthrilly deleted the feature/better-external-data branch September 4, 2023 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Updates to external data
2 participants