Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Optimize dataset preview performance #1299

Merged
merged 24 commits into from
Feb 13, 2024

Conversation

bprusinowski
Copy link
Collaborator

@bprusinowski bprusinowski commented Dec 12, 2023

This PR improves the performance of fetching dataset preview by ditching the current logic (using rdf-cube-view-query library) in favor of a custom CONSTRUCT query. It also removes Run SPARQL query button from dataset preview page.

In the future, we might also adapt and extract the logic to be reused when fetching components, to avoid querying the whole shape (and e.g. unit labels separately, as was the case before) and instead focus on the properties that we actually need.

Improvement: one of the largest datasets (Electricity tariff per provider) loads in around 5s on TEST; in the PR, it takes around 0.5s.

Thanks @Rdataflow for the inspiration and providing an example query 🎉

Copy link

vercel bot commented Dec 12, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
visualization-tool ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 13, 2024 9:03am

@bprusinowski bprusinowski force-pushed the perf/optimize-dataset-preview-performance branch from a7f2e51 to 9467c6d Compare February 7, 2024 14:58
@bprusinowski
Copy link
Collaborator Author

bprusinowski commented Feb 12, 2024

@Rdataflow dataset preview for BLW cubes is broken due to errors in the data, see that the landing page is localized while it shouldn't be: https://s.zazuko.com/2ionzw1. It works on TEST, because there we query for cube metadata using the old way, while now we switched to a dedicated query (DataCubeMetadata).

This problem will be addressed in the near future by Zazuko 👍

Copy link
Collaborator

@ptbrowne ptbrowne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks !

@bprusinowski bprusinowski merged commit 8cf7ffc into main Feb 13, 2024
4 of 6 checks passed
@bprusinowski bprusinowski deleted the perf/optimize-dataset-preview-performance branch February 13, 2024 09:32
@Rdataflow
Copy link
Contributor

@bprusinowski well done 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants