Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize constructor time for tiledb-cloud case #222

Merged
merged 7 commits into from
Jul 15, 2022

Conversation

johnkerl
Copy link
Member

@johnkerl johnkerl commented Jul 13, 2022

Analysis for various read & write operations on tiledb://... URIs reveals that:

  • Core-to-REST-server requests appear in things like tiledb.object_type, tiledb.group.Group, etc.
  • These have server-side latency
  • There are server-side optimizations we can do, certainly, but also on the client side we can batch and cache:
    • Getting group-element name-to-URI mappings takes about half a second per request whether we ask for one group member at a time, or for all group elements at once.
    • Once we've gotten a name-to-URI mapping -- or a construction of an object which contains such -- we can cache the result to avoid subsequent, redundant requests (with, of course, cache-invalidation logic in the right places)

Later note: more in #223.

@johnkerl johnkerl force-pushed the kerl/cloud-soma-ctor-optimize branch from ef6a1bc to 8a251f0 Compare July 15, 2022 04:17
@johnkerl johnkerl force-pushed the kerl/cloud-soma-ctor-optimize branch 7 times, most recently from 6872c43 to becaac5 Compare July 15, 2022 16:22
@johnkerl johnkerl force-pushed the kerl/cloud-soma-ctor-optimize branch from becaac5 to 7971580 Compare July 15, 2022 16:27
@johnkerl johnkerl requested review from aaronwolen and Shelnutt2 and removed request for aaronwolen July 15, 2022 17:02
@johnkerl johnkerl marked this pull request as ready for review July 15, 2022 17:02
@johnkerl johnkerl force-pushed the kerl/cloud-soma-ctor-optimize branch from 1444211 to 4a44a58 Compare July 15, 2022 17:03
@johnkerl johnkerl force-pushed the kerl/cloud-soma-ctor-optimize branch from 4a44a58 to 82f4843 Compare July 15, 2022 17:04
@johnkerl johnkerl changed the title Optimize constructor time for tiledb-cloud case [WIP] Optimize constructor time for tiledb-cloud case Jul 15, 2022
Copy link
Member

@aaronwolen aaronwolen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@johnkerl johnkerl merged commit 8d8c94d into main Jul 15, 2022
@johnkerl johnkerl deleted the kerl/cloud-soma-ctor-optimize branch July 15, 2022 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants