Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared storage cache #4988

Closed

Conversation

ghoshkaj
Copy link
Member

@ghoshkaj ghoshkaj commented Apr 1, 2018

Issue

Use a shared storage cache to compare against performance of the threadlocal cache in #4876. This PR is to test out which is faster and keep the implementation that is faster. Related to cache considerations detailed here.

Tasklist

  • implement singleton pattern to make the cache accessible from global storage
  • ensure that ^ is thread safe. see comment here and here
  • implement mutexes for all functions in the cache (to lock for every read and every write)
  • swap mutex for ReadWriteLock (only lock on write so that reads can still happen in parallel)
  • compare request speeds for this type of cache vs the threadlocal cache in the base PR and keep the implementation that is faster
  • either merge into the base PR or close as not necessary
  • CHANGELOG.md entry (How to write a changelog entry)
  • update relevant Wiki pages
  • add tests (see testing documentation
  • review
  • adjust for comments
  • cherry pick to release branch

Requirements / Relations

#4876

Karen Shea and others added 30 commits January 29, 2018 12:35
Rename module partition to partitioner.
This cultivates naming used in existing modules like extractor,
customizer, etc. - noun vs verb (word partition is both though).
Make entry points of individual pipeline stages responsible for
configuring the task scheduler with requested number of threads
passed in corresponding configuration bundle (ie. follow extractor).
Intersection analysis occupy in osrm::extractor::intersection namespace
and guidance code osrm::guidance
We now build using GCC 7 and Clang 5 and only build
binaries for node 8.x and 4.x.
This PR makes the path of host `.osm.pbf` file clearer.
@ghoshkaj ghoshkaj force-pushed the shared_storage_cache branch from bc1ca99 to e57c949 Compare April 1, 2018 17:18
@ghoshkaj ghoshkaj force-pushed the shared_storage_cache branch from 358ba75 to 181c4c7 Compare April 1, 2018 17:53
ghoshkaj added 10 commits April 1, 2018 18:09
log stuff

comments to self on how to compute durations

moar notes
calculate packed_path_from_source_to_middle

debugging the retrievePackedPathFromSearchSpace function implementation

more debugging

so much clearner and it works!
commentary

answers to the questions I had as comments

everything that was broken last night works this morning! what?!?!

notes

cache is partway working
the computeDurationForEdge method

remove all vestiges of step 5

it works! only 2 test failures to solve!

2 moar bugs sqaushed so 2 more tests fixed!! but found more failing cuke tests!

comment out all std::outs

comment out cache

clean up the code

move vector creation and allocation to outside of loop
…and facade.GetUncompressedReverseDurations

attempt to clean up hack

clean up hack
rebase against vectors->range pr
@ghoshkaj ghoshkaj closed this Apr 1, 2018
@ghoshkaj ghoshkaj deleted the shared_storage_cache branch April 1, 2018 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.