-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GraphBolt] CPU RAM Feature Cache for DiskBasedFeature #7339
Comments
@mfbalin |
@Rhett-Ying io_uring is more efficient and faster compared to using mmap. With io_uring, you need fewer threads to saturate the SSD bandwidth. When it comes to caching, the OS caches pages usually in sizes 4KB, however, feature dimension * dtype_bytes is usually smaller than that. Thus when the OS caches a page, it will cache unnecessary vertex features along with it too. The cache will be less effective because of that. |
And I believe we can use a better caching strategy than the one used inside the Linux kernel. For example, see this paper on a state-of-the-art simple caching policy: https://dl.acm.org/doi/10.1145/3600006.3613147 |
As the
Is it achieved by submit many I/O request to submission queue and wait for completion? |
Yes, that is how io_uring works, you batch your requests and submit them with a single linux system call. When we also have a cache, it will outperform mmap approach significantly. |
I am not sure if it's easy and clean to implement caching policy in app-level. The trade-off on performance improvement and code logic complexity needs to be taken into consideration. @pyynb Please read the paper @mfbalin suggested for caching policy: https://dl.acm.org/doi/10.1145/3600006.3613147. |
Last month, we compared three different cache libraries and various cache eviction policies. Regarding the eviction policies, we found that the hit rate of S3-FIFO cache was higher than LRU, but the time usage was slightly higher. Both of them are significantly better than other eviction methods(see documentation for details). |
Thank you for the preliminary study. |
I have decided to implement a parallel S3-fifo cache implementation in the upcoming weeks. Assigning the issue to myself. |
#7492 implements the s3-fifo caching policy and the FeatureCache classes. The design is made to be easily extendible in case we want to try more caching policies in the future. @frozenbugs @Rhett-Ying |
🚀 Feature
When we use a DiskBasedFeatureStore, we will need to cache frequently accessed items in a CPU cache so that the disk read bandwidth requirements are reduced.
Motivation
Will improve performance immensely on large datasets whose data do not fit the CPU RAM.
CPUFeatureCache
and bug fixes. [1] #7539 implements CPUCachedFeatureThe text was updated successfully, but these errors were encountered: