Skip to content
This repository has been archived by the owner on Mar 22, 2023. It is now read-only.

Radix DRAM caching layer: accessing freed memory in case of OOM #1014

Open
igchor opened this issue Jul 8, 2021 · 0 comments
Open

Radix DRAM caching layer: accessing freed memory in case of OOM #1014

igchor opened this issue Jul 8, 2021 · 0 comments
Labels
radix Needs to be resolved to productize radix Type: Bug

Comments

@igchor
Copy link
Contributor

igchor commented Jul 8, 2021

ISSUE: Radix DRAM caching layer: put/get/remove does not offer read commited guarantee in case of OOM

In heterogenous_radix bg thread inserts elements to radix tree in batch (in a single transaction). If inserting Nth element fails due to OOM all previous inserts will be rolled back. This means that pointers in dram cache will point to memory that is not allocated anymore.

This also violates read committed guarantees in some circumstances. When element is consumed by bg thread, cache entry for that element is marked as possible to evict. If such eviction happens and the bg thread fails due to OOM we lose that element.

One solution to this is to allow "partial commit" - committing operations that succeeded up to the point of OOM. This requires some changes in radix_tee: mainly, all operations which could throw OOM should be done at the beginning of the insert/erase so that the radix tree state stays consistent if we commit the transaction.

@lukaszstolarczuk lukaszstolarczuk added the radix Needs to be resolved to productize radix label Jul 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
radix Needs to be resolved to productize radix Type: Bug
Projects
None yet
Development

No branches or pull requests

2 participants