Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot range scan #1390

Open
drmingdrmer opened this issue Dec 8, 2021 · 1 comment
Open

Snapshot range scan #1390

drmingdrmer opened this issue Dec 8, 2021 · 1 comment
Labels

Comments

@drmingdrmer
Copy link

Hi Spacejam,
I am a developer from https://github.com/datafuselabs/databend and we has been using sled to store meta-data of the databend-DB cluster. As an embedded high performance kv store, sled fits our needs very well.
Except that in some cases, we need to retrieve a list of key-value pairs, such as a bunch of users.

It would be very nice If sled is able to provide a way to create snapshot view for an application to repeatedly scan it and get the same response.

Is it possible to add such a feature to sled?
And we'd be glad to help on this:DDD

Use Case:

Meta data of a database cluster.
It stores database, table, schema.

Proposed Change:

Range scan on a snapshot view. E.g.:

let s = db.snapshot();

// Expect a snapshot always returns the same result no matter if other key-values are added or removed.
let prev_res = s.range(..);
db.insert(b"a", b"b");
db.remove(b"c", b"d");
assert_eq!(s.range(..), prev_res);

Who Benefits From The Change(s)?

The https://github.com/datafuselabs/databend team and some other guys :D

Alternative Approaches

Add a RWLock to contain the sled db, to prevent writes during db.range(..).

@tv42
Copy link

tv42 commented Mar 27, 2022

Related, it would be great to be able to atomically read a snapshot of the database, and start a subscription that sees changes from that snapshot onward.

tv42 added a commit to tv42/choosy that referenced this issue Mar 27, 2022
Sled's subscription functionality is hard to operate, and at this time
doesn't really work the right way anyway; can't coordinate subscribing
with taking a snapshot of the existing state:
spacejam/sled#1390 (comment)

Since this is a major change, use this opportunity to port from Tide
to Axum, and thus from async-std to Tokio.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants