-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
encoding compatible hypercores #24
Comments
Working on this now |
Here's some notes from looking at how the replication in hypercore works from the protocol down to the storage. When we get data from a peer, it gets invoked here, this will be the prompt to intercept the data and encode it / add it to the encoded hypercore This in turn calls _putBuffer on the unencoded hypercore. This bit of code is super complicated to understand. 😅😅😅 Most of it seems to be checking if there's data that's missing, I guess data that we don't already have locally? Then it invokes _verifyandWrite which does some stuff to check if stuff is valid, and then invokes _write to actually write the data. It seems we can hook into this with the Eventually the data for the hypercore is written using the storage instance in the putData method. That will then calculate the offset within the file that it should write the data at and finally invoke A custom random-access-storage thing would work, but we'd need to create the reverse of the Another option would be to subclass Storage and provide custom |
Here's a HackMD with some diagrams talking about how the communication between the compressed / uncompressed hypercores could work. https://hackmd.io/uNQsTqDORmOaUD9-48X13w |
Pausing for today |
Wrote up ideas here: https://hackmd.io/6Wyij7_uTbGxfOSSlJOgZQ |
Boom: https://github.com/RangerMauve/intercept-hypercore-storage This lets us intercept storage events and stuff. Next I'll work on encoders storing encoding data on hosts in a hypertrie, then I'll work on having the host set up a hypercore which will be intercepted to serve data stored in the hypertrie. |
@todo
hyperdrives
are used, make sure the "datdot app" (when a user pastes a hyperdrive address) can figure out all related hypercores and have them seeded by the networkrandom-access-???
module or a random-access patcher module?The text was updated successfully, but these errors were encountered: