-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove mmapSize check, it has wrong value on ARM64 #281
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @yarysh)
I hope this will fix the following critical error that I have seen whenever writing a lot of data to Badger on my Raspberry Pi: Unable to truncate file: "[…]/000001.vlog" error: mremap size mismatch: requested: 271382146 got: 536870912 This error causes data loss (yikes.) It seems to have been fixed in Ristretto already [1] so I'm going to try that. [1] dgraph-io/ristretto#281
I hope this will fix the following critical error that I have seen whenever writing a lot of data to Badger on my Raspberry Pi: Unable to truncate file: "[…]/000001.vlog" error: mremap size mismatch: requested: 271382146 got: 536870912 This error causes data loss (yikes.) It seems to have been fixed in Ristretto already [1] so I'm going to try that. [1] dgraph-io/ristretto#281
@manishrjain could you please release this fix? It has been merged almost half a year ago, and there is still no release with a fix. |
Hi. I’m no longer maintaining these repos, as I’m no longer with Dgraph Labs. We have fork of this repo that we maintain at outcaste-io. Feel free to file an issue there and we can make a release. |
(cherry picked from commit c2901dc)
This PR adds a specific arm64 implementation for the `mremap` function which ignores the second return argument from the syscall. **All other Linux implementations are unchanged.** This PR reworks [an earlier PR](#281) so that the functionality on non-arm64 platforms remains the same. Tested on Linux amd64 and Linux arm64.
In #281 mmap size check has been disabled for the ARM64 arch. But we at [anytype](https://github.com/anyproto) quite often face the same problem on the arm arch on Android devices (armeabi-v7a target) with the [latest badger v4.2.0](https://github.com/dgraph-io/badger/releases/tag/v4.2.0) and [latest ristretto v0.1.1](https://github.com/dgraph-io/ristretto). ```memtables error: while opening fid: 1 error: while updating skiplist error: mremap size mismatch: requested: 1479692 got: 67108864``` This PR extends the #281 to cover all arm architectures. Here is one more mention of this issue dgraph-io/badger#2027 (comment) It would also make sense to create the PR on top of v0.1.1 release, but there is not release branch to base on
In this statement:
mmapAddr, mmapSize, errno := unix.Syscall6(unix.SYS_MREMAP,...)
mmapSize
- it is not an allocated size, at least for ARM64 arch.For ARM64,
mmapSize
will be equal to the old size.Example script: https://gist.github.com/yarysh/2b4db19ebe1236f31ff6780d956628f5
I run this script on AMD64 machine with Ubuntu 20.04, and it returns:
And on ARM64 machine (Raspberry PI and AWS Graviton) with Ubuntu 20.04, it returns:
I have checked man for mremap and it describes returns as:
This issue also affected badger on ARM64: https://discuss.dgraph.io/t/error-mremap-size-mismatch-on-arm64/15333
This change is