Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add routines for peeking/poking from/to file #64

Open
TerrorJack opened this issue Jul 27, 2016 · 4 comments
Open

Add routines for peeking/poking from/to file #64

TerrorJack opened this issue Jul 27, 2016 · 4 comments

Comments

@TerrorJack
Copy link

Currently Peek/Poke operations can be run on strict ByteStrings or Ptrs. I suggest adding routines that run them given a FilePath, and implement it using mmap, which avoids copying and should have good performance.

@mgsloan
Copy link
Owner

mgsloan commented Jul 28, 2016

Good idea! Could probably use https://hackage.haskell.org/package/mmap-0.5.9/docs/System-IO-MMap.html very directly. Not sure if I want to add mmap as a direct dep, could be worth suggesting in the documentation. The docs make it sound like using this approach for writing the file could be dicey.

How about just decode <$> mmapFileByteString fp Nothing ?

@TerrorJack
Copy link
Author

That would be fine, but works only for peeking (mmapFileByteString maps file in ReadOnly mode). We should use mmapFilePtr and play with raw Ptrs. I'll write a benchmark to see if it is truly beneficial for using mmap when I got spare time.

@TerrorJack
Copy link
Author

@mgsloan Sorry for the late reply. I implemented the operations in a separate package here: https://github.com/TerrorJack/store-file (not on Hackage yet). It can run Peek/Poke operations given a FilePath, and has mmap-based/ByteString-based flavors.

I also implemented a simple test-suite and a benchmark, reading/writing HashMap ByteString ByteString. Surprisingly, benchmark shows mmap-based implementation actually a little bit slower with large examples. Any idea why?

@mgsloan
Copy link
Owner

mgsloan commented Aug 8, 2016

Cool, good stuff! Perhaps the performance difference is because there are no {-# INLINE #-} pragmas?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants