Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Apache Commons IO ReadAheadInputStream performance #1834

Closed
lfcnassif opened this issue Aug 22, 2023 · 1 comment
Closed

Test Apache Commons IO ReadAheadInputStream performance #1834

lfcnassif opened this issue Aug 22, 2023 · 1 comment
Labels

Comments

@lfcnassif
Copy link
Member

lfcnassif commented Aug 22, 2023

Most of our current IO is synchronous. ReadAheadInputStream provides asynchronous reads, reading bytes ahead in a background thread, so they are already in memory when requested. I'm not sure if it would make processing noticeably faster, because while some threads are blocked on IO, usually others are using CPU. But sometimes I've seen some kind of CPU usage patterns like Hills, opposite to IO usage, when processing large files like unallocated, this situation may benefit of ReadAheadInputStream. Or maybe this would make processing slower because of more threads and context switches, just testing...

@lfcnassif
Copy link
Member Author

lfcnassif commented Aug 24, 2023

Processed a case with 2.2M items and processing time increased in more than 50% (much more than I expected), so I'm closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant