-
-
Notifications
You must be signed in to change notification settings - Fork 346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't shuffle each batch, instead reverse them half the time. #1580
Conversation
I consistently see batch shuffling taking 1% CPU on a real-world application, and this is a simple fix to make this overhead disappear. Suggested by @njsmith.
Codecov Report
@@ Coverage Diff @@
## master #1580 +/- ##
=======================================
Coverage 99.69% 99.69%
=======================================
Files 111 111
Lines 13866 13869 +3
Branches 1059 1060 +1
=======================================
+ Hits 13824 13827 +3
Misses 27 27
Partials 15 15
|
The scheduler is still non-deterministic, but exhibit much less variety of behavior, making the test a lot more likely to fail. So we increase the size of the traces.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, thanks!
I don't think we need a newsfragment here, since this isn't user-visible?
Hey @palkeo, it looks like that was the first time we merged one of your PRs! Thanks so much! 🎉 🎂 If you want to keep contributing, we'd love to have you. So, I just sent you an invitation to join the python-trio organization on Github! If you accept, then here's what will happen:
If you want to read more, here's the relevant section in our contributing guide. Alternatively, you're free to decline or ignore the invitation. You'll still be able to contribute as much or as little as you like, and I won't hassle you about joining again. But if you ever change your mind, just let us know and we'll send another invitation. We'd love to have you, but more importantly we want you to do whatever's best for you. If you have any questions, well... I am just a humble Python script, so I probably can't help. But please do post a comment here, or in our chat, or on our forum, whatever's easiest, and someone will help you out! |
I consistently see batch shuffling taking 1% CPU on a real-world
application, and this is a simple fix to make this overhead disappear.
Suggested by @njsmith.