-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[red-knot] fix incremental benchmark #12400
Conversation
|
@@ -138,7 +138,7 @@ fn benchmark_incremental(criterion: &mut Criterion) { | |||
case.fs | |||
.write_file( | |||
SystemPath::new("/src/foo.py"), | |||
format!("{BAR_CODE}\n# A comment\n"), | |||
format!("{FOO_CODE}\n# A comment\n"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the real error here is that we write /src/foo
instead of bar. Because we then touch bar
and not foo
.
The idea is to invalidate a dependency and then re-run checking which should show how well we can cope with local changes.
ac32227
to
5afe2a2
Compare
CodSpeed Performance ReportMerging #12400 will degrade performances by 55.21%Comparing Summary
Benchmarks breakdown
|
This change of course makes the incremental benchmark slower, since we now change the dependency and impact both files, requiring more validation of dependencies. It's still faster than the non-incremental benchmark though, which is good; the results of type inference on bar don't change due to the added comment and we can avoid redoing some work in foo. |
Was it intentional that we rewrite
foo.py
with theBAR_CODE
in the incremental benchmark?