Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add benchmark script #2359

Merged
merged 1 commit into from
Feb 16, 2024
Merged

add benchmark script #2359

merged 1 commit into from
Feb 16, 2024

Conversation

exterm
Copy link
Contributor

@exterm exterm commented Feb 3, 2024

Similar to what @kddnewton did in parser-prism.

I needed an easy way to compare performance of the different parser options. I think it would make sense for it to live in this repo for now.

These are the results from a run on my local machine:

Prism 0.23.0

ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
Warming up --------------------------------------
 Parser::CurrentRuby     1.000 i/100ms
       Parser::Prism    13.000 i/100ms
Prism::Translation::Parser
                         2.000 i/100ms
          RubyParser     1.000 i/100ms
Prism::Translation::RubyParser
                         2.000 i/100ms
Calculating -------------------------------------
 Parser::CurrentRuby      8.399 (±11.9%) i/s -     42.000 in   5.028823s
       Parser::Prism    108.428 (±15.7%) i/s -    533.000 in   5.054914s
Prism::Translation::Parser
                         25.832 (± 7.7%) i/s -    130.000 in   5.065884s
          RubyParser      7.040 (± 0.0%) i/s -     36.000 in   5.125236s
Prism::Translation::RubyParser
                         28.607 (± 7.0%) i/s -    144.000 in   5.059851s

Comparison:
       Parser::Prism:      108.4 i/s
Prism::Translation::RubyParser:       28.6 i/s - 3.79x  slower
Prism::Translation::Parser:       25.8 i/s - 4.20x  slower
 Parser::CurrentRuby:        8.4 i/s - 12.91x  slower
          RubyParser:        7.0 i/s - 15.40x  slower

Prism 0.24.0

ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
Warming up --------------------------------------
 Parser::CurrentRuby     1.000 i/100ms
       Parser::Prism    25.000 i/100ms
Prism::Translation::Parser
                         2.000 i/100ms
          RubyParser     1.000 i/100ms
Prism::Translation::RubyParser
                         3.000 i/100ms
Calculating -------------------------------------
 Parser::CurrentRuby      7.647 (± 0.0%) i/s -     39.000 in   5.108574s
       Parser::Prism    244.080 (± 4.1%) i/s -      1.225k in   5.026868s
Prism::Translation::Parser
                         25.203 (± 4.0%) i/s -    126.000 in   5.016542s
          RubyParser      7.065 (± 0.0%) i/s -     36.000 in   5.112801s
Prism::Translation::RubyParser
                         29.630 (± 6.8%) i/s -    150.000 in   5.078599s

Comparison:
       Parser::Prism:      244.1 i/s
Prism::Translation::RubyParser:       29.6 i/s - 8.24x  slower
Prism::Translation::Parser:       25.2 i/s - 9.68x  slower
 Parser::CurrentRuby:        7.6 i/s - 31.92x  slower
          RubyParser:        7.1 i/s - 34.55x  slower

@kddnewton
Copy link
Collaborator

Thanks @exterm! Would you mind adding it to bin/prism? We've moved all of the individual scripts into there.

Copy link
Collaborator

@kddnewton kddnewton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just marking as "request changes"

@exterm exterm force-pushed the add-benchmark-script branch from 4b56eef to 9ad7968 Compare February 15, 2024 20:45
@exterm
Copy link
Contributor Author

exterm commented Feb 15, 2024

Done and I also added ruby_parser and its translator to the mix.

@exterm exterm force-pushed the add-benchmark-script branch from 9ad7968 to f09ace2 Compare February 15, 2024 22:04
Copy link
Collaborator

@kddnewton kddnewton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@kddnewton kddnewton merged commit b2b6e2b into ruby:main Feb 16, 2024
49 of 54 checks passed

Benchmark.ips do |x|
x.report("Parser::CurrentRuby") { Parser::CurrentRuby.parse_file(filepath) }
x.report("Parser::Prism") { Prism.parse_file(filepath) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be labeled just Prism as there is no Parser::Prism (it confused me for the results in the PR description).
I was about to make a PR but @kddnewton already fixed it in 40059d3 :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah dang, that's copy-pasta. Thanks for the note, Benoit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants