Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmarks for performance tuning #70

Open
1 of 3 tasks
neoeinstein opened this issue Jul 7, 2016 · 3 comments
Open
1 of 3 tasks

Add benchmarks for performance tuning #70

neoeinstein opened this issue Jul 7, 2016 · 3 comments
Milestone
Chiron 7

Comments

@neoeinstein
Copy link
Member

neoeinstein commented Jul 7, 2016 •
edited
Loading

  • Decide on a benchmarking library that works well with xUnit
  • Develop a few representative schemas along with Chiron encoders/decoders
    • Serialize/Deserialize
      • Small records with one or two fields.
      • Large records
    • Parse/Format
      • Trivial
      • Large strings
      • Object with many properties
      • Deeply nested objects/arrays
      • Different parsing/formatting strategies
  • Determine metrics by which to determine improvement
@neoeinstein neoeinstein added this to the Chiron 7 milestone Jul 7, 2016
@neoeinstein
Copy link
Member Author

neoeinstein commented Jul 8, 2016 •
edited
Loading

I'm leaning toward NBench at the moment. It requires an external runner, so it doesn't integrate directly with xUnit. Nonetheless, it seems to have some nice features, including memory and GC benchmarking. It's not quite idiomatic F#, though, so each benchmark needs to be implemented with an OO type. I've posted the work-in-progress into the add-benchmarks branch.

neoeinstein added a commit to neoeinstein/chiron that referenced this issue Sep 20, 2016
@neoeinstein
Add timing tests for escaping JSON strings
269fc15 
On my machine, the escaped test case has a runtime of 74ms. The chosen range of up to 5000ms provides a wide range that precludes the pathological case while still being acheivable on underpowered hardware.

Tests should be converted to proper benchmarks in a later PR (see xyncro#70).

Also exposes internals to `Chiron.Tests`.
@neoeinstein
Copy link
Member Author

After more exploration, I've found BenchmarkDotNet which seems to be becoming the preferred benchmark tool. It works well integrated as part of an executable and allows for doing A/B testing.

@kolektiv
Copy link
Member

kolektiv commented Oct 2, 2016

I like BenchmarkDotNet, I've had nice experiences with it elsewhere. More than happy for that to be the story for Chiron!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Chiron 7
Development

No branches or pull requests
2 participants
@kolektiv @neoeinstein