-
Notifications
You must be signed in to change notification settings - Fork 273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Port CoreClr benchmarks to BenchmarkDotNet #36
Merged
Merged
Changes from 18 commits
Commits
Show all changes
107 commits
Select commit
Hold shift + click to select a range
9476fed
copy some coreclr benchmarks
adamsitnik 5071775
port some some benchmarks
adamsitnik f4b81c3
copy some coreclr benchmarks 2
adamsitnik cf5951f
port some some benchmarks 2
adamsitnik df07e67
rename the folder with CoreClr benchmarks to BenchmarksGame
adamsitnik 46070ae
fix SpectralNorm_1 benchmark
adamsitnik 702be05
copy knucleotide benchmarks
adamsitnik 1761381
get knucleotide working
adamsitnik 54ff968
copy RegexRedux
adamsitnik 7b1d96e
make RegexRedux work
adamsitnik 2e40808
copy reverse-complement
adamsitnik 2e4b9b6
make reverse-complement work
adamsitnik 052fe9f
copy all .txt files to bin
adamsitnik ea468a3
update to the latest version
adamsitnik 201ba6b
read the file length, dont hardcode it (it was buggy)
adamsitnik 141f2f8
use Description to give benchmarks some nice display name, but preser…
adamsitnik 79a21f8
remove the Rider files from repo, ignore them
adamsitnik ffc9fb2
dont run more than 20 iterations
adamsitnik 2c5b539
add possibility to run benchmarks for Legacy Jits
adamsitnik 0ec180c
copy all benchstones
adamsitnik 5eff7f1
port Benchstones to BenchmarkDotNet
adamsitnik 475793f
change the default iteration time from 0,5s to 0,25s to run benchmark…
adamsitnik 78bd131
update to the version which supports jagged arrays
adamsitnik be2104e
copy Burgers
adamsitnik 8581484
port Burgers
adamsitnik 79675a2
copy DefaultEqualityComparerPerf
adamsitnik 5e00870
port DefaultEqualityComparerPerf
adamsitnik cf2a12d
copy FractalPerf
adamsitnik eafa45c
port FractalPerf
adamsitnik 3cf971a
copy Inlining benchmarks
adamsitnik 5ecfcfc
port Inlining benchmarks
adamsitnik 2e64eab
rename Jit folder to Devirtualization to match the coreclr folder str…
adamsitnik 121d8b9
copy SearchLoops
adamsitnik 945a91f
port SearchLoops
adamsitnik e2614df
copy Linq
adamsitnik 5a9754e
port Linq
adamsitnik b6190e0
copy perflab
adamsitnik a79729c
port BlockCopyPerf
adamsitnik 7a09b0f
port CastingPerf
adamsitnik d4534e5
port CastingPerf2
adamsitnik 902832a
port DelegatePerf
adamsitnik 0640e17
remove Program.cs
adamsitnik 518fe86
port EnumPerf
adamsitnik 1959317
fix the benchmarks
adamsitnik f7f0ae9
port StackWalk
adamsitnik 0ad84ad
port ThreadingPerf
adamsitnik 0324338
port LowLevelPerf
adamsitnik fff3e4e
port ReflectionPerf
adamsitnik bbd6468
copy V8.Crypto and V8.Richards
adamsitnik e3e4bbf
port V8.Crypto
adamsitnik 9cd02f8
port V8.Richards
adamsitnik e6a79b5
copy SpanBench
adamsitnik a838265
port Span/Indexer
adamsitnik 6e6bb8f
port SpanBench
adamsitnik 7a19d6f
update dependencies (which contain required fixes)
adamsitnik 26ff77e
copy SIMD benchmarks
adamsitnik f0cc07a
bump .NET to 4.6.1 so we can use Vector.LessThanOrEqualAny
adamsitnik 57f3f84
port ConsoleMandel
adamsitnik 1a5384f
port RayTracerBench
adamsitnik 2eb2ea1
port SeekUnroll
adamsitnik ef09660
move all internal types of RayTracer to a separate namespace to avoid…
adamsitnik 94ea670
move all the CoreCLR benchmarks to coreclr subfolder
adamsitnik fa0083b
copy CscBench
adamsitnik adee6ce
port CscBench
adamsitnik c0eb4a5
copy SciMark
adamsitnik 128d584
port SciMark
adamsitnik a3a5e33
copy ByteMark
adamsitnik c79f955
port ByteMark
adamsitnik 4aafc15
copy Math
adamsitnik 4658e1d
port Math
adamsitnik 5404062
dont try to compile Single type benchmarks which use MathF which does…
adamsitnik 225d09d
fix line endings ;)
adamsitnik 655fc1c
group the benchmarks into categories, allow filtering and joining int…
adamsitnik 572c2ab
export to json (for BenchView intergration purpose), allow users to s…
adamsitnik cff83e7
always show Min and Max column to mimic xunit-performance behaviour
adamsitnik 09f88e7
assign the right type to a category (the one which actually contains …
adamsitnik be7d67d
the benchmarks should not be static
adamsitnik f2d14ce
read the array from field to preserve old benchmark id
adamsitnik fad4695
don't use argument, keep old benchmark full name
adamsitnik e029abb
Merge remote-tracking branch 'upstream/master' into coreClrBdn
adamsitnik f582530
get correct allocated bytes after a bug fix in BenchmarkDotNet
adamsitnik d62beb0
don't use argument, keep old benchmark full name + the value was 16, …
adamsitnik 2387962
missing * 2!!
adamsitnik e891e79
update to BDN which uses full type name for exported results (we have…
adamsitnik fdf84dc
move the arguments to fields to match existing benchmark id
adamsitnik 3d5254c
don't use argument, keep old benchmark full name
adamsitnik 8df41a9
expectedSum argument needs to remain to keep old benchmark id in Benc…
adamsitnik 52e6d8e
don't use argument, keep old benchmark full name
adamsitnik c3a4ed0
move the arguments to fields to match existing benchmark id
adamsitnik 5be5f96
add missing category
adamsitnik ed48860
use Params from BDN to express things that were arguments used to set…
adamsitnik 66cc17f
should be a part of previous commit
adamsitnik ccba0e1
don't make PerfLabTests nano-benchmarks, it would blow up the scaling…
adamsitnik e175d9e
make sure ALL ids are the same
adamsitnik 503e8d8
ResultsValidator - small helper utility to print a diff, I will remov…
adamsitnik a18473e
remove the workaround for handling whitespaced in benchmarks Ids, pro…
adamsitnik 0a0cd71
allow the users to specify outlier removal mode from console args
adamsitnik afadbd8
add median to the default columns
adamsitnik 56bec57
allow the users to test COMPlus_JitAlignLoop 0 vs 1 and affinity set …
adamsitnik d5c3bf0
make sure JIT does not optimize the benchmarks to empty loops!
adamsitnik 84e0822
update to latest BDN which gives us some nice attributes
adamsitnik e38e951
apply some non-default settings to make the results from multimodal b…
adamsitnik cd0fa8f
add comments to benchmarks whic hare very dependent on loop alignment
adamsitnik 101bc24
fix ObjectGetTypeNoBoxing, disable ObjectGetType which is getting opt…
adamsitnik d0e0250
change the order of exported columns in CSV to make it more easy to c…
adamsitnik 21da2d6
rename console attributes, don't introduce new standards ;) + don't w…
adamsitnik 14013d3
add docs for test alignment and test affinity + fix a bug in argument…
adamsitnik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
using System; | ||
using System.IO; | ||
|
||
namespace BenchmarksGame | ||
{ | ||
internal static class InputFileHelper | ||
{ | ||
internal static string FindInputFile(string inputFile) | ||
{ | ||
if (string.IsNullOrEmpty(inputFile)) | ||
throw new ArgumentNullException(nameof(inputFile)); | ||
|
||
var fullPath = GetFullPath(inputFile); | ||
if (!File.Exists(fullPath)) | ||
throw new FileNotFoundException("Unable to find input file.", inputFile); | ||
|
||
return fullPath; | ||
} | ||
|
||
internal static int GetFileLength(string filePath) => (int) new FileInfo(filePath).Length; | ||
|
||
private static string GetFullPath(string fileName) | ||
=> Path.Combine(Path.GetDirectoryName(typeof(InputFileHelper).Assembly.Location), "BenchmarksGame", "Inputs", fileName); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
// Helper functionality to locate inputs and find outputs for | ||
// k-nucleotide benchmark in CoreCLR test harness | ||
|
||
using System; | ||
using System.IO; | ||
using System.Text; | ||
|
||
namespace BenchmarksGame | ||
{ | ||
class NucleotideHelpers | ||
{ | ||
public string InputFile; | ||
public int[] expectedCountLetter; | ||
public int[] expectedCountPairs; | ||
public int[] expectedCountFragments; | ||
public int[][] expectedFrequencies; | ||
|
||
public NucleotideHelpers(bool bigInput) | ||
{ | ||
if (bigInput) | ||
{ | ||
InputFile = InputFileHelper.FindInputFile("knucleotide-input-big.txt"); | ||
expectedCountLetter = new int[] { 302923, 301375, 198136, 197566 }; | ||
expectedCountPairs = new int[] { 91779, 91253, 91225, 90837, 60096, 60030, 59889, 59795, 59756, 59713, 59572, 59557, 39203, 39190, 39081, 39023 }; | ||
expectedCountFragments = new int[] { 11765, 3572, 380, 7, 7 }; | ||
} | ||
else | ||
{ | ||
InputFile = InputFileHelper.FindInputFile("knucleotide-input.txt"); | ||
expectedCountLetter = new int[] { 1576, 1480, 974, 970 }; | ||
expectedCountPairs = new int[] { 496, 480, 470, 420, 316, 315, 310, 302, 298, 292, 273, 272, 202, 201, 185, 167 }; | ||
expectedCountFragments = new int[] { 54, 24, 4, 0, 0 }; | ||
} | ||
expectedFrequencies = new int[][] { expectedCountLetter, expectedCountPairs }; | ||
} | ||
|
||
|
||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
// Helper functionality to locate inputs and find outputs for | ||
// regex-redux benchmark in CoreCLR test harness | ||
|
||
using System; | ||
using System.IO; | ||
using System.Text; | ||
|
||
namespace BenchmarksGame | ||
{ | ||
class RegexReduxHelpers | ||
{ | ||
public string InputFile; | ||
public int ExpectedLength; | ||
|
||
public RegexReduxHelpers(bool bigInput) | ||
{ | ||
if (bigInput) | ||
{ | ||
InputFile = InputFileHelper.FindInputFile("regexdna-input25000.txt"); | ||
ExpectedLength = 136381; | ||
} | ||
else | ||
{ | ||
InputFile = InputFileHelper.FindInputFile("regexdna-input25.txt"); | ||
ExpectedLength = 152; | ||
} | ||
} | ||
} | ||
} |
36 changes: 36 additions & 0 deletions
36
src/benchmarks/BenchmarksGame/Inputs/ReverseComplementHelpers.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
// Helper functionality to locate inputs and find outputs for | ||
// reverse-complement benchmark in CoreCLR test harness | ||
|
||
using System; | ||
using System.IO; | ||
using System.Text; | ||
|
||
namespace BenchmarksGame | ||
{ | ||
class ReverseComplementHelpers | ||
{ | ||
public string InputFile; | ||
public int FileLength; | ||
public string CheckSum; | ||
|
||
public ReverseComplementHelpers(bool bigInput) | ||
{ | ||
if (bigInput) | ||
{ | ||
InputFile = InputFileHelper.FindInputFile("revcomp-input25000.txt"); | ||
CheckSum = "61-A4-CC-6D-15-8D-26-77-88-93-4F-E2-29-A2-8D-FB"; | ||
} | ||
else | ||
{ | ||
InputFile = InputFileHelper.FindInputFile("revcomp-input25.txt"); | ||
CheckSum = "62-45-8E-09-2E-89-A0-69-8C-17-F5-D8-C7-63-5B-50"; | ||
} | ||
|
||
FileLength = InputFileHelper.GetFileLength(InputFile); | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it need to be
int
? Can we keeplong
instead? The API user can cast the result when needed. Thoughts?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure thing, I added the cast here because it was hardcoded to an
int
previously