Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.
/ corefx Public archive

Adding some basic perf tests for Double.TryParse and Single.TryParse #32392

Merged
merged 1 commit into from
Sep 21, 2018
Merged

Adding some basic perf tests for Double.TryParse and Single.TryParse #32392

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions src/System.Runtime/tests/Performance/Perf.Double.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ public class Perf_Double
// NOTE: Consider duplicating any tests added here in Perf.Single.cs

private volatile string _string;
private volatile bool _bool;

[Benchmark]
[InlineData(double.NegativeInfinity, 10_000_000)] // Negative Infinity
Expand Down Expand Up @@ -49,6 +50,40 @@ public void DefaultToString(double number, int innerIterations)
}
}

[Benchmark]
[InlineData("-∞", 10_000_000)] // Negative Infinity
[InlineData("-1.7976931348623157E+308", 100_000)] // Min Negative Normal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how much it matters, but do we want numbers that are larger than 10?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That shouldn't matter as much. The current algorithm is mostly dependent on the length of the input string rather than the actual magnitude of its value.

[InlineData("-3.1415926535897931", 1_000_000)] // Negative pi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about numbers with commas (thousands separator)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also - numbers with white space at the beginning/end. from https://docs.microsoft.com/en-us/dotnet/api/system.double.tryparse

or a string of the form:

[ws][sign][integral-digits,]integral-digits[.[fractional-digits]][e[sign]exponential-digits][ws]

In reply to: 219635010 [](ancestors = 219635010)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These few tests are already adding on 6min 20sec onto the total benchmark time (38 new tests at 10s each). I think there are a number of other scenarios that would be good to test, but the current Benchmark.NET setup does not make that very feasible, so I tried to cover a number of useful/interesting inputs for the most common string format.

Copy link
Member

@eerhardt eerhardt Sep 21, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I tried to cover a number of useful/interesting inputs for the most common string format.

From a cursory look, I think some of your existing ones can be eliminated. For example:

System.Tests.Perf_Single.DefaultTryParse(input: "2.71828175", innerIterations: 1000000)                                                          | Duration | msec |    116     |     86.756 |      3.454 |     82.339 |  104.550
System.Tests.Perf_Single.DefaultTryParse(input: "-2.71828175", innerIterations: 1000000)                                                         | Duration | msec |    106     |     94.820 |      8.288 |     85.799 |  131.644
System.Tests.Perf_Single.DefaultTryParse(input: "3.14159274", innerIterations: 1000000)                                                          | Duration | msec |    108     |     92.847 |      7.307 |     83.224 |  120.448
System.Tests.Perf_Single.DefaultTryParse(input: "-3.14159274", innerIterations: 1000000)                                                         | Duration | msec |    115     |     87.533 |      3.175 |     84.733 |  106.748

Yes, these are important numbers: e and pi, but is parsing these numbers really that much different of a scenario?

Similar comment for:

   System.Tests.Perf_Double.DefaultTryParse(input: "1.7976931348623157E+308", innerIterations: 100000)                                              | Duration | msec |    863     |     11.594 |      0.479 |     11.033 |   15.891
   System.Tests.Perf_Double.DefaultTryParse(input: "-1.7976931348623157E+308", innerIterations: 100000)                                             | Duration | msec |    776     |     12.890 |      1.045 |     11.703 |   22.712
   System.Tests.Perf_Double.DefaultTryParse(input: "2.2250738585072009E-308", innerIterations: 100000)                                              | Duration | msec |    805     |     12.421 |      0.531 |     11.805 |   18.015
   System.Tests.Perf_Double.DefaultTryParse(input: "-2.2250738585072009E-308", innerIterations: 100000)                                             | Duration | msec |    777     |     12.867 |      0.841 |     12.156 |   24.272
   System.Tests.Perf_Double.DefaultTryParse(input: "2.2250738585072014E-308", innerIterations: 100000)                                              | Duration | msec |    845     |     11.839 |      0.536 |     11.271 |   18.168
   System.Tests.Perf_Double.DefaultTryParse(input: "-2.2250738585072014E-308", innerIterations: 100000)                                             | Duration | msec |    827     |     12.096 |      0.389 |     11.574 |   15.560

I think we could probably drop 2-3 from each of those blocks, and add at least 1 scenario with a comma in it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talked with @eerhardt briefly offline.

I definitely agree that we should add more tests, but I am waiting to hear back on if we have or plan to have a better story for "end to end" benchmarks (so that we can actually validate the average perf for a broad range of inputs, or for benchmarks that take longer than 1 second per outer iteration to run).

The current tests where chosen because they have the broadest impact on the current algorithm (overall) and will be the most useful in validating some related changes going in.

[InlineData("-2.7182818284590451", 1_000_000)] // Negative e
[InlineData("-1", 1_000_000)] // Negative One
[InlineData("-2.2250738585072014E-308", 100_000)] // Max Negative Normal
[InlineData("-2.2250738585072009E-308", 100_000)] // Min Negative Subnormal
[InlineData("-4.94065645841247E-324", 100_000)] // Max Negative Subnormal (Negative Epsilon)
[InlineData("-0.0", 10_000_000)] // Negative Zero
[InlineData("NaN", 10_000_000)] // NaN
[InlineData("0", 10_000_000)] // Positive Zero
[InlineData("4.94065645841247E-324", 100_000)] // Min Positive Subnormal (Positive Epsilon)
[InlineData("2.2250738585072009E-308", 100_000)] // Max Positive Subnormal
[InlineData("2.2250738585072014E-308", 100_000)] // Min Positive Normal
[InlineData("1", 1_000_000)] // Positive One
[InlineData("2.7182818284590451", 1_000_000)] // Positive e
[InlineData("3.1415926535897931", 1_000_000)] // Positive pi
[InlineData("1.7976931348623157E+308", 100_000)] // Max Positive Normal
[InlineData("∞", 10_000_000)] // Positive Infinity
public void DefaultTryParse(string input, int innerIterations)
{
foreach (var iteration in Benchmark.Iterations)
{
using (iteration.StartMeasurement())
{
for (int i = 0; i < innerIterations; i++)
{
_bool = double.TryParse(input, out var result);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want (or would it be valuable) to test TryParse that takes a Span<char>?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double.TryParse(string, out double) is already calling the Span overload internally: https://source.dot.net/#System.Private.CoreLib/shared/System/Double.cs,315

}
}
}
}

[Benchmark]
[InlineData("zh", double.NegativeInfinity, 10_000_000)] // Negative Infinity
[InlineData("zh", double.MinValue, 100_000)] // Min Negative Normal
Expand Down
35 changes: 35 additions & 0 deletions src/System.Runtime/tests/Performance/Perf.Single.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ public class Perf_Single
// NOTE: Consider duplicating any tests added here in Perf.Double.cs

private volatile string _string;
private volatile bool _bool;

[Benchmark]
[InlineData(float.NegativeInfinity, 10_000_000)] // Negative Infinity
Expand Down Expand Up @@ -49,6 +50,40 @@ public void DefaultToString(float number, int innerIterations)
}
}

[Benchmark]
[InlineData("-∞", 10_000_000)] // Negative Infinity
[InlineData("-3.40282347E+38", 100_000)] // Min Negative Normal
[InlineData("-3.14159274", 1_000_000)] // Negative pi
[InlineData("-2.71828175", 1_000_000)] // Negative e
[InlineData("-1", 1_000_000)] // Negative One
[InlineData("-1.17549435E-38", 100_000)] // Max Negative Normal
[InlineData("-1.17549421E-38", 100_000)] // Min Negative Subnormal
[InlineData("-1.401298E-45", 100_000)] // Max Negative Subnormal (Negative Epsilon)
[InlineData("-0.0", 10_000_000)] // Negative Zero
[InlineData("NaN", 10_000_000)] // NaN
[InlineData("0", 10_000_000)] // Positive Zero
[InlineData("1.401298E-45", 100_000)] // Min Positive Subnormal (Positive Epsilon)
[InlineData("1.17549421E-38", 100_000)] // Max Positive Subnormal
[InlineData("1.17549435E-38", 100_000)] // Min Positive Normal
[InlineData("1", 1_000_000)] // Positive One
[InlineData("2.71828175", 1_000_000)] // Positive e
[InlineData("3.14159274", 1_000_000)] // Positive pi
[InlineData("3.40282347E+38", 100_000)] // Max Positive Normal
[InlineData("∞", 10_000_000)] // Positive Infinity
public void DefaultTryParse(string input, int innerIterations)
{
foreach (var iteration in Benchmark.Iterations)
{
using (iteration.StartMeasurement())
{
for (int i = 0; i < innerIterations; i++)
{
_bool = double.TryParse(input, out var result);
}
}
}
}

[Benchmark]
[InlineData("zh", float.NegativeInfinity, 10_000_000)] // Negative Infinity
[InlineData("zh", float.MinValue, 100_000)] // Min Negative Normal
Expand Down