Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmarks for Utf8JsonReader.Get* #1381

Merged
merged 5 commits into from
Jul 2, 2020
Merged

Add benchmarks for Utf8JsonReader.Get* #1381

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f229919
Add benchmarks for Utf8JsonReader.Get*
jozkee Jun 26, 2020
dc83ab7
Address feedback
jozkee Jun 26, 2020
8f1b2bb
Add scenarios using default value
jozkee Jun 30, 2020
cfa56f9
Address suggestion from adamsitnik
jozkee Jun 30, 2020
b663b52
Remove ReadToEnd* methods, perform only one read on benchmarks, and a…
jozkee Jul 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
236 changes: 236 additions & 0 deletions src/benchmarks/micro/libraries/System.Text.Json/Utf8JsonReader/Perf.Get.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using BenchmarkDotNet.Attributes;
using MicroBenchmarks;
using System.Linq;

namespace System.Text.Json.Tests
{
[BenchmarkCategory(Categories.Libraries, Categories.JSON)]
public class Perf_Get
{
private static readonly byte[] _jsonIntegerNumberBytes = GetJsonBytes(123);
private static readonly byte[] _jsonDecimalNumberBytes = GetJsonBytes(123.456f);
private static readonly byte[] _jsonStringBytes = GetJsonBytes("\"The quick brown fox jumps over the lazy dog.\"");
private static readonly byte[] _jsonGuidBytes = GetJsonBytes($"\"{Guid.Empty}\"");
private static readonly byte[] _jsonDateTimeBytes = GetJsonBytes($"\"{DateTime.MaxValue:O}\"");
private static readonly byte[] _jsonDateTimeOffsetBytes = GetJsonBytes($"\"{DateTimeOffset.MaxValue:O}\"");

private static byte[] GetJsonBytes<T>(T elem)
{
return Encoding.UTF8.GetBytes(elem.ToString());
}
jozkee marked this conversation as resolved.
Show resolved Hide resolved

[Benchmark]
public byte GetByte()
{
byte result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetByte();
}
return result;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jozkee do you know what part of the total time is spent for the creation of Utf8JsonReader, calling Read and calling GetByte()? What is the typical usage of reader.GetX() methods?

I am asking because the creation of the reader and call to reader.Read might be more expensive than the call to GetX which we are trying to measure here. If this is true, we should reconsider rewriting the benchmark to something like:

public sbyte ReadAndGetSBytes()
{
    var reader = new Utf8JsonReader(_jsonSByteBytes); // _jsonSByteBytes should contain more than 1 SByte (100 should be enough)
    sbyte result = default;
    while (reader.Read())
    {
        result += reader.GetSByte();
    }
    return result;
}

this should allow for removal of UseLargeValue and hence reducing the number of new benchmarks and the time it takes to run them by half. Please see https://github.com/dotnet/performance/blob/master/docs/microbenchmark-design-guidelines.md#Test-Cases for more

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, for GetInt32, ~80% time is spent in the first two lines:

[Benchmark]
public void CreateAndReadInt32()
{
    var reader = new Utf8JsonReader(_jsonInt32Bytes);
    reader.Read();
}
Method UseLargeValue Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
CreateAndReadInt32 False 39.95 ns 0.548 ns 0.512 ns 39.81 ns 39.42 ns 41.11 ns - - - -
GetInt32 False 46.41 ns 0.302 ns 0.283 ns 46.47 ns 45.90 ns 46.81 ns - - - -
CreateAndReadInt32 True 49.73 ns 0.211 ns 0.187 ns 49.71 ns 49.48 ns 50.02 ns - - - -
GetInt32 True 62.31 ns 0.647 ns 0.573 ns 62.42 ns 60.74 ns 63.15 ns - - - -

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I added ReadToEnd* benchmarks in order to measure how much time is spent in traversing the 100 elements of the json.

Copy link
Member Author

@jozkee jozkee Jul 2, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another approach that we could take to avoid having the ReadtoEnd* methods could be just call reader.GetInt32() over the same token a hundred times, that could remove the overhead of reader.Read().


[Benchmark]
public sbyte GetSByte()
{
sbyte result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetSByte();
}
return result;
}

[Benchmark]
public short GetInt16()
{
short result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetInt16();
}
return result;
}

[Benchmark]
public int GetInt32()
{
int result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetInt32();
}
return result;
}

[Benchmark]
public long GetInt64()
{
long result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetInt64();
}
return result;
}

[Benchmark]
public ushort GetUInt16()
{
ushort result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetUInt16();
}
return result;
}

[Benchmark]
public uint GetUInt32()
{
uint result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetUInt32();
}
return result;
}

[Benchmark]
public ulong GetUInt64()
{
ulong result = 0;
var reader = new Utf8JsonReader(_jsonIntegerNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetUInt64();
}
return result;
}

[Benchmark]
public float GetSingle()
{
float result = 0;
var reader = new Utf8JsonReader(_jsonDecimalNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetSingle();
jozkee marked this conversation as resolved.
Show resolved Hide resolved
}
return result;
}

[Benchmark]
public double GetDouble()
{
double result = 0;
var reader = new Utf8JsonReader(_jsonDecimalNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetDouble();
}
return result;
}

[Benchmark]
public decimal GetDecimal()
{
decimal result = 0;
var reader = new Utf8JsonReader(_jsonDecimalNumberBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result += reader.GetDecimal();
}
return result;
}

[Benchmark]
public DateTime GetDateTime()
{
DateTime result = default;
var reader = new Utf8JsonReader(_jsonDateTimeBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result = reader.GetDateTime();
}
return result;
}

[Benchmark]
public DateTimeOffset GetDateTimeOffset()
{
DateTimeOffset result = default;
var reader = new Utf8JsonReader(_jsonDateTimeOffsetBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result = reader.GetDateTimeOffset();
}
return result;
}

[Benchmark]
public Guid GetGuid()
{
Guid result = default;
var reader = new Utf8JsonReader(_jsonGuidBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result = reader.GetGuid();
}
return result;
}

[Benchmark]
public string GetString()
{
string result = default;
var reader = new Utf8JsonReader(_jsonStringBytes);
reader.Read();

for (int i = 0; i < 100; i++)
{
result = reader.GetString();
}
return result;
}
}
}