Create Roslyn code analyzer to streamline review of proper usage of format/parse methods for numeric types #924
Labels
is:feature
is:task
A chore to be done
pri:normal
up-for-grabs
This issue is open to be worked on by anyone
Milestone
Is there an existing issue for this?
Task description
Lucene almost exclusively uses invariant culture when converting numeric data types to/from strings. This is because the methods in Java that do the conversion use invariant culture by default. However, the default in .NET is to use the current culture.
In Java, there are also two ways to provide culture-specific formatting:
String.format()
MessageFormat
Generally speaking, we want to be able to maintain compatibility with Lucene. This means that most methods should be designed to format/parse in the invariant culture unless Lucene uses a specific culture. However,
ToString()
is a special case that should both default to the current culture and have an overload that acceptsIFormatProvider
in many/most cases to match the expected behavior in .NET. Tests should then be set up to pass invariant culture to match the default behavior of Lucene. There may be other rare situations where adding a method overload other thanToString()
that acceptsIFormatProvider
makes sense.Unfortunately, the Find and Replace tools in the IDE are inadequate for reviewing these methods because many of the methods to review are named
ToString
and we need to be able to differentiate betweenToString()
methods that convert numeric types to strings and those that don't. So, we will need a Roslyn code analyzer for the review task to target these method calls more narrowly.We already have a project set up for code analyzers for internal use: https://github.com/NightOwl888/lucenenet-codeanalysis-dev where the code analyzers can be added. For now, it is installed using a VSIX installer and we need to manually ensure the assembly version 1, assembly version 2, and manifest version are updated on each code change so Visual Studio will properly upgrade the analyzer.
We should definitely have at least 3 separate analyzers with different IDs to differentiate between the 3 cases as described below. It may be advantageous to break them up further into logical groups (parse vs format or BCL vs J2N, for example), but we should avoid getting too fine grained, as that will mean there are more buckets to analyze for the review.
Current culture overloads (.NET Default)
Use of these overloads is almost always the wrong choice outside of
ToString()
methods of Lucene classes. So, an analyzer should be created to locate calls to the following members.String to Number without IFormatProvider Overload:
int.Parse(string)
long.Parse(string)
double.Parse(string)
float.Parse(string)
decimal.Parse(string)
byte.Parse(string)
sbyte.Parse(string)
short.Parse(string)
uint.Parse(string)
ulong.Parse(string)
ushort.Parse(string)
String to Number with NumberStyles and without IFormatProvider Overload:
int.Parse(string, NumberStyles)
long.Parse(string, NumberStyles)
double.Parse(string, NumberStyles)
float.Parse(string, NumberStyles)
decimal.Parse(string, NumberStyles)
byte.Parse(string, NumberStyles)
sbyte.Parse(string, NumberStyles)
short.Parse(string, NumberStyles)
uint.Parse(string, NumberStyles)
ulong.Parse(string, NumberStyles)
ushort.Parse(string, NumberStyles)
Number to String without IFormatProvider Overload:
int.ToString()
long.ToString()
double.ToString()
float.ToString()
decimal.ToString()
byte.ToString()
sbyte.ToString()
short.ToString()
uint.ToString()
ulong.ToString()
ushort.ToString()
int.ToString(string)
long.ToString(string)
double.ToString(string)
float.ToString(string)
decimal.ToString(string)
byte.ToString(string)
sbyte.ToString(string)
short.ToString(string)
uint.ToString(string)
ulong.ToString(string)
ushort.ToString(string)
J2N.Numerics.Int32.ToString()
J2N.Numerics.Int64.ToString()
J2N.Numerics.Double.ToString()
J2N.Numerics.Single.ToString()
J2N.Numerics.Byte.ToString()
J2N.Numerics.SByte.ToString()
J2N.Numerics.Int16.ToString()
J2N.Numerics.Int32.ToString(int)
J2N.Numerics.Int64.ToString(long)
J2N.Numerics.Double.ToString(double)
J2N.Numerics.Single.ToString(float)
J2N.Numerics.Byte.ToString(byte)
J2N.Numerics.SByte.ToString(sbyte)
J2N.Numerics.Int16.ToString(short)
J2N.Numerics.Int32.ToString(string)
J2N.Numerics.Int64.ToString(string)
J2N.Numerics.Double.ToString(string)
J2N.Numerics.Single.ToString(string)
J2N.Numerics.Byte.ToString(string)
J2N.Numerics.SByte.ToString(string)
J2N.Numerics.Int16.ToString(string)
J2N.Numerics.Int32.ToString(int, string)
J2N.Numerics.Int64.ToString(long, string)
J2N.Numerics.Double.ToString(double, string)
J2N.Numerics.Single.ToString(float, string)
J2N.Numerics.Byte.ToString(byte, string)
J2N.Numerics.SByte.ToString(sbyte, string)
J2N.Numerics.Int16.ToString(short, string)
Additional Methods:
Convert.ToInt32(string)
Convert.ToInt64(string)
Convert.ToDouble(string)
Convert.ToSingle(string)
Convert.ToDecimal(string)
Convert.ToByte(string)
Convert.ToSByte(string)
Convert.ToInt16(string)
Convert.ToUInt32(string)
Convert.ToUInt64(string)
Convert.ToUInt16(string)
Convert.ToString(int)
Convert.ToString(long)
Convert.ToString(double)
Convert.ToString(float)
Convert.ToString(decimal)
Convert.ToString(byte)
Convert.ToString(sbyte)
Convert.ToString(short)
Convert.ToString(uint)
Convert.ToString(ulong)
Convert.ToString(ushort)
String.Format(string, object)
String.Format(string, object, object)
String.Format(string, object, object, object)
String.Format(string, object[])
TryParse:
int.TryParse(string, out int)
long.TryParse(string, out long)
double.TryParse(string, out double)
float.TryParse(string, out float)
decimal.TryParse(string, out decimal)
byte.TryParse(string, out byte)
sbyte.TryParse(string, out sbyte)
short.TryParse(string, out short)
uint.TryParse(string, out uint)
ulong.TryParse(string, out ulong)
ushort.TryParse(string, out ushort)
int.TryParse(ReadOnlySpan<char>, out int)
long.TryParse(ReadOnlySpan<char>, out long)
double.TryParse(ReadOnlySpan<char>, out double)
float.TryParse(ReadOnlySpan<char>, out float)
decimal.TryParse(ReadOnlySpan<char>, out decimal)
byte.TryParse(ReadOnlySpan<char>, out byte)
sbyte.TryParse(ReadOnlySpan<char>, out sbyte)
short.TryParse(ReadOnlySpan<char>, out short)
uint.TryParse(ReadOnlySpan<char>, out uint)
ulong.TryParse(ReadOnlySpan<char>, out ulong)
ushort.TryParse(ReadOnlySpan<char>, out ushort)
J2N.Numerics.Int32.TryParse(string, out int)
J2N.Numerics.Int64.TryParse(string, out long)
J2N.Numerics.Double.TryParse(string, out double)
J2N.Numerics.Single.TryParse(string, out float)
J2N.Numerics.Byte.TryParse(string, out byte)
J2N.Numerics.SByte.TryParse(string, out sbyte)
J2N.Numerics.Int16.TryParse(string, out short)
J2N.Numerics.Int32.TryParse(ReadOnlySpan<char>, out int)
J2N.Numerics.Int64.TryParse(ReadOnlySpan<char>, out long)
J2N.Numerics.Double.TryParse(ReadOnlySpan<char>, out double)
J2N.Numerics.Single.TryParse(ReadOnlySpan<char>, out float)
J2N.Numerics.Byte.TryParse(ReadOnlySpan<char>, out byte)
J2N.Numerics.SByte.TryParse(ReadOnlySpan<char>, out sbyte)
J2N.Numerics.Int16.TryParse(ReadOnlySpan<char>, out short)
Implicit Formatting during Concatenation
All strings that are concatenated with expressions that return numeric types will need to be analyzed as well.
Example
The Java line:
Would typically be ported as:
But should actually be using the invariant culture to match Java:
Implicit Formatting during String Interpolation
As above, if we have converted any lines to use string interpolation, we will need to account for the fact that .NET automatically formats numeric types in the current culture.
Should instead be:
Note that we can also use
string.Create(CultureInfo.InvariantCulture, $"...");
orFormattableString.Invariant()
, where supported.Explicit Culture Overloads
Use of these overloads is usually required and the typical case is to pass
CultureInfo.InvariantCulture
as the parameter.CultureInfo.InvariantCulture
, there should be a warning that is always enabled and we should suppress it to override in special cases.CultureInfo.InvariantCulture
, there should be a warning that is disabled by default that we can enable to review these calls.Number to String with IFormatProvider:
int.ToString(IFormatProvider)
long.ToString(IFormatProvider)
double.ToString(IFormatProvider)
float.ToString(IFormatProvider)
decimal.ToString(IFormatProvider)
byte.ToString(IFormatProvider)
sbyte.ToString(IFormatProvider)
short.ToString(IFormatProvider)
uint.ToString(IFormatProvider)
ulong.ToString(IFormatProvider)
ushort.ToString(IFormatProvider)
J2N.Numerics.Int32.ToString(IFormatProvider)
J2N.Numerics.Int64.ToString(IFormatProvider)
J2N.Numerics.Double.ToString(IFormatProvider)
J2N.Numerics.Single.ToString(IFormatProvider)
J2N.Numerics.Byte.ToString(IFormatProvider)
J2N.Numerics.SByte.ToString(IFormatProvider)
J2N.Numerics.Int16.ToString(IFormatProvider)
J2N.Numerics.Int32.ToString(int, IFormatProvider)
J2N.Numerics.Int64.ToString(long, IFormatProvider)
J2N.Numerics.Double.ToString(double, IFormatProvider)
J2N.Numerics.Single.ToString(single, IFormatProvider)
J2N.Numerics.Byte.ToString(byte, IFormatProvider)
J2N.Numerics.SByte.ToString(sbyte, IFormatProvider)
J2N.Numerics.Int16.ToString(short, IFormatProvider)
Number to String with Format String Overload:
int.ToString(string format, IFormatProvider)
long.ToString(string format, IFormatProvider)
double.ToString(string format, IFormatProvider)
float.ToString(string format, IFormatProvider)
decimal.ToString(string format, IFormatProvider)
byte.ToString(string format, IFormatProvider)
sbyte.ToString(string format, IFormatProvider)
short.ToString(string format, IFormatProvider)
uint.ToString(string format, IFormatProvider)
ulong.ToString(string format, IFormatProvider)
ushort.ToString(string format, IFormatProvider)
J2N.Numerics.Int32.ToString(string format, IFormatProvider)
J2N.Numerics.Int64.ToString(string format, IFormatProvider)
J2N.Numerics.Double.ToString(string format, IFormatProvider)
J2N.Numerics.Single.ToString(string format, IFormatProvider)
J2N.Numerics.Byte.ToString(string format, IFormatProvider)
J2N.Numerics.SByte.ToString(string format, IFormatProvider)
J2N.Numerics.Int16.ToString(string format, IFormatProvider)
J2N.Numerics.Int32.ToString(int, string format, IFormatProvider)
J2N.Numerics.Int64.ToString(long, string format, IFormatProvider)
J2N.Numerics.Double.ToString(double, string format, IFormatProvider)
J2N.Numerics.Single.ToString(single, string format, IFormatProvider)
J2N.Numerics.Byte.ToString(byte, string format, IFormatProvider)
J2N.Numerics.SByte.ToString(sbyte, string format, IFormatProvider)
J2N.Numerics.Int16.ToString(short, string format, IFormatProvider)
Number to String with IFormatProvider and Format String using TryFormat:
int.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
long.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
double.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
float.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
decimal.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
byte.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
sbyte.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
short.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
uint.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
ulong.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
ushort.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Int32.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Int64.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Double.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Single.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Byte.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.SByte.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Int16.TryFormat(Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Int32.TryFormat(int, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Int64.TryFormat(long, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Double.TryFormat(double, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Single.TryFormat(float, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Byte.TryFormat(byte, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.SByte.TryFormat(sbyte, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
J2N.Numerics.Int16.TryFormat(short, Span<char>, out int, ReadOnlySpan<char> = default, IFormatProvider = default)
String to Number with IFormatProvider and NumberStyles Overload:
int.Parse(string, NumberStyles, IFormatProvider)
long.Parse(string, NumberStyles, IFormatProvider)
double.Parse(string, NumberStyles, IFormatProvider)
float.Parse(string, NumberStyles, IFormatProvider)
decimal.Parse(string, NumberStyles, IFormatProvider)
byte.Parse(string, NumberStyles, IFormatProvider)
sbyte.Parse(string, NumberStyles, IFormatProvider)
short.Parse(string, NumberStyles, IFormatProvider)
uint.Parse(string, NumberStyles, IFormatProvider)
ulong.Parse(string, NumberStyles, IFormatProvider)
ushort.Parse(string, NumberStyles, IFormatProvider)
int.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
long.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
double.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
float.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
decimal.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
byte.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
sbyte.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
short.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
uint.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
ulong.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
ushort.Parse(ReadOnlySpan<char>, NumberStyles, IFormatProvider)
J2N.Numerics.Int32.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.Int64.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.Double.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.Single.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.Byte.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.SByte.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.Int16.Parse(string, NumberStyle, IFormatProvider)
J2N.Numerics.Int32.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
J2N.Numerics.Int64.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
J2N.Numerics.Double.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
J2N.Numerics.Single.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
J2N.Numerics.Byte.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
J2N.Numerics.SByte.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
J2N.Numerics.Int16.Parse(ReadOnlySpan<char>, NumberStyle, IFormatProvider)
Custom Parsing with IFormatProvider and NumberStyles:
int.TryParse(string, NumberStyles, IFormatProvider, out int)
long.TryParse(string, NumberStyles, IFormatProvider, out long)
double.TryParse(string, NumberStyles, IFormatProvider, out double)
float.TryParse(string, NumberStyles, IFormatProvider, out float)
decimal.TryParse(string, NumberStyles, IFormatProvider, out decimal)
byte.TryParse(string, NumberStyles, IFormatProvider, out byte)
sbyte.TryParse(string, NumberStyles, IFormatProvider, out sbyte)
short.TryParse(string, NumberStyles, IFormatProvider, out short)
uint.TryParse(string, NumberStyles, IFormatProvider, out uint)
ulong.TryParse(string, NumberStyles, IFormatProvider, out ulong)
ushort.TryParse(string, NumberStyles, IFormatProvider, out ushort)
int.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out int)
long.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out long)
double.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out double)
float.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out float)
decimal.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out decimal)
byte.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out byte)
sbyte.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out sbyte)
short.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out short)
uint.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out uint)
ulong.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out ulong)
ushort.TryParse(ReadOnlySpan<char>, NumberStyles, IFormatProvider, out ushort)
J2N.Numerics.Int32.TryParse(string, NumberStyle, IFormatProvider, out int)
J2N.Numerics.Int64.TryParse(string, NumberStyle, IFormatProvider, out long)
J2N.Numerics.Double.TryParse(string, NumberStyle, IFormatProvider, out double)
J2N.Numerics.Single.TryParse(string, NumberStyle, IFormatProvider, out float)
J2N.Numerics.Byte.TryParse(string, NumberStyle, IFormatProvider, out byte)
J2N.Numerics.SByte.TryParse(string, NumberStyle, IFormatProvider, out sbyte)
J2N.Numerics.Int16.TryParse(string, NumberStyle, IFormatProvider, out short)
J2N.Numerics.Int32.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out int)
J2N.Numerics.Int64.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out long)
J2N.Numerics.Double.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out double)
J2N.Numerics.Single.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out float)
J2N.Numerics.Byte.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out byte)
J2N.Numerics.SByte.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out sbyte)
J2N.Numerics.Int16.TryParse(ReadOnlySpan<char>, NumberStyle, IFormatProvider, out short)
Convert class Number to String with IFormatProvider:
Convert.ToString(int, IFormatProvider)
Convert.ToString(long, IFormatProvider)
Convert.ToString(double, IFormatProvider)
Convert.ToString(float, IFormatProvider)
Convert.ToString(decimal, IFormatProvider)
Convert.ToString(byte, IFormatProvider)
Convert.ToString(sbyte, IFormatProvider)
Convert.ToString(short, IFormatProvider)
Convert.ToString(uint, IFormatProvider)
Convert.ToString(ulong, IFormatProvider)
Convert.ToString(ushort, IFormatProvider)
Convert class String to Number with IFormatProvider:
Convert.ToInt32(string, IFormatProvider)
Convert.ToInt64(string, IFormatProvider)
Convert.ToDouble(string, IFormatProvider)
Convert.ToSingle(string, IFormatProvider)
Convert.ToDecimal(string, IFormatProvider)
Convert.ToByte(string, IFormatProvider)
Convert.ToSByte(string, IFormatProvider)
Convert.ToInt16(string, IFormatProvider)
Convert.ToUInt32(string, IFormatProvider)
Convert.ToUInt64(string, IFormatProvider)
Convert.ToUInt16(string, IFormatProvider)
String.Format()
String.Format(IFormatProvider, string, object)
String.Format(IFormatProvider, string, object, object)
String.Format(IFormatProvider, string, object, object, object)
String.Format(IFormatProvider, string, object[])
The text was updated successfully, but these errors were encountered: