Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance: Eliminate Regex overhead in AvoidTrailingWhitespace -> Speedup of 5% (PowerShell 5.1) or 2.5 % (PowerShell 7.1-preview.2) #1465

Merged
merged 7 commits into from
Apr 28, 2020
Merged

Performance: Eliminate Regex overhead in AvoidTrailingWhitespace -> Speedup of 5% (PowerShell 5.1) or 2.5 % (PowerShell 7.1-preview.2) #1465

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9a6db4c
Improve performance by not using regex (2% improvement)
Apr 22, 2020
095c2f1
Merge branch 'master' of https://github.com/bergmeister/psscriptanaly…
Apr 27, 2020
eab6312
replace regex
Apr 27, 2020
172ebdd
fix index and simplify
Apr 27, 2020
9456321
tidy
Apr 27, 2020
c35ec92
Apply suggestions from code review
bergmeister Apr 27, 2020
aedbc13
Use IsWhiteSpace
Apr 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 52 additions & 37 deletions Rules/AvoidTrailingWhitespace.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,52 +36,67 @@ public IEnumerable<DiagnosticRecord> AnalyzeScript(Ast ast, string fileName)

var diagnosticRecords = new List<DiagnosticRecord>();

string[] lines = Regex.Split(ast.Extent.Text, @"\r?\n");
string[] lines = ast.Extent.Text.Split(new[] { "\r\n", "\r", "\n" }, StringSplitOptions.None);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me wonder: if we're just trying to find the extents of trailing whitespace, there's no need to split the string at all; we should just read through ourselves without allocating all these strings... But too much burden for this PR!

Copy link
Collaborator Author

@bergmeister bergmeister Apr 27, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yh, I hear what you say, I guess for perf what counts is the 80-20 rule :-) Technically speaking string.IndexOf would probably the fastest way of finding the indices where \s\r or \s\n occurs....
I'm aware of lot's of other small micro optimisations that one can make and even tried some but they didn't have a measurable outcome. Therefore I am focussed on just fixing what gives at least a measurable return.


for (int lineNumber = 0; lineNumber < lines.Length; lineNumber++)
{
var line = lines[lineNumber];

var match = Regex.Match(line, @"\s+$");
if (match.Success)
if (line.Length == 0)
{
var startLine = lineNumber + 1;
var endLine = startLine;
var startColumn = match.Index + 1;
var endColumn = startColumn + match.Length;

var violationExtent = new ScriptExtent(
new ScriptPosition(
ast.Extent.File,
startLine,
startColumn,
line
),
new ScriptPosition(
ast.Extent.File,
endLine,
endColumn,
line
));

var suggestedCorrections = new List<CorrectionExtent>();
suggestedCorrections.Add(new CorrectionExtent(
violationExtent,
string.Empty,
ast.Extent.File
));
continue;
}
bergmeister marked this conversation as resolved.
Show resolved Hide resolved
if (line[line.Length - 1] != ' ' &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this be better as char.IsWhiteSpace(line[line.Length - 1])?

Copy link
Collaborator Author

@bergmeister bergmeister Apr 27, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rjmholt because of

  • readablity
  • perfomance
  • covering the variety of unicode chars? from the docs here, it would probably be good but what about the UnicodeCategory.LineSeparator char? I don't have much Unicode experience to make a judgement call here tbh if this list includes too much or not
    image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking here is actually just that PowerShell uses that API to see whitespace.

Given how we split the string already, it's possibly dangerous to go by unicode whitespace, but possibly not...

I suspect that really this won't make much difference; leaving non-ASCII whitespace at the ends of lines isn't something I can imagine being an issue for anyone really.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so that sounds more like a tendency to use IsWhiteSpace? I'd be OK with that, you are right that the impact is probably quite low, especially since this rules is not enabled by default for vs-code users.

line[line.Length - 1] != '\t')
{
continue;
}
bergmeister marked this conversation as resolved.
Show resolved Hide resolved
int startColumnOfTrailingWhitespace = 1;
for (int i = line.Length - 2; i > 0; i--)
{
if (line[i] != ' ' && line[i] != '\t')
{
startColumnOfTrailingWhitespace = i + 2;
break;
}
}

diagnosticRecords.Add(
new DiagnosticRecord(
String.Format(CultureInfo.CurrentCulture, Strings.AvoidTrailingWhitespaceError),
var startLine = lineNumber + 1;
bergmeister marked this conversation as resolved.
Show resolved Hide resolved
var endLine = startLine;
bergmeister marked this conversation as resolved.
Show resolved Hide resolved
var startColumn = startColumnOfTrailingWhitespace;
bergmeister marked this conversation as resolved.
Show resolved Hide resolved
var endColumn = line.Length + 1;
bergmeister marked this conversation as resolved.
Show resolved Hide resolved

var violationExtent = new ScriptExtent(
new ScriptPosition(
ast.Extent.File,
startLine,
startColumn,
line
),
new ScriptPosition(
ast.Extent.File,
endLine,
endColumn,
line
));

var suggestedCorrections = new List<CorrectionExtent>();
suggestedCorrections.Add(new CorrectionExtent(
violationExtent,
GetName(),
GetDiagnosticSeverity(),
ast.Extent.File,
null,
suggestedCorrections
string.Empty,
ast.Extent.File
));
}

diagnosticRecords.Add(
new DiagnosticRecord(
String.Format(CultureInfo.CurrentCulture, Strings.AvoidTrailingWhitespaceError),
violationExtent,
GetName(),
GetDiagnosticSeverity(),
ast.Extent.File,
null,
suggestedCorrections
));
}

return diagnosticRecords;
Expand Down