GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Performance: Eliminate Regex overhead in AvoidTrailingWhitespace -> Speedup of 5% (PowerShell 5.1) or 2.5 % (PowerShell 7.1-preview.2) by bergmeister · Pull Request #1465 · PowerShell/PSScriptAnalyzer

Conversation

@bergmeister

@bergmeister Christoph Bergmeister (bergmeister) commented Apr 27, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

PR Summary

Whitespace ignoring diff makes it clearer. This was the most expensive script analysis rule when being run in warm mode and also easy to fix :-)
It also shows the performance improvements in .Net Core 5

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Change is not breaking
  • Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
  • Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Christoph Bergmeister added 4 commits April 22, 2020 20:33
Christoph Bergmeister
Improve performance by not using regex (2% improvement)
9a6db4c
Christoph Bergmeister
Merge branch 'master' of https://github.com/bergmeister/psscriptanalyzer
095c2f1
 into perf/regex
Christoph Bergmeister
replace regex
eab6312
Christoph Bergmeister
fix index and simplify
172ebdd
@bergmeister Christoph Bergmeister (bergmeister) changed the title Performance: Eliminate Regex overhead in AvoidTrailingWhitespace -> Speedup of 5% (PowerShell 5.1) or 2.5 % (PowerShell 7) Performance: Eliminate Regex overhead in AvoidTrailingWhitespace -> Speedup of 5% (PowerShell 5.1) or 2.5 % (PowerShell 7.1-preview.2) Apr 27, 2020
@bergmeister Christoph Bergmeister (bergmeister) added Area - Performance Area - Rules labels Apr 27, 2020
@bergmeister
Christoph Bergmeister (bergmeister) requested review from James Truher (JamesWTruher) and Rob Holt (rjmholt) April 27, 2020 18:57
Christoph Bergmeister
tidy
9456321
rjmholt
Rob Holt (rjmholt) approved these changes Apr 27, 2020
View reviewed changes

@rjmholt Rob Holt (rjmholt) left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use regexes anywhere else in the codebase, we could probably save some performance by just making the regex static and constructing it with RegexOptions.Compile

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Rules/AvoidTrailingWhitespace.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Rules/AvoidTrailingWhitespace.cs Outdated
));
continue;
}
if (line[line.Length - 1] != ' ' &&

@rjmholt Rob Holt (rjmholt) Apr 27, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this be better as char.IsWhiteSpace(line[line.Length - 1])?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@bergmeister Christoph Bergmeister (bergmeister) Apr 27, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rob Holt (@rjmholt) because of

  • readablity
  • perfomance
  • covering the variety of unicode chars? from the docs here, it would probably be good but what about the UnicodeCategory.LineSeparator char? I don't have much Unicode experience to make a judgement call here tbh if this list includes too much or not
    image

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@rjmholt Rob Holt (rjmholt) Apr 28, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking here is actually just that PowerShell uses that API to see whitespace.

Given how we split the string already, it's possibly dangerous to go by unicode whitespace, but possibly not...

I suspect that really this won't make much difference; leaving non-ASCII whitespace at the ends of lines isn't something I can imagine being an issue for anyone really.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@bergmeister Christoph Bergmeister (bergmeister) Apr 28, 2020

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so that sounds more like a tendency to use IsWhiteSpace? I'd be OK with that, you are right that the impact is probably quite low, especially since this rules is not enabled by default for vs-code users.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Rules/AvoidTrailingWhitespace.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Rules/AvoidTrailingWhitespace.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Rules/AvoidTrailingWhitespace.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Rules/AvoidTrailingWhitespace.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Rules/AvoidTrailingWhitespace.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Rules/AvoidTrailingWhitespace.cs
var diagnosticRecords = new List<DiagnosticRecord>();

string[] lines = Regex.Split(ast.Extent.Text, @"\r?\n");
string[] lines = ast.Extent.Text.Split(new[] { "\r\n", "\r", "\n" }, StringSplitOptions.None);

@rjmholt Rob Holt (rjmholt) Apr 27, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me wonder: if we're just trying to find the extents of trailing whitespace, there's no need to split the string at all; we should just read through ourselves without allocating all these strings... But too much burden for this PR!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@bergmeister Christoph Bergmeister (bergmeister) Apr 27, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yh, I hear what you say, I guess for perf what counts is the 80-20 rule :-) Technically speaking string.IndexOf would probably the fastest way of finding the indices where \s\r or \s\n occurs....
I'm aware of lot's of other small micro optimisations that one can make and even tried some but they didn't have a measurable outcome. Therefore I am focussed on just fixing what gives at least a measurable return.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

rjmholt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Christoph Bergmeister (bergmeister) and others added 2 commits April 27, 2020 22:05
@bergmeister @rjmholt
Apply suggestions from code review
c35ec92
Co-Authored-By: Robert Holt <[email protected]>
Christoph Bergmeister
Use IsWhiteSpace
aedbc13
@bergmeister
Christoph Bergmeister (bergmeister) merged commit 6fa29cb into PowerShell:master Apr 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JamesWTruher James Truher (JamesWTruher) Awaiting requested review from JamesWTruher
1 more reviewer
@rjmholt Rob Holt (rjmholt) rjmholt approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

Area - Performance Area - Rules

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@bergmeister @rjmholt
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.