GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix race condition to access powershell.config.json by iSazonov · Pull Request #8249 · PowerShell/PowerShell

Conversation

@iSazonov

@iSazonov Ilya (iSazonov) commented Nov 13, 2018 •
edited by TravisEz13
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

PR Summary

See #8237 (comment)

(Also there can be a race condition if some processes access the config file. I don't now how to fix.)

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Change is not breaking
  • Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.
  • User-facing changes
    • Not Applicable
    • OR
    • User-facing Documentation needed
      • Issue filed - Issue link:
  • Testing - New and feature
    • Not Applicable or can only be tested interactively
    • OR
    • Make sure you've added a new test if existing tests do not effectively test the code changed
      • Add [feature] if the change is significant or affects feature tests

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iSazonov
Ilya (iSazonov) requested review from Steve Lee (SteveL-MSFT), Travis Plunk (TravisEz13), Aditya Patwardhan (adityapatwardhan) and Dongbo Wang (daxian-dbw) November 13, 2018 15:32
@iSazonov
Ilya (iSazonov) requested a review from Bruce Payette (BrucePay) as a code owner November 13, 2018 15:32
@anmenaga

Andrew (anmenaga) commented Nov 13, 2018

Copy link
Copy Markdown

Restarted builds because of "Get-PSGalleryApiAvailability : PowerShell Gallery is currently unavailable. Please try again later."

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

anmenaga
Andrew (anmenaga) approved these changes Nov 13, 2018
View reviewed changes
SteveL-MSFT
Steve Lee (SteveL-MSFT) approved these changes Nov 13, 2018
View reviewed changes
TravisEz13
Travis Plunk (TravisEz13) approved these changes Nov 14, 2018
View reviewed changes
@TravisEz13 Travis Plunk (TravisEz13) added WG-Engine core PowerShell engine, interpreter, and runtime CL-Engine Indicates that a PR should be marked as an engine change in the Change Log labels Nov 14, 2018
@TravisEz13 Travis Plunk (TravisEz13) assigned Aditya Patwardhan (adityapatwardhan) Nov 14, 2018
rjmholt
Rob Holt (rjmholt) approved these changes Nov 14, 2018
View reviewed changes

@rjmholt Rob Holt (rjmholt) left a comment

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

To prevent other processes from accessing, is it possible to take out an OS-level write lock on the json file? After searching for a bit on the web, it seems like new FileStream() with the right arguments already does this, no?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/PSConfiguration.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Nov 14, 2018

Copy link
Copy Markdown
Collaborator Author

Rob Holt (@rjmholt) A problem I indicated is that we don't process an exception if file is locked for read. Scenario: run two pwsh-s, one lock the config file, second is trying to access the file and - what? Wait and stop whole process? Currently we'll crash. On a system where many PowerShell processes run, this can cause unpredictable (rarely) crashes. (At minimum, we could not put an exclusive write lock.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov
Fix race condition to access powershell.config.json
a70f6b6
@iSazonov
Ilya (iSazonov) force-pushed the fix-race-cond branch from e2b1fbb to a70f6b6 Compare November 14, 2018 05:24
@iSazonov
Use var
f79a4f5
@rjmholt

Rob Holt (rjmholt) commented Nov 14, 2018

Copy link
Copy Markdown
Collaborator

Hmmmm. Well given that the config file is changed at runtime by Set-ExecutionPolicy (and possibly others), maybe it should crash (more gracefully) -- given that config files may be changed by the PowerShell session, maybe they are session specific?

Otherwise we are forced to use some OS-level lock to get an atomic interleaving of PowerShell config writes?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov
Reduce an impact of the inter process race condition
3f4e604
@iSazonov

Ilya (iSazonov) commented Nov 14, 2018

Copy link
Copy Markdown
Collaborator Author

I added a simple file unlock wait. I think it will resolve the inter process race condition in most of cases. Of cause an admin user can still exclusively open the file with read lock and crash pwsh processes. Seems it is an edge case.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Nov 14, 2018

Copy link
Copy Markdown
Collaborator Author

MSFT team please review new commit.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov Ilya (iSazonov) mentioned this pull request Nov 14, 2018
Move xUnit tests to new folder #8237
Closed
5 of 11 tasks
@iSazonov

Ilya (iSazonov) commented Nov 15, 2018

Copy link
Copy Markdown
Collaborator Author

Andrew (@anmenaga) Travis Plunk (@TravisEz13) Steve Lee (@SteveL-MSFT) Aditya Patwardhan (@adityapatwardhan) Dongbo Wang (@daxian-dbw) Rob Holt (@rjmholt)
Please review last commit. The PR block #8237 and my follow work.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@anmenaga

Andrew (anmenaga) commented Nov 15, 2018

Copy link
Copy Markdown

I'm not a fan of this WaitForFile addition (and looped wait timers in general unless they are absolutely necessary).
Original problem was intermittently sharing read access to the file between PS processes. FileShare.ReadWrite is good enough for this. I would remove WaitForFile addition (especially considering that it only partially addresses the issue that it's trying to address).

TravisEz13 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rjmholt

Rob Holt (rjmholt) commented Nov 15, 2018

Copy link
Copy Markdown
Collaborator

Yes, sorry, only meant to discuss rather than request changes (couldn't find a good issue to discuss in)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Nov 16, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator Author

Original problem was intermittently sharing read access to the file between PS processes.

Original problem is a bug that is in-process race condition. It is hosting scenarios. Fixed by first commit.
Second commit is for inter-process race condition. It is scenarios with many powershell processes (like background scheduled tasks). Fixed by second commit.

I think both commits is important because pwsh crashed.

looped wait timers in general unless they are absolutely necessary

The solution looks good for me because it is very rare race condition and we will never block a process for long especially since we cache powershell.config.json.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rjmholt

Rob Holt (rjmholt) commented Nov 16, 2018

Copy link
Copy Markdown
Collaborator

The looped wait timer approach I think looks good in comparison to the alternative: a corrupt powershell.config.json which ideally leads to a crash, but otherwise leads to one of the PowerShell processes incorrectly picking up a new configuration.

I get the impression though that perhaps a shared mutable configuration between processes is something we should reconsider as an idea?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Nov 16, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator Author

I get the impression though that perhaps a shared mutable configuration between processes is something we should reconsider as an idea?

This could be addressed by PowerShell/PowerShell-RFC#111 too.

rjmholt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TravisEz13
Travis Plunk (TravisEz13) approved these changes Nov 16, 2018
View reviewed changes
@adityapatwardhan

Aditya Patwardhan (adityapatwardhan) commented Nov 21, 2018

Copy link
Copy Markdown
Member

Dongbo Wang (@daxian-dbw) Can you have a look at this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Nov 22, 2018

Copy link
Copy Markdown
Collaborator Author

Aditya Patwardhan (@adityapatwardhan) Dongbo Wang (@daxian-dbw) The PR block sequence of other PRs. Could you please continue?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

daxian-dbw
Dongbo Wang (daxian-dbw) reviewed Nov 23, 2018
View reviewed changes

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look mostly good to me. Please address the 3 comments.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/PSConfiguration.cs
using (var streamReader = new StreamReader(fileName))
// The config file can be locked by another process
// so we wait some milliseconds in 'WaitForFile()' for recovery before stop current process.
using (var readerStream = WaitForFile(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 23, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By having FileShare.ReadWrite, we allow the file to be opened for writing by another process. FileShare.ReadWrite could only possibly work in a logging scenario, where new content is always appended to the end of a file and you read the file through seek. I think it should be changed to FileShare.Read.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@iSazonov Ilya (iSazonov) Nov 23, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With FileShare.Read the PowerShell Core process will terminate if anybody open the file for writing. We aim to avoid this and open the file with minimal lock level. Also we don't keep the file opened: every time we create new StreamReader on very short time. There still can be a race condition if we read partially changed file but it is another issue - what PowerShell Core should do if the config file is broken.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 26, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, this permission raises more confusion than the potential race condition it might mitigate. I think the right approach would be to return a default setting when the file cannot be opened after the retry, or when the file is corrupted (schema validation would be good but that would impact the startup time).

The current design is to only read the particular setting that is requested, which will result in opening the files multiple times but actually reading the whole file every time. I don't think this is the right thing to do. We probably should be very explicit that the setting file is read only one time at startup, and we process all settings that is present in the file. If the file cannot be opened after retry or the content is corrupted, we use the default settings and somehow convey the error to user (yes, we need to define the default settings first). In this way, the file is read only once, which means less race conditions, and less file access cost. Anyways, this should be a separate PR, not a change request for this PR. Here we only focus on the share permission.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@iSazonov Ilya (iSazonov) Nov 26, 2018

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I change anything in the PR?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 26, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, this permission raises more confusion than the potential race condition it might mitigate. I think the right approach would be to return a default setting when the file cannot be opened after the retry, or when the file is corrupted (schema validation would be good but that would impact the startup time).

Yes, I think changing FileShare.ReadWrite to FileShare.Read :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@iSazonov Ilya (iSazonov) Nov 26, 2018

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure that this resolve the race condition in our xUnit tests - I started the PR to get this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@iSazonov Ilya (iSazonov) Nov 26, 2018

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your xUnit tests actively modify the config file (write lock?) and this lead to crash other tests .
I agree that here we can only temporary resolve the problem and we should work on better solution. (Address in my RFC or make PR?)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@TravisEz13 Travis Plunk (TravisEz13) Nov 26, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dongbo Wang (@daxian-dbw) returning a default settings file could result in security settings in the config file being bypassed.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 26, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your xUnit tests actively modify the config file (write lock?) and this lead to crash other tests .

Ilya (@iSazonov) that's unfortunate to hear :( Then let's go with FileShare.ReadWrite for now.

Travis Plunk (@TravisEz13) I agree. It sounds to me crashing pwsh when config file parsing fails is more securer :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

TravisEz13 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@iSazonov Ilya (iSazonov) Nov 27, 2018

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dongbo Wang (@daxian-dbw) Travis Plunk (@TravisEz13) Could you please add more feedback about "security settings" to my RFC? We could discuss this there. I don't like the "crashing pwsh" because it could be a DoS attack even by admin user.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/PSConfiguration.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/PSConfiguration.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov
Add private modifier
e5462e7
daxian-dbw
Dongbo Wang (daxian-dbw) approved these changes Nov 26, 2018
View reviewed changes
@iSazonov

Ilya (iSazonov) commented Nov 27, 2018

Copy link
Copy Markdown
Collaborator Author

Aditya Patwardhan (@adityapatwardhan) Please merge the PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@adityapatwardhan

Aditya Patwardhan (adityapatwardhan) commented Nov 27, 2018

Copy link
Copy Markdown
Member

Ilya (@iSazonov) Can you fix the two CodeFactor issues? Then the PR can be merged.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov
Fix CodeFactor issues
3556792
@iSazonov

Ilya (iSazonov) commented Nov 28, 2018

Copy link
Copy Markdown
Collaborator Author

Aditya Patwardhan (@adityapatwardhan) CodeFactor issues was fixed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@adityapatwardhan
Aditya Patwardhan (adityapatwardhan) merged commit 9381d03 into PowerShell:master Nov 28, 2018
@iSazonov
Ilya (iSazonov) deleted the fix-race-cond branch November 29, 2018 03:06
Ilya (iSazonov) added a commit to iSazonov/PowerShell that referenced this pull request Nov 29, 2018
@iSazonov
Move xUnit tests in new folder
ef95b24
Remove unneeded dotnet restore

Add  --no-restore

Add extraParams for MacOS

Fix TestRunspaceWithPowerShellAndInitialSessionState

Fix race condition to access powershell.config.json

Reduce an impact of the inter process race condition

Fix race condition to access powershell.config.json (PowerShell#8249)
Ilya (iSazonov) added a commit to iSazonov/PowerShell that referenced this pull request Nov 29, 2018
@iSazonov
Move xUnit tests in new folder
6fe441b
Remove unneeded dotnet restore

Add  --no-restore

Add extraParams for MacOS

Fix TestRunspaceWithPowerShellAndInitialSessionState

Fix race condition to access powershell.config.json

Reduce an impact of the inter process race condition

Fix race condition to access powershell.config.json (PowerShell#8249)
Travis Plunk (TravisEz13) pushed a commit to iSazonov/PowerShell that referenced this pull request Dec 12, 2018
@iSazonov @TravisEz13
Move xUnit tests in new folder
c4a86e5
Remove unneeded dotnet restore

Add  --no-restore

Add extraParams for MacOS

Fix TestRunspaceWithPowerShellAndInitialSessionState

Fix race condition to access powershell.config.json

Reduce an impact of the inter process race condition

Fix race condition to access powershell.config.json (PowerShell#8249)
@daxian-dbw Dongbo Wang (daxian-dbw) mentioned this pull request Feb 20, 2019
Reduce race condition probability for config file access #8779
Merged
7 of 11 tasks
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
@iSazonov @adityapatwardhan
Fix race condition to access powershell.config.json (PowerShell#8249)
19e6fc9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TravisEz13 Travis Plunk (TravisEz13) TravisEz13 approved these changes
@daxian-dbw Dongbo Wang (daxian-dbw) daxian-dbw approved these changes
@SteveL-MSFT Steve Lee (SteveL-MSFT) SteveL-MSFT approved these changes
@adityapatwardhan Aditya Patwardhan (adityapatwardhan) Awaiting requested review from adityapatwardhan
@BrucePay Bruce Payette (BrucePay) Awaiting requested review from BrucePay
+2 more reviewers
@rjmholt Rob Holt (rjmholt) rjmholt approved these changes
@anmenaga Andrew (anmenaga) anmenaga approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@adityapatwardhan Aditya Patwardhan (adityapatwardhan)

Labels

CL-Engine Indicates that a PR should be marked as an engine change in the Change Log WG-Engine core PowerShell engine, interpreter, and runtime

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@iSazonov @anmenaga @rjmholt @adityapatwardhan @daxian-dbw @TravisEz13 @SteveL-MSFT
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.