GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||
GeezerVison is showing github.com — Fix race condition to access powershell.config.json by iSazonov · Pull Request #8249 · PowerShell/PowerShell Conversation
|
|
Restarted builds because of "Get-PSGalleryApiAvailability : PowerShell Gallery is currently unavailable. Please try again later." |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
To prevent other processes from accessing, is it possible to take out an OS-level write lock on the json file? After searching for a bit on the web, it seems like new FileStream() with the right arguments already does this, no?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Rob Holt (@rjmholt) A problem I indicated is that we don't process an exception if file is locked for read. Scenario: run two pwsh-s, one lock the config file, second is trying to access the file and - what? Wait and stop whole process? Currently we'll crash. On a system where many PowerShell processes run, this can cause unpredictable (rarely) crashes. (At minimum, we could not put an exclusive write lock.) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
e2b1fbb to
a70f6b6
Compare
|
Hmmmm. Well given that the config file is changed at runtime by Otherwise we are forced to use some OS-level lock to get an atomic interleaving of PowerShell config writes? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I added a simple file unlock wait. I think it will resolve the inter process race condition in most of cases. Of cause an admin user can still exclusively open the file with read lock and crash pwsh processes. Seems it is an edge case. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
MSFT team please review new commit. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Andrew (@anmenaga) Travis Plunk (@TravisEz13) Steve Lee (@SteveL-MSFT) Aditya Patwardhan (@adityapatwardhan) Dongbo Wang (@daxian-dbw) Rob Holt (@rjmholt) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I'm not a fan of this |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Yes, sorry, only meant to discuss rather than request changes (couldn't find a good issue to discuss in) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Original problem is a bug that is in-process race condition. It is hosting scenarios. Fixed by first commit. I think both commits is important because pwsh crashed.
The solution looks good for me because it is very rare race condition and we will never block a process for long especially since we cache |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
The looped wait timer approach I think looks good in comparison to the alternative: a corrupt powershell.config.json which ideally leads to a crash, but otherwise leads to one of the PowerShell processes incorrectly picking up a new configuration. I get the impression though that perhaps a shared mutable configuration between processes is something we should reconsider as an idea? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
This could be addressed by PowerShell/PowerShell-RFC#111 too. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Dongbo Wang (@daxian-dbw) Can you have a look at this? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Aditya Patwardhan (@adityapatwardhan) Dongbo Wang (@daxian-dbw) The PR block sequence of other PRs. Could you please continue? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Look mostly good to me. Please address the 3 comments.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| using (var streamReader = new StreamReader(fileName)) | ||
| // The config file can be locked by another process | ||
| // so we wait some milliseconds in 'WaitForFile()' for recovery before stop current process. | ||
| using (var readerStream = WaitForFile(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
By having FileShare.ReadWrite, we allow the file to be opened for writing by another process. FileShare.ReadWrite could only possibly work in a logging scenario, where new content is always appended to the end of a file and you read the file through seek. I think it should be changed to FileShare.Read.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
With FileShare.Read the PowerShell Core process will terminate if anybody open the file for writing. We aim to avoid this and open the file with minimal lock level. Also we don't keep the file opened: every time we create new StreamReader on very short time. There still can be a race condition if we read partially changed file but it is another issue - what PowerShell Core should do if the config file is broken.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
IMO, this permission raises more confusion than the potential race condition it might mitigate. I think the right approach would be to return a default setting when the file cannot be opened after the retry, or when the file is corrupted (schema validation would be good but that would impact the startup time).
The current design is to only read the particular setting that is requested, which will result in opening the files multiple times but actually reading the whole file every time. I don't think this is the right thing to do. We probably should be very explicit that the setting file is read only one time at startup, and we process all settings that is present in the file. If the file cannot be opened after retry or the content is corrupted, we use the default settings and somehow convey the error to user (yes, we need to define the default settings first). In this way, the file is read only once, which means less race conditions, and less file access cost. Anyways, this should be a separate PR, not a change request for this PR. Here we only focus on the share permission.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Should I change anything in the PR?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
IMO, this permission raises more confusion than the potential race condition it might mitigate. I think the right approach would be to return a default setting when the file cannot be opened after the retry, or when the file is corrupted (schema validation would be good but that would impact the startup time).
Yes, I think changing FileShare.ReadWrite to FileShare.Read :)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I am not sure that this resolve the race condition in our xUnit tests - I started the PR to get this.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Your xUnit tests actively modify the config file (write lock?) and this lead to crash other tests .
I agree that here we can only temporary resolve the problem and we should work on better solution. (Address in my RFC or make PR?)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Dongbo Wang (@daxian-dbw) returning a default settings file could result in security settings in the config file being bypassed.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Your xUnit tests actively modify the config file (write lock?) and this lead to crash other tests .
Ilya (@iSazonov) that's unfortunate to hear :( Then let's go with FileShare.ReadWrite for now.
Travis Plunk (@TravisEz13) I agree. It sounds to me crashing pwsh when config file parsing fails is more securer :)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Dongbo Wang (@daxian-dbw) Travis Plunk (@TravisEz13) Could you please add more feedback about "security settings" to my RFC? We could discuss this there. I don't like the "crashing pwsh" because it could be a DoS attack even by admin user.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Aditya Patwardhan (@adityapatwardhan) Please merge the PR. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Ilya (@iSazonov) Can you fix the two CodeFactor issues? Then the PR can be merged. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Aditya Patwardhan (@adityapatwardhan) CodeFactor issues was fixed. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Remove unneeded dotnet restore Add --no-restore Add extraParams for MacOS Fix TestRunspaceWithPowerShellAndInitialSessionState Fix race condition to access powershell.config.json Reduce an impact of the inter process race condition Fix race condition to access powershell.config.json (PowerShell#8249)
Remove unneeded dotnet restore Add --no-restore Add extraParams for MacOS Fix TestRunspaceWithPowerShellAndInitialSessionState Fix race condition to access powershell.config.json Reduce an impact of the inter process race condition Fix race condition to access powershell.config.json (PowerShell#8249)
Remove unneeded dotnet restore Add --no-restore Add extraParams for MacOS Fix TestRunspaceWithPowerShellAndInitialSessionState Fix race condition to access powershell.config.json Reduce an impact of the inter process race condition Fix race condition to access powershell.config.json (PowerShell#8249)
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.