GeezerVison is showing github.com — Fix deadlock when piping to shell associated file extension by SeeminglyScience · Pull Request #19940 · PowerShell/PowerShell
Conversation
Fixes #19876
PR Summary
Marking the process as initialized was done inside a try that can be part of normal operation if shell32!FindExecutableW finds a valid console subsystem executable. This change moves it after the try so that scenario will be caught as well.
Also reduced some allocations in IsExecutable. That change is separated into it's own commit for easier review, but I'm also fine with reverting that commit if desired.
I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.
PR Context
PR Checklist
- PR has a meaningful title
- Use the present tense and imperative mood when describing your changes
- Summarized changes
- Make sure all
.h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
- This PR is ready to merge and is not Work in Progress.
- If the PR is work in progress, please add the prefix
WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
- Breaking changes
- User-facing changes
- Testing - New and feature
- Tooling
- I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
- OR
- I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
- Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
(which runs in a different PS Host).
- Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
- Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
- Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
If the initial process creation failed and `shell32!FindExecutableW` was
used, the process initialization flag was getting skipped. Moving the
handle release until after the try/catch fixed a hang.
I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.
Search cmd /c assoc in our tests for examples.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Must have some automatic formatting set that I'm not aware of. Didn't
mean for these changes to be added.
I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.
Search cmd /c assoc in our tests for examples.
Hmm does that work outside of CI? I wasn't able to get assoc to do anything meaningful on Windows 11. Maybe that still works in nano/Windows Server.
Thanks I'll look into it
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Also remove redundant lock

This PR has 9 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!
Quantification details
Label : Extra Small
Size : +3 -6
Percentile : 3.6%
Total files changed: 1
Change summary by file extension:
.cs : +3 -6
Change counts above are quantified counts, based on the PullRequestQuantifier customizations.
Why proper sizing of changes matters
Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:
- Fast and predictable releases to production:
- Optimal size changes are more likely to be reviewed faster with fewer
iterations.
- Similarity in low PR complexity drives similar review times.
- Review quality is likely higher as complexity is lower:
- Bugs are more likely to be detected.
- Code inconsistencies are more likely to be detected.
- Knowledge sharing is improved within the participants:
- Small portions can be assimilated better.
- Better engineering practices are exercised:
- Solving big problems by dividing them in well contained, smaller problems.
- Exercising separation of concerns within the code changes.
What can I do to optimize my changes
- Use the PullRequestQuantifier to quantify your PR accurately
- Create a context profile for your repo using the context generator
- Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the
Excluded section from your prquantifier.yaml context profile.
- Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your
prquantifier.yaml context profile.
- Only use the labels that matter to you, see context specification to customize your
prquantifier.yaml context profile.
- Change your engineering behaviors
- For PRs that fall outside of the desired spectrum, review the details and check if:
- Your PR could be split in smaller, self-contained PRs instead
- Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).
How to interpret the change counts in git diff output
- One line was added:
+1 -0
- One line was deleted:
+0 -1
- One line was modified:
+1 -1 (git diff doesn't know about modified, it will
interpret that line like one addition plus one deletion)
- Change percentiles: Change characteristics (addition, deletion, modification)
of this PR in relation to all other PRs within the repository.
Was this comment helpful? 👍 :ok_hand: :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.
Search cmd /c assoc in our tests for examples.
Hmm does that work outside of CI? I wasn't able to get assoc to do anything meaningful on Windows 11. Maybe that still works in nano/Windows Server.
Thanks I'll look into it
So even if assoc works in CI, it doesn't actually test what I'm fixing here. The deadlock only occurred once kernel32!CreateProcess has already failed and we then call shell32!FindExecutable to get the shell association. But if assoc works, then kernel32!CreateProcess will work, and the deadlock wouldn't get hit anyway.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
|
{
|
|
|
_processInitialized ??= new SemaphoreSlim(0, 1);
|
|
|
_processInitialized.Release();
|
|
|
}
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw I removed the lock because we're already in a lock for that object and I just didn't notice (line 596)
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.
Search cmd /c assoc in our tests for examples.
Hmm does that work outside of CI? I wasn't able to get assoc to do anything meaningful on Windows 11. Maybe that still works in nano/Windows Server.
Thanks I'll look into it
So even if assoc works in CI, it doesn't actually test what I'm fixing here. The deadlock only occurred once kernel32!CreateProcess has already failed and we then call shell32!FindExecutable to get the shell association. But if assoc works, then kernel32!CreateProcess will work, and the deadlock wouldn't get hit anyway.
Interesting.... if it is possible to reproduce the issue with Python can we emulate the same behavior with cmd.exe and .bat and use this in tests?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Interesting.... if it is possible to reproduce the issue with Python can we emulate the same behavior with cmd.exe and .bat and use this in tests?
.bat just straight up works as if it were an executable file, handling for that is hard coded. Even if it wasn't though, the shell (explorer) already has an association for .bat so in theory that would work anyway.
The main issue is that in recent history it's been made explicitly unsupported to set shell associations outside of the UI. I've heard of some ways, but it involves reverse engineered formats that change between Windows versions? Something like that, but either way not something we'd want to use in a test.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Hi, do you know if there is going to be a preview and/or release build with this fix any time soon?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|