GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix deadlock when piping to shell associated file extension by SeeminglyScience · Pull Request #19940 · PowerShell/PowerShell

Conversation

@SeeminglyScience

@SeeminglyScience Patrick Meinecke (SeeminglyScience) commented Jul 12, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Fixes #19876

PR Summary

Marking the process as initialized was done inside a try that can be part of normal operation if shell32!FindExecutableW finds a valid console subsystem executable. This change moves it after the try so that scenario will be caught as well.

Also reduced some allocations in IsExecutable. That change is separated into it's own commit for easier review, but I'm also fine with reverting that commit if desired.

I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.

PR Context

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
  • Breaking changes
    • None
    • OR
    • Experimental feature(s) needed
      • Experimental feature name(s):
  • User-facing changes
    • Not Applicable
    • OR
    • Documentation needed
      • Issue filed:
  • Testing - New and feature
    • N/A or can only be tested interactively
    • OR
    • Make sure you've added a new test if existing tests do not effectively test the code changed
  • Tooling
    • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
    • OR
    • I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
      • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
        (which runs in a different PS Host).
        • Issue filed:
      • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
        • Issue filed:
      • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
        • Issue filed:
      • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
        • Issue filed:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Patrick Meinecke (SeeminglyScience) added 2 commits July 11, 2023 16:29
@SeeminglyScience
Reduce some allocations in IsExecutable
4052701
@SeeminglyScience
Move initialized flag set until after creation
159b389
If the initial process creation failed and `shell32!FindExecutableW` was
used, the process initialization flag was getting skipped. Moving the
handle release until after the try/catch fixed a hang.
@pull-request-quantifier-deprecated pull-request-quantifier-deprecated Bot added the Extra Small label Jul 12, 2023
@SeeminglyScience
Patrick Meinecke (SeeminglyScience) requested a review from Dongbo Wang (daxian-dbw) July 12, 2023 03:52
@iSazonov

Ilya (iSazonov) commented Jul 12, 2023

Copy link
Copy Markdown
Collaborator

I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.

Search cmd /c assoc in our tests for examples.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iSazonov
Ilya (iSazonov) reviewed Jul 12, 2023
View reviewed changes
Comment thread src/System.Management.Automation/engine/NativeCommandProcessor.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/NativeCommandProcessor.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/NativeCommandProcessor.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Patrick Meinecke (SeeminglyScience) added 2 commits July 12, 2023 09:56
@SeeminglyScience
Revert "Reduce some allocations in IsExecutable"
395e076
This reverts commit 4052701.
@SeeminglyScience
Revert some unintended formatting changes
bdc8fc9
Must have some automatic formatting set that I'm not aware of. Didn't
mean for these changes to be added.
@SeeminglyScience

Patrick Meinecke (SeeminglyScience) commented Jul 12, 2023

Copy link
Copy Markdown
Contributor Author

I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.

Search cmd /c assoc in our tests for examples.

Hmm does that work outside of CI? I wasn't able to get assoc to do anything meaningful on Windows 11. Maybe that still works in nano/Windows Server.

Thanks I'll look into it

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SeeminglyScience
Move the flag set back into the lock
43febac
Also remove redundant lock
@pull-request-quantifier-deprecated

pull-request-quantifier-deprecated Bot commented Jul 12, 2023

Copy link
Copy Markdown

This PR has 9 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!


Quantification details

Label      : Extra Small
Size       : +3 -6
Percentile : 3.6%

Total files changed: 1

Change summary by file extension:
.cs : +3 -6

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

  • Fast and predictable releases to production:
    • Optimal size changes are more likely to be reviewed faster with fewer
      iterations.
    • Similarity in low PR complexity drives similar review times.
  • Review quality is likely higher as complexity is lower:
    • Bugs are more likely to be detected.
    • Code inconsistencies are more likely to be detected.
  • Knowledge sharing is improved within the participants:
    • Small portions can be assimilated better.
  • Better engineering practices are exercised:
    • Solving big problems by dividing them in well contained, smaller problems.
    • Exercising separation of concerns within the code changes.

What can I do to optimize my changes

  • Use the PullRequestQuantifier to quantify your PR accurately
    • Create a context profile for your repo using the context generator
    • Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
    • Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
    • Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
  • Change your engineering behaviors
    • For PRs that fall outside of the desired spectrum, review the details and check if:
      • Your PR could be split in smaller, self-contained PRs instead
      • Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

  • One line was added: +1 -0
  • One line was deleted: +0 -1
  • One line was modified: +1 -1 (git diff doesn't know about modified, it will
    interpret that line like one addition plus one deletion)
  • Change percentiles: Change characteristics (addition, deletion, modification)
    of this PR in relation to all other PRs within the repository.


Was this comment helpful? 👍  :ok_hand:  :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SeeminglyScience

Patrick Meinecke (SeeminglyScience) commented Jul 12, 2023

Copy link
Copy Markdown
Contributor Author

I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.

Search cmd /c assoc in our tests for examples.

Hmm does that work outside of CI? I wasn't able to get assoc to do anything meaningful on Windows 11. Maybe that still works in nano/Windows Server.

Thanks I'll look into it

So even if assoc works in CI, it doesn't actually test what I'm fixing here. The deadlock only occurred once kernel32!CreateProcess has already failed and we then call shell32!FindExecutable to get the shell association. But if assoc works, then kernel32!CreateProcess will work, and the deadlock wouldn't get hit anyway.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

SeeminglyScience
Patrick Meinecke (SeeminglyScience) commented Jul 12, 2023
View reviewed changes
Comment thread src/System.Management.Automation/engine/NativeCommandProcessor.cs
{
_processInitialized ??= new SemaphoreSlim(0, 1);
_processInitialized.Release();
}

@SeeminglyScience Patrick Meinecke (SeeminglyScience) Jul 12, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw I removed the lock because we're already in a lock for that object and I just didn't notice (line 596)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iSazonov

Ilya (iSazonov) commented Jul 13, 2023

Copy link
Copy Markdown
Collaborator

I don't think there's an non-interactive way to set a file extension these days so I'm not sure how this could be tested. If you know of a way (that doesn't use undocumented APIs) please comment.

Search cmd /c assoc in our tests for examples.

Hmm does that work outside of CI? I wasn't able to get assoc to do anything meaningful on Windows 11. Maybe that still works in nano/Windows Server.
Thanks I'll look into it

So even if assoc works in CI, it doesn't actually test what I'm fixing here. The deadlock only occurred once kernel32!CreateProcess has already failed and we then call shell32!FindExecutable to get the shell association. But if assoc works, then kernel32!CreateProcess will work, and the deadlock wouldn't get hit anyway.

Interesting.... if it is possible to reproduce the issue with Python can we emulate the same behavior with cmd.exe and .bat and use this in tests?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SeeminglyScience

Patrick Meinecke (SeeminglyScience) commented Jul 13, 2023

Copy link
Copy Markdown
Contributor Author

Interesting.... if it is possible to reproduce the issue with Python can we emulate the same behavior with cmd.exe and .bat and use this in tests?

.bat just straight up works as if it were an executable file, handling for that is hard coded. Even if it wasn't though, the shell (explorer) already has an association for .bat so in theory that would work anyway.

The main issue is that in recent history it's been made explicitly unsupported to set shell associations outside of the UI. I've heard of some ways, but it involves reverse engineered formats that change between Windows versions? Something like that, but either way not something we'd want to use in a test.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StevenBucher98 Steven Bucher (StevenBucher98) added the PowerShell-Docs not needed The PR was reviewed and doesn't appear to require a PowerShell Docs update label Jul 17, 2023
daxian-dbw
Dongbo Wang (daxian-dbw) approved these changes Jul 17, 2023
View reviewed changes

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@daxian-dbw
Dongbo Wang (daxian-dbw) merged commit 153feb2 into PowerShell:master Jul 17, 2023
@daxian-dbw Dongbo Wang (daxian-dbw) added the CL-Engine Indicates that a PR should be marked as an engine change in the Change Log label Jul 17, 2023
@SeeminglyScience
Patrick Meinecke (SeeminglyScience) deleted the fix-native-stdout-hang branch July 17, 2023 18:32
@rmccampbell

Ryan McCampbell (rmccampbell) commented Aug 12, 2023

Copy link
Copy Markdown

Hi, do you know if there is going to be a preview and/or release build with this fix any time soon?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
@SeeminglyScience
Fix deadlock when piping to shell associated file extension (PowerShe…
fa42109
…ll#19940)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iSazonov Ilya (iSazonov) iSazonov left review comments
@daxian-dbw Dongbo Wang (daxian-dbw) daxian-dbw approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

CL-Engine Indicates that a PR should be marked as an engine change in the Change Log Extra Small PowerShell-Docs not needed The PR was reviewed and doesn't appear to require a PowerShell Docs update

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

PSNativeCommandPreserveBytePipe hangs on piping to python scripts in Windows

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@SeeminglyScience @iSazonov @rmccampbell @daxian-dbw @StevenBucher98
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.