GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix incorrect name check when autoloading required modules by rjmholt · Pull Request #8218 · PowerShell/PowerShell

Conversation

@rjmholt

@rjmholt Rob Holt (rjmholt) commented Nov 9, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

PR Summary

Fixes #8204.

The change I made in #7125 made the check too strict on the module we try to autoload it from requires and rejected it based on name.

This PR removes that name check but also sets up a fix for the more general problem that a ModuleSpecification can always specify a path rather than a name. I added some reusable logic to make this easier to service.

The main contribution is regression testing for RequiredModules, #requires -Modules and other cmdlets that can use ModuleSpecifications.

However, the general pervasive problem is that ModuleSpecifications contain raw user input and we don't do very much about handling paths that might be given as module names, namely:

  • Normalising directory separators
  • Resolving relative paths (which should be resolved differently depending on context)
  • Dealing with case-sensitivity
  • Handling trailing slashes
  • Matching on module directories accounting for the versioned directory structure
  • Working with special directory names like . and ..

The methods I've added deal with these and I've applied them in one place to help correct the immediate problem and make it more efficient.

But the fact remains that we have no way in the type system to identify when this resolution has already occurred; there are over 120 uses of the ModuleSpecification type in the code base, and we don't have a way to differentiate when they contain raw user input and when they can be trusted to have a canonical path. The way the module cmdlets are written currently, we redo a fair amount of this validation and normalisation logic, and it's probably somewhere we could find serious perf improvements.

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Change is not breaking
  • Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.
  • User-facing changes
    • Not Applicable
    • OR
    • User-facing Documentation needed
      • Issue filed - Issue link:
  • Testing - New and feature
    • Not Applicable or can only be tested interactively
    • OR
    • Make sure you've added a new test if existing tests do not effectively test the code changed
      • Add [feature] if the change is significant or affects feature tests

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@rjmholt
Skip name check when loading required modules
c6dcb43
@rjmholt
Rob Holt (rjmholt) requested review from Bruce Payette (BrucePay) and Dongbo Wang (daxian-dbw) as code owners November 9, 2018 06:39
@iSazonov
Ilya (iSazonov) requested review from Travis Plunk (TravisEz13) and Aditya Patwardhan (adityapatwardhan) November 9, 2018 06:43
@iSazonov Ilya (iSazonov) assigned Aditya Patwardhan (adityapatwardhan) Nov 9, 2018
@bergmeister

Christoph Bergmeister (bergmeister) commented Nov 9, 2018

Copy link
Copy Markdown
Contributor

Thanks for handling the issue. I'd consider a regression test for this important feature to be essential.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rjmholt

Rob Holt (rjmholt) commented Nov 9, 2018

Copy link
Copy Markdown
Collaborator Author

Agreed that it's essential -- I'm pretty surprised we didn't have one already.

I'm halfway through writing some already.

bergmeister reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Rob Holt (rjmholt) added 2 commits November 8, 2018 23:47
@rjmholt
[Feature] Fix null ref with max version
4e693d2
@rjmholt
[Feature] Fix path comparisons with required modules
ae5c56b
@rjmholt
Rob Holt (rjmholt) force-pushed the fix-required-module-load-failure branch from 3205e0d to ae5c56b Compare November 9, 2018 09:05
@bergmeister

Christoph Bergmeister (bergmeister) commented Nov 9, 2018

Copy link
Copy Markdown
Contributor

Ok, feel free to add the test in a 2nd PR as I can understand that this is a time-pressing issue that can initially be verified with some manual testing (please include Windows as test platform though)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Rob Holt (rjmholt) added 4 commits November 9, 2018 01:55
@rjmholt
[Feature] Match module when required path is base dir
b3c9a87
@rjmholt
[Feature] Trim end of required module path
5c04829
@rjmholt
[Feature] Add testing for required module autoloading failure
28981df
@rjmholt
[Feature] Address CodeFactor issues
cea4dd5
@rjmholt

Rob Holt (rjmholt) commented Nov 9, 2018

Copy link
Copy Markdown
Collaborator Author

We also seem to do required module autoloading in #requires pragmas:

PowerShell/src/System.Management.Automation/engine/CommandDiscovery.cs

Lines 307 to 313 in 4118fd2

ModuleCmdletBase.LoadRequiredModule(
context: context,
currentModule: null,
requiredModuleSpecification: requiredModule,
moduleManifestPath: null,
manifestProcessingFlags: ModuleCmdletBase.ManifestProcessingFlags.LoadElements | ModuleCmdletBase.ManifestProcessingFlags.WriteErrors,
error: out error);

If those are supposed to allow paths, I'll need to do the same path normalisation logic there.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@adityapatwardhan

Aditya Patwardhan (adityapatwardhan) commented Nov 9, 2018

Copy link
Copy Markdown
Member

Rob Holt (@rjmholt) 1 new test is failing on Linux and macOS. Please have a look.

rjmholt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rjmholt
[Feature] Make relative paths work in module specification checks
b8e513a
@rjmholt
Rob Holt (rjmholt) requested a review from Francisco Gamino (Francisco-Gamino) as a code owner November 9, 2018 20:16
Rob Holt (rjmholt) added 10 commits November 9, 2018 12:44
@rjmholt
[Feature] Add copyright header
8a843ce
@rjmholt
[Feature] Remove unused vars from tests
fa950a1
@rjmholt
[Feature] Add more testing for relative paths in fullyqualifiednames
2d9cf9c
@rjmholt
[Feature] Ensure required module path is absolute at autoloading time
55f8a7d
@rjmholt
[Feature] Update Get-Module test to account for version ignoring quirk
d3b9651
@rjmholt
[Feature] Reduce complexity of fix - normalize module spec path where…
980b8f0
… needed
@rjmholt
[Feature] Revert further changes
0a414ff
@rjmholt
[Feature] Add path normalization method to ModuleSpecification
9163bc0
@rjmholt
[Feature] Fix module specification method call
59cf3d1
@rjmholt
[Feature] Revert unneeded changes - change to TODOs.
5b4bd92
@rjmholt

Rob Holt (rjmholt) commented Nov 12, 2018

Copy link
Copy Markdown
Collaborator Author

The failing AppVeyor test is passing on my machine. Looking into it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

daxian-dbw
Dongbo Wang (daxian-dbw) reviewed Nov 13, 2018
View reviewed changes
Comment thread src/System.Management.Automation/engine/Modules/GetModuleCommand.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/ModuleSpecification.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/RemoveModuleCommand.cs
// Paths in the module name may fail here because
// they the wrong directory separator or are relative.
// Fix with the code below:
// FullyQualifiedName = FullyQualifiedName.Select(ms => ms.WithNormalizedName(Context, SessionState.Path.CurrentLocation.Path)).ToArray();

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 13, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the concern about putting this fix in place? Regression? Perf degradation?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@rjmholt Rob Holt (rjmholt) Nov 13, 2018

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just keeping the fix constrained in this PR. It shouldn't cause regressions or perf degradations, but to ensure there are no regressions (since this PR is a regression fix itself) I just left it as a comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 13, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will you open issue to track all those todo comments? They can be easily forgotten :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

rjmholt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 13, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you open an issue to track those todo's?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@rjmholt Rob Holt (rjmholt) Nov 13, 2018

Copy link
Copy Markdown
Collaborator Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next on my list

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Rob Holt (rjmholt) added 5 commits November 12, 2018 18:48
@rjmholt
Make required name null check clearer
5950e75
@rjmholt
Protect against path resolution failure
cbc650e
@rjmholt
Fix Path.AltDirectorySeparatorChar misuse
87a9704
@rjmholt
[Feature] Fix non-const const assignment
2378bb8
@rjmholt
[Feature] Undo change to ModuleCmdletBase.IsRooted
9faf21e
daxian-dbw
Dongbo Wang (daxian-dbw) reviewed Nov 13, 2018
View reviewed changes
Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Robert Holt added 6 commits November 13, 2018 11:27
Robert Holt
[Feature] Improve TODOs in GetModuleCommand.cs
60ece17
Robert Holt
[Feature] Add comments to module path matching logic
c21eb25
Robert Holt
[Feature] Fix up path normalization code
336ba07
Robert Holt
[Feature] Add extra fallback normalization
30c07b8
Robert Holt
[Feature] Fix IsModulePath check method
e381a50
Robert Holt
[Feature] Add full stop to satisfy code factor
9344345
daxian-dbw
Dongbo Wang (daxian-dbw) reviewed Nov 13, 2018
View reviewed changes
Comment thread src/System.Management.Automation/engine/Modules/ModuleIntrinsics.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/System.Management.Automation/engine/Modules/RemoveModuleCommand.cs
// Paths in the module name may fail here because
// they the wrong directory separator or are relative.
// Fix with the code below:
// FullyQualifiedName = FullyQualifiedName.Select(ms => ms.WithNormalizedName(Context, SessionState.Path.CurrentLocation.Path)).ToArray();

@daxian-dbw Dongbo Wang (daxian-dbw) Nov 13, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you open an issue to track those todo's?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Robert Holt
[Feature] Fix comment typo
389b245
daxian-dbw
Dongbo Wang (daxian-dbw) approved these changes Nov 13, 2018
View reviewed changes

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only pending action item left is to open an issue tracking all "TODO" comments.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@adityapatwardhan

Aditya Patwardhan (adityapatwardhan) commented Nov 13, 2018

Copy link
Copy Markdown
Member

Since the CodeFactor issues are for existing methods, re-ordering will cause larger diffs. We can ignore CodeFactor issues for this PR.

bergmeister and rjmholt reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@adityapatwardhan
Aditya Patwardhan (adityapatwardhan) merged commit aa76c68 into PowerShell:master Nov 13, 2018
This was referenced Nov 13, 2018
ModuleSpecification/FullyQualifiedName usage does not handle paths well #8261
Closed
Get-Module [-ListAvailable] -FullyQualifiedName with path ignores other constraints #8262
Closed
@bergmeister Christoph Bergmeister (bergmeister) mentioned this pull request Nov 14, 2018
RequiredModules manifest property stopped working recently (noticed in daily build) #8204
Closed
@iSazonov Ilya (iSazonov) added the CL-Engine Indicates that a PR should be marked as an engine change in the Change Log label Jan 17, 2019
@rjmholt Rob Holt (rjmholt) mentioned this pull request Feb 21, 2019
Refactor ImportModule commandlet #8936
Closed
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
@rjmholt @adityapatwardhan
Fix incorrect name check when autoloading required modules (PowerShel…
3b86f0b
…l#8218)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@daxian-dbw Dongbo Wang (daxian-dbw) daxian-dbw approved these changes
@BrucePay Bruce Payette (BrucePay) Awaiting requested review from BrucePay
@adityapatwardhan Aditya Patwardhan (adityapatwardhan) Awaiting requested review from adityapatwardhan
@TravisEz13 Travis Plunk (TravisEz13) Awaiting requested review from TravisEz13
@Francisco-Gamino Francisco Gamino (Francisco-Gamino) Awaiting requested review from Francisco-Gamino

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@adityapatwardhan Aditya Patwardhan (adityapatwardhan)

Labels

CL-Engine Indicates that a PR should be marked as an engine change in the Change Log

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

RequiredModules manifest property stopped working recently (noticed in daily build)

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@rjmholt @bergmeister @adityapatwardhan @daxian-dbw @iSazonov
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.