GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Refactor module code related to 'Get-Module -ListAvailable' by daxian-dbw · Pull Request #7145 · PowerShell/PowerShell

Conversation

@daxian-dbw

@daxian-dbw Dongbo Wang (daxian-dbw) commented Jun 22, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

PR Summary

The major refactoring changes are:

  • In ModuleIntrisic.cs, remove unneeded Windows-PowerShell-only code.
  • In ModuleUtils.cs
    • use the new API Directory.GetDirectories(string path, string searchPattern, EnumerationOptions enumerationOptions) and Directory.GetFiles(string path, string searchPattern, EnumerationOptions enumerationOptions) to enumerate files and sub-directories within a directory path.
    • remove the unused parameter bool force from GetDefaultAvailableModuleFiles(bool force, bool isForAutoDiscovery, ExecutionContext context)
    • refactor the method GetModuleVersionsFromAbsolutePath. Add more comments and rename the method name.
  • In ModuleCmdletBase.cs, refactor the method GetModuleForNonRootedPaths to GetModuleForNames to simply its implementation.
  • In PSModuleInfo.cs
    • group the declarations of Declared*Exports fields together
    • rename _detected*Exports fields to Detected*Exports to group them together. They are internal fields and used outside PSModuleInfo.

Pref improvement

NOTE: Use Windows PowerShell default module paths to exercies the code with more modules.
The following measurement are done without the crossgen'ed assemblies.

There is some perf improvement after this refactoring change:

  • For Get-Module -ListAvailable, there is about 36% speed improvement for 94 default modules.
  • For Get-Module -ListAvailable -All, there is about 14% speed improvement for totally 600 module files.
  • For Get-Module <name> -ListAvailable -List, there is over 17x speed improvement for finding 13 modules from 600 modules. This is because we now filter names using the module file before creating a PSMdouleInfo object.

Before

PS> $env:PSModulePath = "C:\Users\dongbow\Documents\WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\windows\system32\WindowsPowerShell\v1.0\Modules"
PS> $time = foreach ($i in 1..20) { Measure-Command { gmo -ListAvailable } | % TotalMilliseconds }
PS> $time | Measure-Object -Maximum -Minimum -Average | select Count, Maximum, Minimum, Average

Count  Maximum  Minimum    Average
-----  -------  -------    -------
   20 701.3139 623.5225 642.628685

PS> $time = foreach ($i in 1..20) { Measure-Command { gmo -ListAvailable -All } | % TotalMilliseconds }
PS> $time | Measure-Object -Maximum -Minimum -Average | select Count, Maximum, Minimum, Average

Count   Maximum   Minimum     Average
-----   -------   -------     -------
   20 1369.1893 1243.1195 1310.147175

PS> $time = foreach ($i in 1..20) { Measure-Command { gmo windows* -ListAvailable -All } | % TotalMilliseconds }
PS> $time | Measure-Object -Maximum -Minimum -Average | select Count, Maximum, Minimum, Average

Count   Maximum   Minimum    Average
-----   -------   -------    -------
   20 1535.3805 1292.2188 1353.62972

After

PS> $env:PSModulePath = "C:\Users\dongbow\Documents\WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\windows\system32\WindowsPowerShell\v1.0\Modules"
PS> $time = foreach ($i in 1..20) { Measure-Command { gmo -ListAvailable } | % TotalMilliseconds }
PS> $time | Measure-Object -Maximum -Minimum -Average | select Count, Maximum, Minimum, Average

Count Maximum  Minimum  Average
----- -------  -------  -------
   20 438.338 393.7476 410.62353

PS> $time = foreach ($i in 1..20) { Measure-Command { gmo -ListAvailable -All } | % TotalMilliseconds }
PS> $time | Measure-Object -Maximum -Minimum -Average | select Count, Maximum, Minimum, Average

Count   Maximum   Minimum Average
-----   -------   -------   -------
   20 1341.5443 1055.1771 1120.4748

PS> $time = foreach ($i in 1..20) { Measure-Command { gmo windows* -ListAvailable -All } | % TotalMilliseconds }
PS> $time | Measure-Object -Maximum -Minimum -Average | select Count, Maximum, Minimum, Average

Count Maximum Minimum Average
----- ------- -------   -------
   20 78.0931 61.9288 66.194415

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Change is not breaking
  • Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.
  • User-facing changes
    • Not Applicable
    • OR
    • User-facing Documentation needed
      • Issue filed - Issue link:
  • Testing - New and feature
    • Not Applicable or can only be tested interactively
    • OR
    • Make sure you've added a new test if existing tests do not effectively test the code changed
      • Add [feature] if the change is significant or affects feature tests

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

rjmholt reacted with thumbs up emoji rjmholt reacted with hooray emoji rjmholt and powercode reacted with heart emoji
All reactions
  • 👍 1 reaction
  • 🎉 1 reaction
  • ❤️ 2 reactions
@daxian-dbw
Dongbo Wang (daxian-dbw) requested a review from Rob Holt (rjmholt) June 22, 2018 19:42
@daxian-dbw
Dongbo Wang (daxian-dbw) requested a review from Bruce Payette (BrucePay) as a code owner June 22, 2018 19:42
@daxian-dbw
Dongbo Wang (daxian-dbw) removed the request for review from Bruce Payette (BrucePay) June 22, 2018 19:42
@daxian-dbw Dongbo Wang (daxian-dbw) added the Area-Maintainers-Documentation specific to documentation in this repo label Jun 22, 2018
@daxian-dbw
Dongbo Wang (daxian-dbw) force-pushed the moduleRefactor branch from c4a82f7 to ce51f0f Compare June 22, 2018 21:17
rjmholt
Rob Holt (rjmholt) reviewed Jun 22, 2018
View reviewed changes
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance we could turn the else here into a continue while we're at it? (I noticed foundModule only gets used after this)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method GetModuleForRootedPaths is massively refactored to change the behavior of Get-Module <directory-path> -ListAvailable and Get-Module <directory-path> -ListAvailable -All. The new behvaior will make them the same as if running Get-Module -ListAvailable and Get-Module -ListAvailable -All when $env:PSModulePath == <directory-path>.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/InitialSessionState.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hooray for this and the one above!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we give the type here instead of the var? The ternary method call makes it less than obvious.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the nature of the change here? Why do we not need to do the duplicate checks now?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method GetModuleForRootedPaths is massively refactored. Please review again. Thanks!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The formatter for Get-Module -ListAvailable does logic like this, and I think it would be very helpful as a property or method on PSModuleInfo in general. I know we've had to do the same logic in PowerShellEditorServices at some point.

I feel like we ought to factor it into a method. That and I'm just averse to such a large inline delegate.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OrderBy call was previous OrderBy(m => m.Name). However, simply ordering by name will cause the modules from the same parent folder to be divided by some modules in more nested sub-folders. I have moved this delegate to a method in ModuleUtils.cs.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleUtils.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe String.IsNullOrEmpty is better here?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bergmeister reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is removed now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleUtils.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume rather than throw this returns null or ""?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method is removed now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 29, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method returns an empty string. I changed to use string.IsNullOrEmpty below.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleUtils.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this method return a Version?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It returns List<Version>. But the method is removed now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleUtils.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic in this method looks very similar to the Import-Module local load logic:

PowerShell/src/System.Management.Automation/engine/Modules/ImportModuleCommand.cs

Lines 530 to 760 in bbb4f2e

private PSModuleInfo ImportModule_LocallyViaName(ImportModuleOptions importModuleOptions, string name)
{
try
{
bool found = false;
PSModuleInfo foundModule = null;
string cachedPath = null;
string rootedPath = null;
// See if we can use the cached path for the file. If a version number has been specified, then
// we won't look in the cache
if (this.MinimumVersion == null && this.MaximumVersion == null && this.RequiredVersion == null && PSModuleInfo.UseAppDomainLevelModuleCache && !this.BaseForce)
{
// See if the name is in the appdomain-level module path name cache...
cachedPath = PSModuleInfo.ResolveUsingAppDomainLevelModuleCache(name);
}
if (!string.IsNullOrEmpty(cachedPath))
{
if (File.Exists(cachedPath))
{
rootedPath = cachedPath;
}
else
{
PSModuleInfo.RemoveFromAppDomainLevelCache(name);
}
}
if (rootedPath == null)
{
// Check for full-qualified paths - either absolute or relative
rootedPath = ResolveRootedFilePath(name, this.Context);
}
bool alreadyLoaded = false;
if (!String.IsNullOrEmpty(rootedPath))
{
// TODO/FIXME: use IsModuleAlreadyLoaded to get consistent behavior
// TODO/FIXME: (for example checking ModuleType != Manifest below seems incorrect - cdxml modules also declare their own version)
// PSModuleInfo alreadyLoadedModule = null;
// Context.Modules.ModuleTable.TryGetValue(rootedPath, out alreadyLoadedModule);
// if (!BaseForce && IsModuleAlreadyLoaded(alreadyLoadedModule))
// If the module has already been loaded, just emit it and continue...
PSModuleInfo module;
if (!BaseForce && Context.Modules.ModuleTable.TryGetValue(rootedPath, out module))
{
if (RequiredVersion == null
|| module.Version.Equals(RequiredVersion)
|| (BaseMinimumVersion == null && BaseMaximumVersion == null)
|| module.ModuleType != ModuleType.Manifest
|| (BaseMinimumVersion == null && BaseMaximumVersion != null && module.Version <= BaseMaximumVersion)
|| (BaseMinimumVersion != null && BaseMaximumVersion == null && module.Version >= BaseMinimumVersion)
|| (BaseMinimumVersion != null && BaseMaximumVersion != null && module.Version >= BaseMinimumVersion && module.Version <= BaseMaximumVersion))
{
alreadyLoaded = true;
AddModuleToModuleTables(this.Context, this.TargetSessionState.Internal, module);
ImportModuleMembers(module, this.BasePrefix, importModuleOptions);
if (BaseAsCustomObject)
{
if (module.ModuleType != ModuleType.Script)
{
string message = StringUtil.Format(Modules.CantUseAsCustomObjectWithBinaryModule, module.Path);
InvalidOperationException invalidOp = new InvalidOperationException(message);
ErrorRecord er = new ErrorRecord(invalidOp, "Modules_CantUseAsCustomObjectWithBinaryModule",
ErrorCategory.PermissionDenied, null);
WriteError(er);
}
else
{
WriteObject(module.AsCustomObject());
}
}
else if (BasePassThru)
{
WriteObject(module);
}
found = true;
foundModule = module;
}
}
if (!alreadyLoaded)
{
// If the path names a file, load that file...
if (File.Exists(rootedPath))
{
PSModuleInfo moduleToRemove;
if (Context.Modules.ModuleTable.TryGetValue(rootedPath, out moduleToRemove))
{
RemoveModule(moduleToRemove);
}
foundModule = LoadModule(rootedPath, null, this.BasePrefix, null, ref importModuleOptions,
ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError,
out found);
}
else if (Directory.Exists(rootedPath))
{
// If the path ends with a directory separator, remove it
if (rootedPath.EndsWith(Path.DirectorySeparatorChar))
{
rootedPath = Path.GetDirectoryName(rootedPath);
}
// Load the latest valid version if it is a multi-version module directory
foundModule = LoadUsingMultiVersionModuleBase(rootedPath,
ManifestProcessingFlags.LoadElements |
ManifestProcessingFlags.WriteErrors |
ManifestProcessingFlags.NullOnFirstError,
importModuleOptions, out found);
if (!found)
{
// If the path is a directory, double up the end of the string
// then try to load that using extensions...
rootedPath = Path.Combine(rootedPath, Path.GetFileName(rootedPath));
foundModule = LoadUsingExtensions(null, rootedPath, rootedPath, null, null, this.BasePrefix, /*SessionState*/ null,
importModuleOptions,
ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError,
out found);
}
}
}
}
else
{
// Check if module could be a snapin. This was the case for PowerShell version 2 engine modules.
if (InitialSessionState.IsEngineModule(name))
{
PSSnapInInfo snapin = ModuleCmdletBase.GetEngineSnapIn(Context, name);
// Return the command if we found a module
if (snapin != null)
{
// warn that this module already exists as a snapin
string warningMessage = string.Format(
CultureInfo.InvariantCulture,
Modules.ModuleLoadedAsASnapin,
snapin.Name);
WriteWarning(warningMessage);
found = true;
return foundModule;
}
}
// At this point, the name didn't resolve to an existing file or directory.
// It may still be rooted (relative or absolute). If it is, then we'll only use
// the extension search. If it's not rooted, use a path-based search.
if (IsRooted(name))
{
// If there is no extension, we'll have to search using the extensions
if (!string.IsNullOrEmpty(Path.GetExtension(name)))
{
foundModule = LoadModule(name, null, this.BasePrefix, null, ref importModuleOptions,
ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError,
out found);
}
else
{
foundModule = LoadUsingExtensions(null, name, name, null, null, this.BasePrefix, /*SessionState*/ null,
importModuleOptions,
ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError,
out found);
}
}
else
{
IEnumerable<string> modulePath = ModuleIntrinsics.GetModulePath(false, this.Context);
if (this.MinimumVersion == null && this.RequiredVersion == null && this.MaximumVersion == null)
{
this.AddToAppDomainLevelCache = true;
}
found = LoadUsingModulePath(found, modulePath, name, /* SessionState*/ null,
importModuleOptions,
ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError,
out foundModule);
}
}
if (!found)
{
ErrorRecord er = null;
string message = null;
if (BaseRequiredVersion != null)
{
message = StringUtil.Format(Modules.ModuleWithVersionNotFound, name, BaseRequiredVersion);
}
else if (BaseMinimumVersion != null && BaseMaximumVersion != null)
{
message = StringUtil.Format(Modules.MinimumVersionAndMaximumVersionNotFound, name, BaseMinimumVersion, BaseMaximumVersion);
}
else if (BaseMinimumVersion != null)
{
message = StringUtil.Format(Modules.ModuleWithVersionNotFound, name, BaseMinimumVersion);
}
else if (BaseMaximumVersion != null)
{
message = StringUtil.Format(Modules.MaximumVersionNotFound, name, BaseMaximumVersion);
}
if (BaseRequiredVersion != null || BaseMinimumVersion != null || BaseMaximumVersion != null)
{
FileNotFoundException fnf = new FileNotFoundException(message);
er = new ErrorRecord(fnf, "Modules_ModuleWithVersionNotFound",
ErrorCategory.ResourceUnavailable, name);
}
else
{
message = StringUtil.Format(Modules.ModuleNotFound, name);
FileNotFoundException fnf = new FileNotFoundException(message);
er = new ErrorRecord(fnf, "Modules_ModuleNotFound",
ErrorCategory.ResourceUnavailable, name);
}
WriteError(er);
}
return foundModule;
}
catch (PSInvalidOperationException e)
{
ErrorRecord er = new ErrorRecord(e.ErrorRecord, e);
WriteError(er);
}
return null;
}

Where we first check the path, then the folder, then the versions, then the extensions, etc.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleUtils.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 22, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments here as above

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 25, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a new line.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@daxian-dbw Dongbo Wang (daxian-dbw) added the Breaking-Change breaking change that may affect users label Jun 25, 2018
@daxian-dbw
Dongbo Wang (daxian-dbw) force-pushed the moduleRefactor branch from 9cf67ae to a24fee9 Compare June 25, 2018 21:31
@daxian-dbw Dongbo Wang (daxian-dbw) mentioned this pull request Jun 28, 2018
'Get-Module <Path-to-directory> -ListAvailable -All' should return all modules within the directory #7160
Closed
@daxian-dbw
Dongbo Wang (daxian-dbw) force-pushed the moduleRefactor branch 3 times, most recently from 4ed9b45 to cdff125 Compare June 28, 2018 21:44
@daxian-dbw Dongbo Wang (daxian-dbw) changed the title [WIP] Refactor module code related to 'Get-Module -ListAvailable' Refactor module code related to 'Get-Module -ListAvailable' Jun 28, 2018
@daxian-dbw Dongbo Wang (daxian-dbw) removed the Breaking-Change breaking change that may affect users label Jun 28, 2018
@daxian-dbw

Dongbo Wang (daxian-dbw) commented Jun 28, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Rob Holt (@rjmholt) I have reverted the breaking change I made to the scenario of Get-Module <Path-to-directory> -ListAvailable (please see details in #7160). Now the changes introduce no change to the current behaviors. Can you please continue the review?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@daxian-dbw

Dongbo Wang (daxian-dbw) commented Jun 29, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Added tests for different scenarios of Get-Module -ListAvailable and Get-Module -ListAvailable -All. All those tests pass in pwsh with or without changes in this PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@daxian-dbw
Dongbo Wang (daxian-dbw) force-pushed the moduleRefactor branch from 4c1c680 to 8663d78 Compare June 29, 2018 18:32
@daxian-dbw
[Feature] Refactor module code related to 'Get-Module -ListAvailable'
ff97ae2
@daxian-dbw
Dongbo Wang (daxian-dbw) force-pushed the moduleRefactor branch from 8663d78 to 1363170 Compare June 29, 2018 18:36
rjmholt
Rob Holt (rjmholt) reviewed Jun 29, 2018
View reviewed changes
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 29, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How hard is the requirement on names being an array? I only ask because I notice this part of the code tends to have collection types in signatures vary a lot (some IEnumerable<string>, some string[], some List<string>) and we should probably work out a better rationale for what types we prefer.

If there is a particular performance implication from using string[] then that makes sense, but in general I think we should favour the principle of using an interface with only the properties required by each method; if we only want to go through entries sequentially, IEnumerable<T>, if we want to know how many there are up front, something like ICollection<T>, if we want indexing IList<T>.

I'm not making a particular suggestion here I suppose. I just have the impression that the various collection types used in the methods in the module cmdlets weren't used for any particular reason, and we should be more judicious about what collection types we use (and the semantics of each) given how intensively we handle them.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 29, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this method, it makes more sense to use List<string>, so the top-level caller doesn't need to cast the List to a string array. I have updated the code.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs Outdated

@rjmholt Rob Holt (rjmholt) Jun 29, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment above I read as saying "we can skip analysis if all of functions/cmdlets/aliases to export are given".

But I would translate that as the condition:

!needToAnalyzeScriptModules && (sawExportedCmdlets && sawExportedFunctions && sawExportedAliases))

The expression here is equivalent to:

!needToAnalyzeScriptModules && !(sawExportedCmdlets && sawExportedFunctions && sawExportedAliases))

Which I think would be "analysis is not required and any of functions/cmdlets/aliases to export is missing".

Do the variable names mean something converse to how I read them?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 29, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment above I read as saying "we can skip analysis if all of functions/cmdlets/aliases to export are given".

That's correct.
That means if needToAnalyzeScriptModules == false when we reaching here, we might still change it to true if we didn't see one of "FunctionToExport", "CmdletToExport" and "AliasToExport".

One thing worth to note is the if block here is not doing further analysis, but checking if we really need to do further analysis -- it's changing the value of needToAnalyzeScriptModules to true.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@rjmholt Rob Holt (rjmholt) Jun 29, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right! The comment is slightly unclear to me because I didn't realise that the if is really a check to see if we maybe do need to do further analysis.

I would have worded it more as "If any of 'FunctionsToExport', 'CmdletsToExport' or 'AliasesToExport' were not given, we must check to see if more analysis is needed". Not saying the wording should be that way, just submitting an alternate description

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 29, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. I like your comment better. Will update.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread test/powershell/Modules/Microsoft.PowerShell.Core/Get-Module.Tests.ps1 Outdated

@rjmholt Rob Holt (rjmholt) Jun 29, 2018

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A much needed file!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

daxian-dbw reacted with laugh emoji
All reactions
  • 😄 1 reaction

@daxian-dbw Dongbo Wang (daxian-dbw) Jun 29, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully, more tests can be added gradually.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@daxian-dbw
[Feature] Add tests for 'Get-Module -List -All'
1ca7e12
@daxian-dbw
Dongbo Wang (daxian-dbw) force-pushed the moduleRefactor branch from 1363170 to 1ca7e12 Compare June 29, 2018 19:44
@daxian-dbw
[Feature] Address feedback about the 'string[]' parameter
406dfc2
rjmholt
Rob Holt (rjmholt) approved these changes Jun 29, 2018
View reviewed changes

@rjmholt Rob Holt (rjmholt) left a comment

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this looks good to me now!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@daxian-dbw
Update a comment based on feedback
4211401
@daxian-dbw Dongbo Wang (daxian-dbw) removed the Area-Maintainers-Documentation specific to documentation in this repo label Jul 1, 2018
@daxian-dbw
Dongbo Wang (daxian-dbw) merged commit 0d9a3ac into PowerShell:master Jul 2, 2018
@daxian-dbw
Dongbo Wang (daxian-dbw) deleted the moduleRefactor branch July 2, 2018 18:00
@SteveL-MSFT Steve Lee (SteveL-MSFT) mentioned this pull request Sep 25, 2018
Ensure NestedModules property gets populated by Test-ModuleManifest #7859
Merged
7 of 11 tasks
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
@daxian-dbw
Refactor module code related to 'Get-Module -ListAvailable' (PowerShe…
a1f7440
…ll#7145)

* The major refactoring changes are:
- In `ModuleIntrisic.cs`, remove unneeded Windows-PowerShell-only code.
- In `ModuleUtils.cs`
   - use the new API `Directory.GetDirectories(string path, string searchPattern, EnumerationOptions enumerationOptions)` and `Directory.GetFiles(string path, string searchPattern, EnumerationOptions enumerationOptions)` to enumerate files and sub-directories within a directory path.
   - remove the unused parameter `bool force` from `GetDefaultAvailableModuleFiles(bool force, bool isForAutoDiscovery, ExecutionContext context)`
   - refactor the method `GetModuleVersionsFromAbsolutePath`. Add more comments and rename the method name.
- In `ModuleCmdletBase.cs`, refactor the method `GetModuleForNonRootedPaths` to `GetModuleForNames` to simply its implementation.
- In `PSModuleInfo.cs`
   - group the declarations of `Declared*Exports` fields together
   - rename `_detected*Exports` fields to `Detected*Exports` to group them together. They are internal fields and used outside `PSModuleInfo`.

* There is some perf improvement after this refactoring change:
- For `Get-Module -ListAvailable`, there is about 36% speed improvement for 94 default modules.
- For `Get-Module -ListAvailable -All`, there is about 14% speed improvement for totally 600 module files.
- For `Get-Module <name> -ListAvailable -List`, there is over 17x speed improvement for finding 13 modules from 600 modules. This is because we now filter names using the module file before creating a `PSMdouleInfo` object.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
@bergmeister Christoph Bergmeister (bergmeister) bergmeister left review comments
@rjmholt Rob Holt (rjmholt) rjmholt approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@daxian-dbw @rjmholt @bergmeister
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.