GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Refactor module code related to 'Get-Module -ListAvailable' by daxian-dbw · Pull Request #7145 · PowerShell/PowerShell Conversation
|
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Any chance we could turn the else here into a continue while we're at it? (I noticed foundModule only gets used after this)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The method GetModuleForRootedPaths is massively refactored to change the behavior of Get-Module <directory-path> -ListAvailable and Get-Module <directory-path> -ListAvailable -All. The new behvaior will make them the same as if running Get-Module -ListAvailable and Get-Module -ListAvailable -All when $env:PSModulePath == <directory-path>.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Hooray for this and the one above!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Could we give the type here instead of the var? The ternary method call makes it less than obvious.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
What's the nature of the change here? Why do we not need to do the duplicate checks now?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The method GetModuleForRootedPaths is massively refactored. Please review again. Thanks!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The formatter for Get-Module -ListAvailable does logic like this, and I think it would be very helpful as a property or method on PSModuleInfo in general. I know we've had to do the same logic in PowerShellEditorServices at some point.
I feel like we ought to factor it into a method. That and I'm just averse to such a large inline delegate.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The OrderBy call was previous OrderBy(m => m.Name). However, simply ordering by name will cause the modules from the same parent folder to be divided by some modules in more nested sub-folders. I have moved this delegate to a method in ModuleUtils.cs.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Maybe String.IsNullOrEmpty is better here?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This method is removed now.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I assume rather than throw this returns null or ""?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The method is removed now.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This method returns an empty string. I changed to use string.IsNullOrEmpty below.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Does this method return a Version?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It returns List<Version>. But the method is removed now.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The logic in this method looks very similar to the Import-Module local load logic:
PowerShell/src/System.Management.Automation/engine/Modules/ImportModuleCommand.cs
Lines 530 to 760 in bbb4f2e
| private PSModuleInfo ImportModule_LocallyViaName(ImportModuleOptions importModuleOptions, string name) | |
| { | |
| try | |
| { | |
| bool found = false; | |
| PSModuleInfo foundModule = null; | |
| string cachedPath = null; | |
| string rootedPath = null; | |
| // See if we can use the cached path for the file. If a version number has been specified, then | |
| // we won't look in the cache | |
| if (this.MinimumVersion == null && this.MaximumVersion == null && this.RequiredVersion == null && PSModuleInfo.UseAppDomainLevelModuleCache && !this.BaseForce) | |
| { | |
| // See if the name is in the appdomain-level module path name cache... | |
| cachedPath = PSModuleInfo.ResolveUsingAppDomainLevelModuleCache(name); | |
| } | |
| if (!string.IsNullOrEmpty(cachedPath)) | |
| { | |
| if (File.Exists(cachedPath)) | |
| { | |
| rootedPath = cachedPath; | |
| } | |
| else | |
| { | |
| PSModuleInfo.RemoveFromAppDomainLevelCache(name); | |
| } | |
| } | |
| if (rootedPath == null) | |
| { | |
| // Check for full-qualified paths - either absolute or relative | |
| rootedPath = ResolveRootedFilePath(name, this.Context); | |
| } | |
| bool alreadyLoaded = false; | |
| if (!String.IsNullOrEmpty(rootedPath)) | |
| { | |
| // TODO/FIXME: use IsModuleAlreadyLoaded to get consistent behavior | |
| // TODO/FIXME: (for example checking ModuleType != Manifest below seems incorrect - cdxml modules also declare their own version) | |
| // PSModuleInfo alreadyLoadedModule = null; | |
| // Context.Modules.ModuleTable.TryGetValue(rootedPath, out alreadyLoadedModule); | |
| // if (!BaseForce && IsModuleAlreadyLoaded(alreadyLoadedModule)) | |
| // If the module has already been loaded, just emit it and continue... | |
| PSModuleInfo module; | |
| if (!BaseForce && Context.Modules.ModuleTable.TryGetValue(rootedPath, out module)) | |
| { | |
| if (RequiredVersion == null | |
| || module.Version.Equals(RequiredVersion) | |
| || (BaseMinimumVersion == null && BaseMaximumVersion == null) | |
| || module.ModuleType != ModuleType.Manifest | |
| || (BaseMinimumVersion == null && BaseMaximumVersion != null && module.Version <= BaseMaximumVersion) | |
| || (BaseMinimumVersion != null && BaseMaximumVersion == null && module.Version >= BaseMinimumVersion) | |
| || (BaseMinimumVersion != null && BaseMaximumVersion != null && module.Version >= BaseMinimumVersion && module.Version <= BaseMaximumVersion)) | |
| { | |
| alreadyLoaded = true; | |
| AddModuleToModuleTables(this.Context, this.TargetSessionState.Internal, module); | |
| ImportModuleMembers(module, this.BasePrefix, importModuleOptions); | |
| if (BaseAsCustomObject) | |
| { | |
| if (module.ModuleType != ModuleType.Script) | |
| { | |
| string message = StringUtil.Format(Modules.CantUseAsCustomObjectWithBinaryModule, module.Path); | |
| InvalidOperationException invalidOp = new InvalidOperationException(message); | |
| ErrorRecord er = new ErrorRecord(invalidOp, "Modules_CantUseAsCustomObjectWithBinaryModule", | |
| ErrorCategory.PermissionDenied, null); | |
| WriteError(er); | |
| } | |
| else | |
| { | |
| WriteObject(module.AsCustomObject()); | |
| } | |
| } | |
| else if (BasePassThru) | |
| { | |
| WriteObject(module); | |
| } | |
| found = true; | |
| foundModule = module; | |
| } | |
| } | |
| if (!alreadyLoaded) | |
| { | |
| // If the path names a file, load that file... | |
| if (File.Exists(rootedPath)) | |
| { | |
| PSModuleInfo moduleToRemove; | |
| if (Context.Modules.ModuleTable.TryGetValue(rootedPath, out moduleToRemove)) | |
| { | |
| RemoveModule(moduleToRemove); | |
| } | |
| foundModule = LoadModule(rootedPath, null, this.BasePrefix, null, ref importModuleOptions, | |
| ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError, | |
| out found); | |
| } | |
| else if (Directory.Exists(rootedPath)) | |
| { | |
| // If the path ends with a directory separator, remove it | |
| if (rootedPath.EndsWith(Path.DirectorySeparatorChar)) | |
| { | |
| rootedPath = Path.GetDirectoryName(rootedPath); | |
| } | |
| // Load the latest valid version if it is a multi-version module directory | |
| foundModule = LoadUsingMultiVersionModuleBase(rootedPath, | |
| ManifestProcessingFlags.LoadElements | | |
| ManifestProcessingFlags.WriteErrors | | |
| ManifestProcessingFlags.NullOnFirstError, | |
| importModuleOptions, out found); | |
| if (!found) | |
| { | |
| // If the path is a directory, double up the end of the string | |
| // then try to load that using extensions... | |
| rootedPath = Path.Combine(rootedPath, Path.GetFileName(rootedPath)); | |
| foundModule = LoadUsingExtensions(null, rootedPath, rootedPath, null, null, this.BasePrefix, /*SessionState*/ null, | |
| importModuleOptions, | |
| ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError, | |
| out found); | |
| } | |
| } | |
| } | |
| } | |
| else | |
| { | |
| // Check if module could be a snapin. This was the case for PowerShell version 2 engine modules. | |
| if (InitialSessionState.IsEngineModule(name)) | |
| { | |
| PSSnapInInfo snapin = ModuleCmdletBase.GetEngineSnapIn(Context, name); | |
| // Return the command if we found a module | |
| if (snapin != null) | |
| { | |
| // warn that this module already exists as a snapin | |
| string warningMessage = string.Format( | |
| CultureInfo.InvariantCulture, | |
| Modules.ModuleLoadedAsASnapin, | |
| snapin.Name); | |
| WriteWarning(warningMessage); | |
| found = true; | |
| return foundModule; | |
| } | |
| } | |
| // At this point, the name didn't resolve to an existing file or directory. | |
| // It may still be rooted (relative or absolute). If it is, then we'll only use | |
| // the extension search. If it's not rooted, use a path-based search. | |
| if (IsRooted(name)) | |
| { | |
| // If there is no extension, we'll have to search using the extensions | |
| if (!string.IsNullOrEmpty(Path.GetExtension(name))) | |
| { | |
| foundModule = LoadModule(name, null, this.BasePrefix, null, ref importModuleOptions, | |
| ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError, | |
| out found); | |
| } | |
| else | |
| { | |
| foundModule = LoadUsingExtensions(null, name, name, null, null, this.BasePrefix, /*SessionState*/ null, | |
| importModuleOptions, | |
| ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError, | |
| out found); | |
| } | |
| } | |
| else | |
| { | |
| IEnumerable<string> modulePath = ModuleIntrinsics.GetModulePath(false, this.Context); | |
| if (this.MinimumVersion == null && this.RequiredVersion == null && this.MaximumVersion == null) | |
| { | |
| this.AddToAppDomainLevelCache = true; | |
| } | |
| found = LoadUsingModulePath(found, modulePath, name, /* SessionState*/ null, | |
| importModuleOptions, | |
| ManifestProcessingFlags.LoadElements | ManifestProcessingFlags.WriteErrors | ManifestProcessingFlags.NullOnFirstError, | |
| out foundModule); | |
| } | |
| } | |
| if (!found) | |
| { | |
| ErrorRecord er = null; | |
| string message = null; | |
| if (BaseRequiredVersion != null) | |
| { | |
| message = StringUtil.Format(Modules.ModuleWithVersionNotFound, name, BaseRequiredVersion); | |
| } | |
| else if (BaseMinimumVersion != null && BaseMaximumVersion != null) | |
| { | |
| message = StringUtil.Format(Modules.MinimumVersionAndMaximumVersionNotFound, name, BaseMinimumVersion, BaseMaximumVersion); | |
| } | |
| else if (BaseMinimumVersion != null) | |
| { | |
| message = StringUtil.Format(Modules.ModuleWithVersionNotFound, name, BaseMinimumVersion); | |
| } | |
| else if (BaseMaximumVersion != null) | |
| { | |
| message = StringUtil.Format(Modules.MaximumVersionNotFound, name, BaseMaximumVersion); | |
| } | |
| if (BaseRequiredVersion != null || BaseMinimumVersion != null || BaseMaximumVersion != null) | |
| { | |
| FileNotFoundException fnf = new FileNotFoundException(message); | |
| er = new ErrorRecord(fnf, "Modules_ModuleWithVersionNotFound", | |
| ErrorCategory.ResourceUnavailable, name); | |
| } | |
| else | |
| { | |
| message = StringUtil.Format(Modules.ModuleNotFound, name); | |
| FileNotFoundException fnf = new FileNotFoundException(message); | |
| er = new ErrorRecord(fnf, "Modules_ModuleNotFound", | |
| ErrorCategory.ResourceUnavailable, name); | |
| } | |
| WriteError(er); | |
| } | |
| return foundModule; | |
| } | |
| catch (PSInvalidOperationException e) | |
| { | |
| ErrorRecord er = new ErrorRecord(e.ErrorRecord, e); | |
| WriteError(er); | |
| } | |
| return null; | |
| } |
Where we first check the path, then the folder, then the versions, then the extensions, etc.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Same comments here as above
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Add a new line.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
9cf67ae to
a24fee9
Compare
4ed9b45 to
cdff125
Compare
There was an error while loading. Please reload this page.
|
Rob Holt (@rjmholt) I have reverted the breaking change I made to the scenario of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Added tests for different scenarios of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
4c1c680 to
8663d78
Compare
8663d78 to
1363170
Compare
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
How hard is the requirement on names being an array? I only ask because I notice this part of the code tends to have collection types in signatures vary a lot (some IEnumerable<string>, some string[], some List<string>) and we should probably work out a better rationale for what types we prefer.
If there is a particular performance implication from using string[] then that makes sense, but in general I think we should favour the principle of using an interface with only the properties required by each method; if we only want to go through entries sequentially, IEnumerable<T>, if we want to know how many there are up front, something like ICollection<T>, if we want indexing IList<T>.
I'm not making a particular suggestion here I suppose. I just have the impression that the various collection types used in the methods in the module cmdlets weren't used for any particular reason, and we should be more judicious about what collection types we use (and the semantics of each) given how intensively we handle them.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
For this method, it makes more sense to use List<string>, so the top-level caller doesn't need to cast the List to a string array. I have updated the code.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The comment above I read as saying "we can skip analysis if all of functions/cmdlets/aliases to export are given".
But I would translate that as the condition:
!needToAnalyzeScriptModules && (sawExportedCmdlets && sawExportedFunctions && sawExportedAliases))
The expression here is equivalent to:
!needToAnalyzeScriptModules && !(sawExportedCmdlets && sawExportedFunctions && sawExportedAliases))
Which I think would be "analysis is not required and any of functions/cmdlets/aliases to export is missing".
Do the variable names mean something converse to how I read them?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The comment above I read as saying "we can skip analysis if all of functions/cmdlets/aliases to export are given".
That's correct.
That means if needToAnalyzeScriptModules == false when we reaching here, we might still change it to true if we didn't see one of "FunctionToExport", "CmdletToExport" and "AliasToExport".
One thing worth to note is the if block here is not doing further analysis, but checking if we really need to do further analysis -- it's changing the value of needToAnalyzeScriptModules to true.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Oh right! The comment is slightly unclear to me because I didn't realise that the if is really a check to see if we maybe do need to do further analysis.
I would have worded it more as "If any of 'FunctionsToExport', 'CmdletsToExport' or 'AliasesToExport' were not given, we must check to see if more analysis is needed". Not saying the wording should be that way, just submitting an alternate description
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Good suggestion. I like your comment better. Will update.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
A much needed file!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Hopefully, more tests can be added gradually.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
1363170 to
1ca7e12
Compare
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Ok, this looks good to me now!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…ll#7145) * The major refactoring changes are: - In `ModuleIntrisic.cs`, remove unneeded Windows-PowerShell-only code. - In `ModuleUtils.cs` - use the new API `Directory.GetDirectories(string path, string searchPattern, EnumerationOptions enumerationOptions)` and `Directory.GetFiles(string path, string searchPattern, EnumerationOptions enumerationOptions)` to enumerate files and sub-directories within a directory path. - remove the unused parameter `bool force` from `GetDefaultAvailableModuleFiles(bool force, bool isForAutoDiscovery, ExecutionContext context)` - refactor the method `GetModuleVersionsFromAbsolutePath`. Add more comments and rename the method name. - In `ModuleCmdletBase.cs`, refactor the method `GetModuleForNonRootedPaths` to `GetModuleForNames` to simply its implementation. - In `PSModuleInfo.cs` - group the declarations of `Declared*Exports` fields together - rename `_detected*Exports` fields to `Detected*Exports` to group them together. They are internal fields and used outside `PSModuleInfo`. * There is some perf improvement after this refactoring change: - For `Get-Module -ListAvailable`, there is about 36% speed improvement for 94 default modules. - For `Get-Module -ListAvailable -All`, there is about 14% speed improvement for totally 600 module files. - For `Get-Module <name> -ListAvailable -List`, there is over 17x speed improvement for finding 13 modules from 600 modules. This is because we now filter names using the module file before creating a `PSMdouleInfo` object.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.