GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||
GeezerVison is showing github.com — Fix Parameter Binder bug with Advanced Functions and ValueFromRemainingArguments by dlwyatt · Pull Request #2038 · PowerShell/PowerShell Conversation
|
|
I think this is a bucket 2 breaking change, so maybe we need an RFC. Dongbo Wang (@daxian-dbw) - can you review? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
If someone's depending on this difference in behavior between |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
this shouldn't been changed even if the fix to parameter binder is accepted. it will break the following scenario:
Write-Output aa,bb cc,dd
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Hmm, that's tricky. In fact I'd say it's been a bug in Write-Output all along.
Write-Output aa,bb cc,dd # is equivalent to Write-Output -InputObject @( @('aa', 'bb'), @('cc', 'dd') )
Either way, InputObject is a 2-element array, and each element also happens to be a 2-element array. What I would expect to happen here is what happens after this PR: You send two objects down the pipeline, both of which are 2-element arrays. If you added the -NoEnumerate switch, then you would send one object down the pipeline, which is a 2-element array containing two other 2-element arrays. (In fact, with the -NoEnumerate switch, you should only ever send one object, an array, down the pipeline, which is sort of the point.)
Today, Write-Output aa,bb cc,dd instead sends 4 strings down the pipeline. It enumerates two levels deep instead of one. If you use the -NoEnumerate switch on that command today, you get two 2-element arrays going down the pipeline (what I'd expect to see without -NoEnumerate).
Without this change, several other Write-Output tests failed when I ran Start-PSPester. With this change, everything was green, but apparently that just means that the test coverage isn't good enough right now.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Here are the results of Write-Output.Tests.ps1 with the binder fix but without the change to Write-Output:
Describing Write-Output DRT Unit Tests
[-] Simple Write Object Test 1.28s
Expected: {4}
But was: {6}
at line: 5 in C:\Users\dlwya_000\Documents\GitHub\powershell\test\powershell\Modules\Microsoft.PowerShell.Utility\Write-Output.Tests.ps1
5: $results.Length | Should Be $objectWritten.Length
[-] Works with NoEnumerate switch 144ms
Expected: {1 2.2 System.Object[] abc}
But was: {1}
at line: 19 in C:\Users\dlwya_000\Documents\GitHub\powershell\test\powershell\Modules\Microsoft.PowerShell.Utility\Write-Output.Tests.ps1
19: Write-Output $objectWritten -NoEnumerate 6>&1 | Should be '1 2.2 System.Object[] abc'
Describing Write-Output
Context Input Tests
[+] Should allow piped input 335ms
[+] Should write output to the output stream when using piped input 23ms
[+] Should use inputobject switch 25ms
[+] Should write output to the output stream when using inputobject switch 38ms
[+] Should be able to write to a variable 21ms
Context Pipeline Command Tests
[+] Should send object to the next command in the pipeline 69ms
[+] Should have the same result between inputobject switch and piped input 23ms
Context Enumerate Objects
[+] Should see individual objects when not using the NoEnumerate switch 62ms
[-] Should be able to treat a collection as a single object using the NoEnumerate switch 32ms
Expected: {1}
But was: {3}
at line: 71 in C:\Users\dlwya_000\Documents\GitHub\powershell\test\powershell\Modules\Microsoft.PowerShell.Utility\Write-Output.Tests.ps1
71: $singleCollection | Should Be 1
Tests completed in 2.06s
Passed: 8 Failed: 3 Skipped: 0 Pending: 0
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Man, what a mess. Here's some of Write-Output's behavior today:
write-output -NoEnumerate a b c | % { $_.GetType().FullName; $_.Count; $_ -join '/' }
System.String
1
a
System.String
1
b
System.String
1
c
write-output -NoEnumerate a,b,c | % { $_.GetType().FullName; $_.Count; $_ -join '/' }
System.Object[]
3
a/b/c
This is a symptom of the flawed binding logic I changed (assuming the first call to BindParameter would fail when the a,b,c syntax was used), and the workaround that was created for it in Write-Output. Whatever the correct behavior is, there should never be a difference between these two calls when ValueFromRemainingArguments is in effect:
Do-Something a b c
Do-Something a,b,c
The values bound to the parameter should be identical in both cases.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's possible to fix the binding logic without somehow changing (breaking or fixing, depending on your point of view) the behavior of Write-Output. The whole point of the binding fix is to ensure consistent behavior between those two syntax options I listed, and it happens before the cmdlet ever gets to look at the values.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Write-Output a,b,c
This is a symptom of the flawed binding logic I changed (assuming the first call to BindParameter would fail when the a,b,c syntax was used)
in this case, the first call to BindParameter is successful, because the parameter type of InputObject is PSObject[], and type conversion from list<object[]> to it would be successful.
write-output -NoEnumerate a b c | % { $_.GetType().FullName; $_.Count; $_ -join '/' }
System.String
1
a
System.String
1
b
System.String
1
c
write-output -NoEnumerate a,b,c | % { $_.GetType().FullName; $_.Count; $_ -join '/' }
System.Object[]
3
a/b/c
IMHO, this behavior of Write-Output is pretty intuitive -- when -NoEnumerate is specified, give me back what ever items I gave you. In case of a b c, I gave you 3 items, so 3 items should be returned by Write-Output; in case of a,b,c, I gave you 1 item, which is an array, so the same item should be returned back.
However, it gets confusing when -InputObject is explicitly specified:
PS D:\> Write-Output -InputObject a,b,c -NoEnumerate | % { $_.GetType().FullName; $_.Count; $_ -join '/' }
System.String
1
a
System.String
1
b
System.String
1
c
PS D:\> Write-Output a,b,c -NoEnumerate | % { $_.GetType().FullName; $_.Count; $_ -join '/' }
System.Object[]
3
a/b/c
This is the part I don't like about ValueFromRemainingArgument, but changing it would definitely break something.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
In case of a b c, I gave you 3 items, so 3 items should be returned by Write-Output
in case of a,b,c, I gave you 1 item,
I disagree with that. ValueFromRemainingArguments is just syntax sugar, similar to a params parameter in C#. You gave 3 objects in both versions of the command.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
as shown by the discussion about Another proposal of the fix: When This won't solve the problem to parameters with Fixing parameter binder bugs is very tricky. It would be so easy to break existing behavior. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
The more I play with this, the more I think it's the existing behavior that's broken. I'd advise people not to declare |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Let me get more data about the usage of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Dave Wyatt (@dlwyatt) I'm not a good person to explain the parsing (maybe Bruce Payette (@BrucePay) can weigh in) but in this example, the parser views this as 1 parameter - an array with 3 values. Or in this example:
I don't see any value in breaking existing semantics but we could consider adding a new attribute [CollapseAllArgumentsIntoASingleArray](name TBD) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Jeffrey Snover (@jpsnover) : That's exactly what |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
We have discussed this and would like more data about potential breakage if we took the change. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
6becaf0 to
578bbfd
Compare
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
We can use '$TestDrive' instead of '$env:temp' and '/tmp'
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
There's already a comment explaining why that doesn't work.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I see. Please clarify - Pester will fail to clean up or Pester will fail all test batch?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Should we use explicit assembly?
Can we use:
Add-Type -TypeDefinition $a -PassThru | ForEach-Object {$_.assembly} | Import-module -Force
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
When you import the temporary module, the assembly is in use until PowerShell exits. That's why I don't create it in TestDrive:, and why I use the same file name every time (so we're not littering the test system with multiple dlls). |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Dave Wyatt (@dlwyatt) Thanks for clafify! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Just rebased. Will check back in a bit to make sure the tests are still passing. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Dave Wyatt (@dlwyatt) Thank you! This will be the last rebase for sure 😄 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Dongbo Wang (@daxian-dbw) Should we add Docs-Needed and remove Review-Needed? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| // Set-ClusterOwnerNode foo bar | ||
| // Set-ClusterOwnerNode foo,bar | ||
| // we unwrap our List, but only if there is a single argument of type object[]. | ||
| if (valueFromRemainingArguments.Count == 1 && valueFromRemainingArguments[0] is object[]) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Should it be valueFromRemainingArguments[0]?.GetType() == typeof(object[]) or valueFromRemainingArguments[0] is Array?
Given array covariance in .NET, any array of reference type (like string[]) will pass is object[] check, but not array of value type (like int[]). Should it be only object[] or all arrays instead?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
PetSerAl - good question, I think the intent was all arrays, though object[] in this context is nearly the same thing.
Dave Wyatt (@dlwyatt) , Dongbo Wang (@daxian-dbw) - thoughts?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Looks like you're right. Most of the time, PowerShell's building object[] arrays from arguments at that point, but if you explicitly make a variable of another type and pass it in, it doesn't get unwrapped.
Rather than is Array, though, I'd recommend using LanguagePrimitives.GetEnumerable(valueFromRemainingArguments[0]) . That will cover other types of collections using the same rules that you'd expect from PowerShell (not treating strings / dictionaries as collections, etc.)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I'm putting together new tests for this and will follow up with a second PR
Sorry, something went wrong.
There was an error while loading. Please reload this page.
when -NoEnumerate is used. Fix PowerShell#5122
Fix is to preserve input collection type in output. The regression was caused by #2038
Fix is to preserve input collection type in output. The regression was caused by #2038
Fix is to preserve input collection type in output. The regression was caused by PowerShell#2038
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.