GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Change New-ModuleManifest encoding to UTF8NoBOM on non-Windows platforms by anmenaga · Pull Request #3940 · PowerShell/PowerShell

Conversation

@anmenaga

@anmenaga Andrew (anmenaga) commented Jun 5, 2017

Copy link
Copy Markdown

This addresses #3789
Currently New-ModuleManifest creates psd1 manifests in UTF-16 with BOM; this creates problems for Linux tools and Git diff'er.
This fix changes encoding of New-ModuleManifest to be UTF8 (no BOM).
Also mentioned New-ModuleManifest in "RFC 0020 DefaultFileEncoding".

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

rkeithhill reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Andrew
Change New-ModuleManifest encoding to UTF8NoBOM
cd06655
@anmenaga
Andrew (anmenaga) requested a review from James Truher (JamesWTruher) June 5, 2017 20:25
@msftclas Microsoft Contribution License Agreements (msftclas) added the cla-not-required label Jun 5, 2017
Francisco-Gamino
Francisco Gamino (Francisco-Gamino) reviewed Jun 5, 2017
View reviewed changes
Comment thread src/System.Management.Automation/engine/Modules/NewModuleManifestCommand.cs
this,
filePath,
EncodingConversion.Unicode,
new UTF8Encoding(false), // UTF-8, no BOM

@Francisco-Gamino Francisco Gamino (Francisco-Gamino) Jun 5, 2017

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update the documentation for this change?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@SteveL-MSFT Steve Lee (SteveL-MSFT) Jun 5, 2017

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Andrew (@anmenaga) says he opened a doc issue, can you link it here?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@anmenaga Andrew (anmenaga) Jun 6, 2017

Copy link
Copy Markdown
Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR in RFC repo.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@joeyaiello Joey Aiello (joeyaiello) added the Breaking-Change breaking change that may affect users label Jun 5, 2017
@joeyaiello

Joey Aiello (joeyaiello) commented Jun 5, 2017

Copy link
Copy Markdown
Contributor

As it currently stands, this is a breaking change (appropriate label added).

Based on our current POR, we're okay changing the default on non-Windows platforms, but this shouldn't be changed (yet) on Windows until we've had the breaking change conversation. /cc James Truher (@JamesWTruher)

iSazonov reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joeyaiello

Joey Aiello (joeyaiello) commented Jun 5, 2017

Copy link
Copy Markdown
Contributor

I understand that this currently breaks the git scenario, we just have to be thoughtful about this.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vors

sergei (vors) commented Jun 6, 2017

Copy link
Copy Markdown
Collaborator

Are there plans to add UTF8-NoBom to powershell supported Encodings?
I.e. for Set-Content and friends.

Changing the most important scenarios is good, but it would be great to address the problem with encodings broader and consistently to help shape the conversation about breaking changes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Jun 6, 2017

Copy link
Copy Markdown
Collaborator

sergei (@vors) UTF8noBOM is discussed in RFC 0020 DefaultFileEncoding.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Jun 6, 2017

Copy link
Copy Markdown
Collaborator

New-PSSessionConfigurationFile uses EncodingConversion.Unicode too.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joeyaiello

Joey Aiello (joeyaiello) commented Jun 6, 2017

Copy link
Copy Markdown
Contributor

sergei (@vors): first, yes. We're adding that (and making it the default across the board on Linux). The hard part is what to do with Windows (change the defaults? leave a legacy enum value for the "traditional" disparate defaults?) Which is why I want to have the discussion.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Andrew
Addressed PR feedback
85cde4c
@anmenaga

Andrew (anmenaga) commented Jun 6, 2017

Copy link
Copy Markdown
Author

Joey Aiello (@joeyaiello) good point;
PR updated to change encoding only for non-Windows.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@anmenaga Andrew (anmenaga) changed the title Change New-ModuleManifest encoding to UTF8NoBOM Change New-ModuleManifest encoding to UTF8NoBOM on non-Windows platforms Jun 6, 2017
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Jun 6, 2017

Copy link
Copy Markdown
Member

Is there an actual problem on Windows using UTF8NoBOM? If only English is used, it's equivalent to an ascii file and Windows tools handle that just fine and I expect that to be the majority of use.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Andrew
Fixed test for Travis CI
880f9b5
@TravisEz13

Travis Plunk (TravisEz13) commented Jun 6, 2017

Copy link
Copy Markdown
Member

Restarted AppVeyor due to MyGet connectivity issue

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joeyaiello

Joey Aiello (joeyaiello) commented Jun 6, 2017

Copy link
Copy Markdown
Contributor

Andrew (@anmenaga) if you could sync with James Truher (@JamesWTruher) before merging this, that'd be awesome. Assuming the @PowerShell/powershell-committee votes to accept tomorrow, we want to move to UTF8NoBOM everywhere.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Jun 7, 2017

Copy link
Copy Markdown
Collaborator

Steve Lee (@SteveL-MSFT)

Is there an actual problem on Windows using UTF8NoBOM?

Users can add non-English comments in the manifest.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Jun 7, 2017

Copy link
Copy Markdown
Member

Ilya (@iSazonov) absolutely, but I believe the majority of usage is English. Even with non-ascii characters, my understanding is that notepad has heuristics to determine encoding to display it correctly. VSCode seems to support UTF8NoBom just fine.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Jun 7, 2017

Copy link
Copy Markdown
Collaborator

notepad has heuristics to determine encoding to display it correctly

It seem works on last versions only not on Windows 7/8(?).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mirichmo Mike Richmond (mirichmo) self-assigned this Jun 7, 2017
@TravisEz13

Travis Plunk (TravisEz13) commented Jun 9, 2017

Copy link
Copy Markdown
Member

Joey Aiello (@joeyaiello) Are you saying this is blocked on a sign off from James Truher (@JamesWTruher) ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@TravisEz13

Travis Plunk (TravisEz13) commented Jun 9, 2017

Copy link
Copy Markdown
Member

Joey Aiello (@joeyaiello) James Truher (@JamesWTruher) Do we want to document the output format of Cmdlets?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joeyaiello

Joey Aiello (joeyaiello) commented Jun 13, 2017

Copy link
Copy Markdown
Contributor

Travis Plunk (@TravisEz13) yes and yes. I think the uber-point is that James Truher (@JamesWTruher) is doing so much work here, that it might get changed anyway (and it might just muddle his extensive notes).

Ultimately up to him, though. I just wanted to make sure everyone was on the same page.

Regarding docs though: heck yes. We'll have a Legacy value for Encoding, and that will be well-documented.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JamesWTruher
James Truher (JamesWTruher) approved these changes Jun 13, 2017
View reviewed changes

@JamesWTruher James Truher (JamesWTruher) left a comment

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine for now, I'll be changing all the cmdlets which provide for encoding as part of https://github.com/PowerShell/PowerShell-RFC/blob/master/1-Draft/RFC0020-DefaultFileEncoding.md. This doesn't muddy the waters for me.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@TravisEz13 Travis Plunk (TravisEz13) added the Documentation Needed in this repo Documentation is needed in this repo label Jun 13, 2017
TravisEz13
Travis Plunk (TravisEz13) reviewed Jun 13, 2017
View reviewed changes
Comment thread test/powershell/engine/Module/NewModuleManifest.Tests.ps1 Outdated
(Get-Content -Encoding Byte -Path $testModulePath -TotalCount $expected.Length) -join ',' | Should Be ($expected -join ',')
}

It "Verify module manifest encoding on Windows " -Skip:(-not $IsWindows) {

@TravisEz13 Travis Plunk (TravisEz13) Jun 13, 2017

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you just calculate the expected value in a beforeall then have one test for all platforms?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@anmenaga Andrew (anmenaga) Jun 16, 2017

Copy link
Copy Markdown
Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea; updated.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
TravisEz13
Travis Plunk (TravisEz13) reviewed Jun 13, 2017
View reviewed changes

@TravisEz13 Travis Plunk (TravisEz13) left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please respond to my comment on your code.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@TravisEz13

Travis Plunk (TravisEz13) commented Jun 13, 2017

Copy link
Copy Markdown
Member

Andrew (@anmenaga) Also, as a PowerShell team member, you need to submit the PR to update the docs before I merge this. Please add a link to the PR here. Thanks.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Jun 14, 2017

Copy link
Copy Markdown
Collaborator

For encoding changes, you might want to delay documenting as we expect big changes in this.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Andrew
Updated tests
dc92b32
@anmenaga

Andrew (anmenaga) commented Jun 16, 2017

Copy link
Copy Markdown
Author

Travis Plunk (@TravisEz13)

  1. This PR in RFC repo was merged;
  2. I've ran a search in PowerShell-Docs and didn't find any mentioning of encoding for New-ModuleManifest, so there is nothing to update there. Besides, this PR will be anyway overwritten by Jim's encoding work that is arriving soon.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

SteveL-MSFT
Steve Lee (SteveL-MSFT) previously requested changes Jun 19, 2017
View reviewed changes

@SteveL-MSFT Steve Lee (SteveL-MSFT) left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should update https://github.com/PowerShell/PowerShell/blob/master/docs/FAQ.md with the recommended settings in VSCode for automatic removing of trailing whitespace for resx files

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@SteveL-MSFT
Steve Lee (SteveL-MSFT) dismissed their stale review June 19, 2017 15:58

wrong PR

@TravisEz13

Travis Plunk (TravisEz13) commented Jun 20, 2017

Copy link
Copy Markdown
Member

Andrew (@anmenaga) Per the conversation in this PR, we want to add encoding documentation to the CmdLet docs.

joeyaiello reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mirichmo Mike Richmond (mirichmo) assigned Travis Plunk (TravisEz13) and unassigned Mike Richmond (mirichmo) Jun 20, 2017
@anmenaga

Andrew (anmenaga) commented Jun 26, 2017

Copy link
Copy Markdown
Author

Travis Plunk (@TravisEz13) Here is the doc PR: MicrosoftDocs/PowerShell-Docs#1405

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kilasuit

Ryan Yates (kilasuit) commented Jun 27, 2017

Copy link
Copy Markdown
Collaborator

My only issue here is that although this is a change that I agree with, I think that this is where I have concerns of this causing a disparate and inconsistent behaviour between all versions of PowerShell below v6 due to the fact that New-ModuleManifest is baked into the Microsoft.PowerShell.Core module.

I personally would much prefer to see that this update to the cmdlet gets released in a manner that fully supports being used on downlevel systems where PowerShell v6 is unable to be installed.

However the only real solutions that I can really see available to be able to make that to happen would be to

  • Investigate the decoupling of a number of the cmdlets from the Microsoft.PowerShell.Core module into their own separate individual modules or a condensed updated singular module that has all the updates to any module in Microsoft.PowerShell.Core (not an overly fantastic solution) - similar in the idea put forward in Decouple the bundled modules to seperate Repo's #1979
  • Look to recreate this cmdlet within the PowerShellGet module.
  • Alongside the v6 updates look to release a completely updated and downlevel compatible version of the Microsoft.PowerShell.Core module to the PowerShell Gallery, which isn't really an overly viable solution either.

If this change could be easily replicated for downlevel versions of PowerShell I think that would be the best outcome all around.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@TravisEz13

Travis Plunk (TravisEz13) commented Jun 27, 2017

Copy link
Copy Markdown
Member

Ryan Yates (@kilasuit) Thanks for the feedback. I think most of the concerns raised would be best tracked in #1979 or a new issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@TravisEz13
Travis Plunk (TravisEz13) merged commit e2e07b1 into PowerShell:master Jun 27, 2017
@joeyaiello Joey Aiello (joeyaiello) mentioned this pull request Oct 15, 2018
Docs needed for 'Change New-ModuleManifest encoding to UTF8NoBOM on non-Windows platforms' MicrosoftDocs/PowerShell-Docs#3098
Closed
@joeyaiello Joey Aiello (joeyaiello) removed Documentation Needed in this repo Documentation is needed in this repo labels Oct 15, 2018
@anmenaga
Andrew (anmenaga) deleted the NewModuleManifest branch October 31, 2018 21:20
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
Andrew @TravisEz13
Change New-ModuleManifest encoding to UTF8NoBOM on non-Windows platfo…
cfd2dff
…rms (PowerShell#3940)

* Change New-ModuleManifest encoding to UTF8NoBOM

* Addressed PR feedback

* Fixed test for Travis CI

* Updated tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TravisEz13 Travis Plunk (TravisEz13) TravisEz13 left review comments
@SteveL-MSFT Steve Lee (SteveL-MSFT) SteveL-MSFT left review comments
+2 more reviewers
@Francisco-Gamino Francisco Gamino (Francisco-Gamino) Francisco-Gamino left review comments
@JamesWTruher James Truher (JamesWTruher) JamesWTruher approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@TravisEz13 Travis Plunk (TravisEz13)

Labels

Breaking-Change breaking change that may affect users

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

11 participants

@anmenaga @joeyaiello @vors @iSazonov @SteveL-MSFT @TravisEz13 @kilasuit @JamesWTruher @Francisco-Gamino @msftclas @mirichmo
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.