GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — crypto: fix default encoding of LazyTransform by lmoe · Pull Request #8611 · nodejs/node

Conversation

@lmoe

@lmoe lmoe commented Sep 17, 2016 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Checklist
  • make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines
Affected core subsystem(s)

crypto

Description of change

Change the default encoding from latin1 to utf8
and extend the test-crypto tests.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
zvictor
zvictor reviewed Sep 17, 2016
View reviewed changes
Comment thread test/parallel/test-crypto.js Outdated

@zvictor zvictor Sep 17, 2016

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superfluous line removal

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@indutny indutny Oct 8, 2016

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@zvictor zvictor mentioned this pull request Sep 17, 2016
crypto: fix default encoding of LazyTransform [2] #8624
Closed
3 tasks done
addaleax
addaleax requested changes Sep 17, 2016
View reviewed changes

@addaleax addaleax left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good so far, thanks for doing this!

/cc @indutny @mscdex ?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread test/parallel/test-crypto.js Outdated

@addaleax addaleax Sep 17, 2016

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use assert.strictEqual instead of assert.equal here, and maybe wrap the end listener in common.mustCall()?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread test/parallel/test-crypto.js Outdated

@addaleax addaleax Sep 17, 2016 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no umlauts in ASCII, but the hash below is valid for latin-1 (aka ISO-8859-1), so I’d recommend naming everything as latin1 here:

$ echo -n öäü | iconv -f utf8 -t iso-8859-1 | sha256sum 
cd37bccd5786e2e76d9b18c871e919e6eb11cc12d868f5ae41c40ccff8e44830  -

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@lmoe lmoe Sep 17, 2016

Copy link
Copy Markdown
Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have trouble figuring out your message. What I would do is renaming the ascii part to latin1 but I'm not sure if that's your point. You have marked the Utf8 part and the comment of the ascii format which kind of makes it hard to understand.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Sep 17, 2016

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmoe yeah, I think you got what I was trying to say. That you see the Utf8 part too is just github’s UI, usually the PR comments just apply to single lines (so, yeah, I’m talking about the part which is currently labelled ascii)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@addaleax addaleax added crypto Issues and PRs related to the crypto subsystem. semver-major PRs that contain breaking changes and should be released in the next major version. labels Sep 17, 2016
@addaleax

addaleax commented Sep 17, 2016

Copy link
Copy Markdown
Member

Also, I’ve tentatively labelled this as semver-major. It aligns the behaviour with our docs, so whether this is actually semver-major is up for debate.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fishrock123

Fishrock123 commented Sep 17, 2016

Copy link
Copy Markdown
Contributor

fix default encoding of LazyTransform

I'm not clear on what this fixes? Is there an issue? If not could we please get a description in the commit message? :D

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax

addaleax commented Sep 17, 2016

Copy link
Copy Markdown
Member

I'm not clear on what this fixes? Is there an issue? If not could we please get a description in the commit message? :D

Not really, I just noticed this while going through or code base. There’s #5522 and #5500, which changed the default encoding for strings that are passed to crypto functions to utf8, but only did so for the non-streaming API… that makes .update() and .write() behave weirdly incongruently. I assumed that to be an oversight, and if I understood @indutny correctly today, he agrees.

A bit on that would be cool for the commit message, yes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lmoe
lmoe force-pushed the fix-issue-7712 branch from 34c2b15 to 0669e2c Compare September 19, 2016 07:41
addaleax
addaleax approved these changes Sep 20, 2016
View reviewed changes

@addaleax addaleax left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@Trott
Trott force-pushed the master branch from b0df363 to c5ce7f4 Compare September 21, 2016 00:10
jasnell
jasnell reviewed Sep 28, 2016
View reviewed changes

@jasnell jasnell left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with green CI.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@addaleax

addaleax commented Sep 30, 2016

Copy link
Copy Markdown
Member

CI: https://ci.nodejs.org/job/node-test-commit/5379/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jasnell

jasnell commented Oct 6, 2016

Copy link
Copy Markdown
Member

@nodejs/crypto ... any objections on this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

indutny
indutny suggested changes Oct 8, 2016
View reviewed changes

@indutny indutny left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One style nit, otherwise LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@rvagg
rvagg force-pushed the master branch 2 times, most recently from c133999 to 83c7a88 Compare October 18, 2016 17:02
indutny
indutny approved these changes Oct 18, 2016
View reviewed changes

@indutny indutny left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, if CI is green.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@lmoe

lmoe commented Jan 5, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

@indutny @addaleax
What should I do about this merge conflict? Just to know it in the future. It seems it occurred a month later or so. Should I resolve this conflict on my own, or will your merge tools handle it on their own when you make a release?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sam-github

sam-github commented Jan 9, 2017

Copy link
Copy Markdown
Contributor

@lmoe Please resolve and re-push, it distributes the maintenance work, and if there are any complexities, we may ask you to do so anyhow.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax addaleax added this to the 8.0.0 milestone Feb 3, 2017
@addaleax

addaleax commented Feb 3, 2017

Copy link
Copy Markdown
Member

bump @lmoe – Could you rebase this against master to resolve the conflict?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax

addaleax commented Feb 15, 2017

Copy link
Copy Markdown
Member

Ping @lmoe again? Otherwise I’ll pick this up in the next couple of days.

jasnell and lmoe reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Lukas Möller added 2 commits February 16, 2017 10:00
Lukas Möller
crypto: fix default encoding of LazyTransform
74b56cd
Change the default encoding from latin1 to utf8
and extend the test-crypto tests.
Lukas Möller
crypto: fix default encoding of LazyTransform
fe9de5d
PullRequest nodejs#5522 and nodejs#5500 described the change
of the default encoding into UTF8 in crypto functions.

This however was only changed for the non-streaming API.
The streaming API still used binary as the default encoding.

This commit will change the default streaming API encoding to UTF8
to make both APIs behave the same.

It will also add tests to validate the behavior.
@lmoe
lmoe force-pushed the fix-issue-7712 branch from 7ad8f75 to fe9de5d Compare February 16, 2017 09:09
@lmoe

lmoe commented Feb 16, 2017

Copy link
Copy Markdown
Author

@addaleax Sorry, I totally forgot about the mail. I hope the branch is corrected now.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

addaleax
addaleax approved these changes Feb 16, 2017
View reviewed changes

@addaleax addaleax left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, still looks good! Going to land this soon.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@addaleax

addaleax commented Feb 16, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

CI: https://ci.nodejs.org/job/node-test-commit/7947/
CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/580/

Edit: The electron-prebuilt failure is real and related, I’m reaching out and fixing it.
Edit²: malept/sumchecker#5 …

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

addaleax added a commit to addaleax/sumchecker that referenced this pull request Feb 16, 2017
@addaleax
Explicitly set default encoding in createHash()
2a395e2
Up until now, the default encoding for this is `binary`, but
this is scheduled to change in Node 8. Since this module is
currently always passing string input to the `Hash` object,
it needs to account for that.

A better fix would likely involve dropping all strings handling
and treating binary data using Buffer objects, but I kept
this change minimal to avoid any breakage.

Refs: nodejs/node#8611
@addaleax addaleax mentioned this pull request Feb 16, 2017
Explicitly set default encoding in createHash() malept/sumchecker#5
Merged
addaleax added a commit to addaleax/sumchecker that referenced this pull request Feb 21, 2017
@addaleax
Explicitly set default encoding in createHash()
1b94a51
Up until now, the default encoding for this is `binary`, but
this is scheduled to change in Node 8. Since this module is
currently always passing string input to the `Hash` object,
it needs to account for that.

A better fix would likely involve dropping all strings handling
and treating binary data using Buffer objects, but I kept
this change minimal to avoid any breakage.

Refs: nodejs/node#8611
@addaleax

addaleax commented Mar 10, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Fresh CI:

CI: https://ci.nodejs.org/job/node-test-commit/8359/
CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/637/
Edit: New CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/638/ (should be a bit better since the electron-prebuilt failure should be fixed now)

I would like to land this unless any new surprises show up. The electron-prebuilt failure is just waiting on a dependency update, and we should definitely have this in Node 8.

(why? because:)
'use strict';
const { createHash } = require('crypto');

const hash1 = createHash('sha256');
hash1.on('data', out => console.log('hash1 => ', out.toString('hex')));
hash1.end('💩');

const hash2 = createHash('sha256');
hash2.on('data', out => console.log('hash2 => ', out.toString('hex')));
hash2.end('⨽⪩');
// Surprise! Different input, same hash.
lmoe reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jasnell
jasnell approved these changes Mar 10, 2017
View reviewed changes
@addaleax addaleax added the notable-change PRs with changes that should be highlighted in changelogs. label Mar 11, 2017
@addaleax

addaleax commented Mar 11, 2017

Copy link
Copy Markdown
Member

CITGM failures seem to be known/unrelated. 🎉

Landed in 443691a, thanks for the PR and for bearing with our process!

lmoe reacted with hooray emoji lmoe reacted with heart emoji
All reactions
  • 🎉 1 reaction
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax addaleax closed this Mar 11, 2017
addaleax pushed a commit that referenced this pull request Mar 11, 2017
Lukas Möller @addaleax
crypto: fix default encoding of LazyTransform
443691a
PullRequest #5522 and #5500 described the change
of the default encoding into UTF8 in crypto functions.

This however was only changed for the non-streaming API.
The streaming API still used binary as the default encoding.

This commit will change the default streaming API encoding to UTF8
to make both APIs behave the same.

It will also add tests to validate the behavior.

Refs: #5522
Refs: #5500
PR-URL: #8611
Reviewed-By: Fedor Indutny <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
jungx098 pushed a commit to jungx098/node that referenced this pull request Mar 21, 2017
Lukas Möller @jungx098
crypto: fix default encoding of LazyTransform
98659fd
PullRequest nodejs#5522 and nodejs#5500 described the change
of the default encoding into UTF8 in crypto functions.

This however was only changed for the non-streaming API.
The streaming API still used binary as the default encoding.

This commit will change the default streaming API encoding to UTF8
to make both APIs behave the same.

It will also add tests to validate the behavior.

Refs: nodejs#5522
Refs: nodejs#5500
PR-URL: nodejs#8611
Reviewed-By: Fedor Indutny <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
@jasnell jasnell mentioned this pull request Apr 4, 2017
8.0.0 Release Proposal #12220
Closed
@tniessen tniessen mentioned this pull request Aug 13, 2023
crypto: do not overwrite _writableState.defaultEncoding #49140
Merged
tniessen added a commit to tniessen/node that referenced this pull request Aug 20, 2023
@tniessen
crypto: do not overwrite _writableState.defaultEncoding
1692e52
This only affects the writable side of LazyTransform and should not
change the behavior of any LazyTransform streams (Cipher, Decipher,
Cipheriv, Decipheriv, Hash, Hmac).

If the user does not set defaultEncoding when creating a transform
stream, WritableState uses 'utf8' by default. Only LazyTransform
overwrites this with 'buffer' for strict backward compatibility. This
was necessary when crypto.DEFAULT_ENCODING still existed. Now that
DEFAULT_ENCODING has been removed, defaultEncoding is always 'buffer'.
The writable side of LazyTransform appears to treat 'utf8' and 'buffer'
in exactly the same way. Therefore, there seems to be no need to
overwrite _writableState.defaultEncoding at this point.

Nevertheless, because Node.js has failed to hide implementation details
such as _writableState from the ecosystem, we may want to consider this
a breaking change.

Refs: nodejs#47182
Refs: nodejs#8611
nodejs-github-bot pushed a commit that referenced this pull request Aug 27, 2023
@tniessen
crypto: do not overwrite _writableState.defaultEncoding
5a07777
This only affects the writable side of LazyTransform and should not
change the behavior of any LazyTransform streams (Cipher, Decipher,
Cipheriv, Decipheriv, Hash, Hmac).

If the user does not set defaultEncoding when creating a transform
stream, WritableState uses 'utf8' by default. Only LazyTransform
overwrites this with 'buffer' for strict backward compatibility. This
was necessary when crypto.DEFAULT_ENCODING still existed. Now that
DEFAULT_ENCODING has been removed, defaultEncoding is always 'buffer'.
The writable side of LazyTransform appears to treat 'utf8' and 'buffer'
in exactly the same way. Therefore, there seems to be no need to
overwrite _writableState.defaultEncoding at this point.

Nevertheless, because Node.js has failed to hide implementation details
such as _writableState from the ecosystem, we may want to consider this
a breaking change.

Refs: #47182
Refs: #8611
PR-URL: #49140
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell approved these changes
@addaleax addaleax addaleax approved these changes
+2 more reviewers
@zvictor zvictor zvictor left review comments
@indutny indutny indutny approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

crypto Issues and PRs related to the crypto subsystem. notable-change PRs with changes that should be highlighted in changelogs. semver-major PRs that contain breaking changes and should be released in the next major version.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@lmoe @addaleax @Fishrock123 @jasnell @sam-github @zvictor @indutny
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.