GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — zlib: decompression throw on truncated input by jhamhader · Pull Request #2595 · nodejs/node

Conversation

@jhamhader

@jhamhader jhamhader commented Aug 28, 2015

Copy link
Copy Markdown
Contributor

Check for unexpected end-of-stream error when decompressing.
If the output buffer still has space after decompressing and deflate
returned Z_OK or Z_BUF_ERROR - that means unexpected end-of-stream.
Added test-zlib-truncated.js for the case of truncated input.
Fixed the zlib dictionary test to not end the inflate stream on a
truncated output (no crc) of deflate
See issue #2043

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@targos targos added the zlib Issues and PRs related to the zlib module and its compression dependencies. label Aug 28, 2015
@Fishrock123

Fishrock123 commented Aug 28, 2015

Copy link
Copy Markdown
Contributor

CI run: https://jenkins-iojs.nodesource.com/job/node-test-pull-request/200/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fishrock123
Fishrock123 reviewed Aug 28, 2015
View reviewed changes
Comment thread test/parallel/test-zlib-truncated.js Outdated

@Fishrock123 Fishrock123 Aug 28, 2015

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use const for values that won't be changed that you either added or modified in this commit. :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@jhamhader jhamhader Aug 28, 2015

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the input :)
Should I use 'let' for the scope-specific variables or is 'var' ok?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@jhamhader

jhamhader commented Aug 28, 2015

Copy link
Copy Markdown
Contributor Author

Committed 'let' and 'const' for 'test-zlib-dictionary.js' and 'test-zlib-truncated.js'

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rvagg
rvagg force-pushed the master branch from 11c25c2 to ba02bd0 Compare September 6, 2015 11:55
@jhamhader
jhamhader force-pushed the zlib-truncated branch 2 times, most recently from 30b47df to 495039d Compare September 19, 2015 12:16
@indutny

indutny commented Sep 24, 2015

Copy link
Copy Markdown
Member

cc @nodejs/collaborators

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@trevnorris

trevnorris commented Sep 24, 2015

Copy link
Copy Markdown
Contributor

Is this done elsewhere (as in, in other programs), and/or is it what the user would expect to happen?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader

jhamhader commented Sep 27, 2015

Copy link
Copy Markdown
Contributor Author
  • gnu gunzip: fails with unexpected end of file
  • python2 gzip: fails with IOError: CRC check failed 0xca600aac != 0x6fe58c67L
  • python3 gzip: fails with EOFError: Compressed file ended before the end-of-stream marker was reached
  • ruby gzip: fails with Zlib::GzipFile::Error: unexpected end of file
  • perl gzip: fails with unexpected end of file at (eval 15)[/usr/share/perl5/core_perl/perl5db.pl:737] line 2.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader
jhamhader force-pushed the zlib-truncated branch from 495039d to 5c796aa Compare September 27, 2015 16:41
@trevnorris

trevnorris commented Sep 27, 2015

Copy link
Copy Markdown
Contributor

Seems at the minimum we should be propagation an error. So will it throw for sync and emit error on async? (not familiar w/ much of the zlib api)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader

jhamhader commented Sep 27, 2015

Copy link
Copy Markdown
Contributor Author

Throw for sync.
What do you mean by emit? The callback of the async function will be called with err on the first argument.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader
zlib: decompression throw on truncated input
aae310e
Check for unexpected end-of-file error when decompressing.
If the output buffer still has space after decompressing and deflate
returned Z_OK or Z_BUF_ERROR - that means unexpected end-of-file.
Added test-zlib-truncated.js for the case of truncated input.
Fixed the zlib dictionary test to not end the inflate stream on a
truncated output (no crc) of deflate
@jhamhader
jhamhader force-pushed the zlib-truncated branch from 5c796aa to aae310e Compare September 28, 2015 08:21
@trevnorris

trevnorris commented Sep 28, 2015

Copy link
Copy Markdown
Contributor

The callback of the async function will be called with err on the first argument.

Nm me. Was thinking about using zlib w/ the streams API and how a pipe would propagate the error.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader

jhamhader commented Sep 28, 2015

Copy link
Copy Markdown
Contributor Author

Sure :)
Anything else missing for the PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@trevnorris

trevnorris commented Sep 28, 2015

Copy link
Copy Markdown
Contributor

Not from me. Would like @indutny to sign off on this as well.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@thefourtheye

thefourtheye commented Sep 29, 2015

Copy link
Copy Markdown
Contributor

I would prefer @chrisdickinson to sign off as well.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader

jhamhader commented Oct 8, 2015

Copy link
Copy Markdown
Contributor Author

@indutny @chrisdickinson could you please take a look and confirm?
Thanks :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@trevnorris

trevnorris commented Oct 8, 2015

Copy link
Copy Markdown
Contributor

@jhamhader ping me tomorrow if @indutny hasn't responded.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jhamhader

jhamhader commented Oct 10, 2015

Copy link
Copy Markdown
Contributor Author

Pinging

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@trevnorris

trevnorris commented Oct 10, 2015

Copy link
Copy Markdown
Contributor

Thanks. Let's go ahead and land this. I'll be back at my computer on Monday.

Also, I'm initially inclined to say this is semver-major. Even if it's not, don't know if this is something LTS would accept. We can check after it's in master. :-)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fishrock123 Fishrock123 added the semver-major PRs that contain breaking changes and should be released in the next major version. label Oct 10, 2015
@Fishrock123 Fishrock123 added this to the 5.0.0 milestone Oct 10, 2015
@rvagg rvagg mentioned this pull request Oct 16, 2015
v5.0.0 Planning & Checklist #3397
Closed
13 tasks done
@Fishrock123

Fishrock123 commented Oct 19, 2015

Copy link
Copy Markdown
Contributor

ping @trevnorris

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@trevnorris

trevnorris commented Oct 19, 2015

Copy link
Copy Markdown
Contributor

Landed on 0fc0902. Thanks much!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@trevnorris trevnorris closed this Oct 19, 2015
trevnorris pushed a commit that referenced this pull request Oct 19, 2015
@jhamhader @trevnorris
zlib: decompression throw on truncated input
0fc0902
Check for unexpected end-of-file error when decompressing. If the output
buffer still has space after decompressing and deflate returned Z_OK or
Z_BUF_ERROR - that means unexpected end-of-file. Added
test-zlib-truncated.js for the case of truncated input. Fixed the zlib
dictionary test to not end the inflate stream on a truncated output (no
crc) of deflate

Fixes: #2043
PR-URL: #2595
Reviewed-By: Trevor Norris <[email protected]>
rvagg pushed a commit that referenced this pull request Oct 21, 2015
@jhamhader @rvagg
zlib: decompression throw on truncated input
80169b1
Check for unexpected end-of-file error when decompressing. If the output
buffer still has space after decompressing and deflate returned Z_OK or
Z_BUF_ERROR - that means unexpected end-of-file. Added
test-zlib-truncated.js for the case of truncated input. Fixed the zlib
dictionary test to not end the inflate stream on a truncated output (no
crc) of deflate

Fixes: #2043
PR-URL: #2595
Reviewed-By: Trevor Norris <[email protected]>
@rvagg rvagg mentioned this pull request Oct 21, 2015
Release proposal: v5.0.0 #3466
Merged
rvagg added a commit to rvagg/io.js that referenced this pull request Oct 29, 2015
@rvagg
2015-10-29, Version 5.0.0 (Stable)
384e6c2
Notable changes:

* buffer: (Breaking) Removed both 'raw' and 'raws' encoding types from Buffer,
  these have been deprecated for a long time (Sakthipriyan Vairamani) nodejs#2859.
* console: (Breaking) Values reported by console.time() now have 3 decimals of
  accuracy added (Michaël Zasso) nodejs#3166.
* fs:
  - fs.readFile*(), fs.writeFile*(), and fs.appendFile*() now also accept a file
    descriptor as their first argument (Johannes Wüller) nodejs#3163.
  - (Breaking) In fs.readFile(), if an encoding is specified and the internal
    toString() fails the error is no longer thrown but is passed to the callback
    (Evan Lucas) nodejs#3485.
  - (Breaking) In fs.read() (using the fs.read(fd, length, position, encoding,
    callback) form), if the internal toString() fails the error is no longer
    thrown but is passed to the callback (Evan Lucas) nodejs#3503.
* http:
  - Fixed a bug where pipelined http requests would stall (Fedor Indutny) nodejs#3342.
  - (Breaking) When parsing HTTP, don't add duplicates of the following headers:
    Retry-After, ETag, Last-Modified, Server, Age, Expires. This is in addition
    to the following headers which already block duplicates: Content-Type,
    Content-Length, User-Agent, Referer, Host, Authorization,
    Proxy-Authorization, If-Modified-Since, If-Unmodified-Since, From, Location,
    Max-Forwards (James M Snell) nodejs#3090.
  - (Breaking) The callback argument to OutgoingMessage#setTimeout() must be a
    function or a TypeError is thrown (James M Snell) nodejs#3090.
  - (Breaking) HTTP methods and header names must now conform to the RFC 2616
    "token" rule, a list of allowed characters that excludes control characters
    and a number of separator characters. Specifically, methods and header names
    must now match /^[a-zA-Z0-9_!#$%&'*+.^`|~-]+$/ or a TypeError will be thrown
    (James M Snell) nodejs#2526.
* node:
  - (Breaking) Deprecated the _linklist module (Rich Trott) nodejs#3078.
  - (Breaking) Removed require.paths and require.registerExtension(), both had
    been previously set to throw Error when accessed
    (Sakthipriyan Vairamani) nodejs#2922.
* npm: Upgraded to version 3.3.6 from 2.14.7, see
  https://github.com/npm/npm/releases/tag/v3.3.6 for more details. This is a
  major version bump for npm and it has seen a significant amount of change.
  Please see the original npm v3.0.0 release notes for a list of major changes
  (Rebecca Turner) nodejs#3310.
* src: (Breaking) Bumped NODE_MODULE_VERSION to 47 from 46, this is necessary
  due to the V8 upgrade. Native add-ons will need to be recompiled
  (Rod Vagg) nodejs#3400.
* timers: Attempt to reuse the timer handle for setTimeout().unref(). This fixes
  a long-standing known issue where unrefed timers would perviously hold
  beforeExit open (Fedor Indutny) nodejs#3407.
* tls:
  - Added ALPN Support (Shigeki Ohtsu) nodejs#2564.
  - TLS options can now be passed in an object to createSecurePair()
    (Коренберг Марк) nodejs#2441.
  - (Breaking) The default minimum DH key size for tls.connect() is now 1024
    bits and a warning is shown when DH key size is less than 2048 bits. This a security consideration to prevent "logjam" attacks. A new minDHSize TLS
    option can be used to override the default. (Shigeki Ohtsu) nodejs#1831.
* util:
  - (Breaking) util.p() was deprecated for years, and has now been removed
    (Wyatt Preul) nodejs#3432.
  - (Breaking) util.inherits() can now work with ES6 classes. This is considered
    a breaking change because of potential subtle side-effects caused by a
    change from directly reassigning the prototype of the constructor using
    `ctor.prototype = Object.create(superCtor.prototype, { constructor: { ... } })`
    to using `Object.setPrototypeOf(ctor.prototype, superCtor.prototype)`
    (Michaël Zasso) nodejs#3455.
* v8: (Breaking) Upgraded to 4.6.85.25 from 4.5.103.35 (Ali Ijaz Sheikh) nodejs#3351.
  - Implements the spread operator, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
    for further information.
  - Implements new.target, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target
    for further information.
* zlib: Decompression now throws on truncated input (e.g. unexpected end of
  file) (Yuval Brik) nodejs#2595.

PR-URL: nodejs#3466
rvagg added a commit that referenced this pull request Oct 29, 2015
@rvagg
2015-10-29, Version 5.0.0 (Stable)
837b174
Notable changes:

* buffer: (Breaking) Removed both 'raw' and 'raws' encoding types from Buffer,
  these have been deprecated for a long time (Sakthipriyan Vairamani) #2859.
* console: (Breaking) Values reported by console.time() now have 3 decimals of
  accuracy added (Michaël Zasso) #3166.
* fs:
  - fs.readFile*(), fs.writeFile*(), and fs.appendFile*() now also accept a file
    descriptor as their first argument (Johannes Wüller) #3163.
  - (Breaking) In fs.readFile(), if an encoding is specified and the internal
    toString() fails the error is no longer thrown but is passed to the callback
    (Evan Lucas) #3485.
  - (Breaking) In fs.read() (using the fs.read(fd, length, position, encoding,
    callback) form), if the internal toString() fails the error is no longer
    thrown but is passed to the callback (Evan Lucas) #3503.
* http:
  - Fixed a bug where pipelined http requests would stall (Fedor Indutny) #3342.
  - (Breaking) When parsing HTTP, don't add duplicates of the following headers:
    Retry-After, ETag, Last-Modified, Server, Age, Expires. This is in addition
    to the following headers which already block duplicates: Content-Type,
    Content-Length, User-Agent, Referer, Host, Authorization,
    Proxy-Authorization, If-Modified-Since, If-Unmodified-Since, From, Location,
    Max-Forwards (James M Snell) #3090.
  - (Breaking) The callback argument to OutgoingMessage#setTimeout() must be a
    function or a TypeError is thrown (James M Snell) #3090.
  - (Breaking) HTTP methods and header names must now conform to the RFC 2616
    "token" rule, a list of allowed characters that excludes control characters
    and a number of separator characters. Specifically, methods and header names
    must now match /^[a-zA-Z0-9_!#$%&'*+.^`|~-]+$/ or a TypeError will be thrown
    (James M Snell) #2526.
* node:
  - (Breaking) Deprecated the _linklist module (Rich Trott) #3078.
  - (Breaking) Removed require.paths and require.registerExtension(), both had
    been previously set to throw Error when accessed
    (Sakthipriyan Vairamani) #2922.
* npm: Upgraded to version 3.3.6 from 2.14.7, see
  https://github.com/npm/npm/releases/tag/v3.3.6 for more details. This is a
  major version bump for npm and it has seen a significant amount of change.
  Please see the original npm v3.0.0 release notes for a list of major changes
  (Rebecca Turner) #3310.
* src: (Breaking) Bumped NODE_MODULE_VERSION to 47 from 46, this is necessary
  due to the V8 upgrade. Native add-ons will need to be recompiled
  (Rod Vagg) #3400.
* timers: Attempt to reuse the timer handle for setTimeout().unref(). This fixes
  a long-standing known issue where unrefed timers would perviously hold
  beforeExit open (Fedor Indutny) #3407.
* tls:
  - Added ALPN Support (Shigeki Ohtsu) #2564.
  - TLS options can now be passed in an object to createSecurePair()
    (Коренберг Марк) #2441.
  - (Breaking) The default minimum DH key size for tls.connect() is now 1024
    bits and a warning is shown when DH key size is less than 2048 bits. This a security consideration to prevent "logjam" attacks. A new minDHSize TLS
    option can be used to override the default. (Shigeki Ohtsu) #1831.
* util:
  - (Breaking) util.p() was deprecated for years, and has now been removed
    (Wyatt Preul) #3432.
  - (Breaking) util.inherits() can now work with ES6 classes. This is considered
    a breaking change because of potential subtle side-effects caused by a
    change from directly reassigning the prototype of the constructor using
    `ctor.prototype = Object.create(superCtor.prototype, { constructor: { ... } })`
    to using `Object.setPrototypeOf(ctor.prototype, superCtor.prototype)`
    (Michaël Zasso) #3455.
* v8: (Breaking) Upgraded to 4.6.85.25 from 4.5.103.35 (Ali Ijaz Sheikh) #3351.
  - Implements the spread operator, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator
    for further information.
  - Implements new.target, see
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target
    for further information.
* zlib: Decompression now throws on truncated input (e.g. unexpected end of
  file) (Yuval Brik) #2595.

PR-URL: #3466
This was referenced Nov 25, 2015
Is there a proper way to send zlib partial files for unzipping? #4030
Closed
Error reading some .nii.gz files bids-standard/legacy-validator#52
Closed
@jhamhader jhamhader mentioned this pull request Mar 17, 2016
Gzip decoder too strict #5761
Closed
@tomhughes tomhughes mentioned this pull request Mar 31, 2016
Fix "incomplete" test fixture to be valid gzip of incomplete data mapbox/tilelive#167
Merged
@conradoplg conradoplg mentioned this pull request Oct 25, 2016
Zlib._handle.onerror unexpected end of file ttezel/twit#300
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

semver-major PRs that contain breaking changes and should be released in the next major version. zlib Issues and PRs related to the zlib module and its compression dependencies.

Projects

None yet

Milestone

5.0.0

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@jhamhader @Fishrock123 @indutny @trevnorris @thefourtheye @targos
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.