GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src: make base64 decoding 10-15% faster by bnoordhuis · Pull Request #2193 · nodejs/node

Conversation

@bnoordhuis

@bnoordhuis bnoordhuis commented Jul 16, 2015

Copy link
Copy Markdown
Member

Make the inner loop execute fewer compare-and-branch executions per
processed byte, resulting in a 10-15% speedup.

This coincidentally fixes an out-of-bounds read:

while (unbase64(*src) < 0 && src < srcEnd)

Should have read:

while (src < srcEnd && unbase64(*src) < 0)

But this commit removes the offending code altogether.

Fixes: #2166

R=@trevnorris?

CI: https://jenkins-iojs.nodesource.com/view/iojs/job/iojs+any-pr+multi/155/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@mscdex mscdex added the c++ Issues and PRs that require attention from people who are familiar with C++. label Jul 16, 2015
thefourtheye
thefourtheye reviewed Jul 16, 2015
View reviewed changes
Comment thread src/string_bytes.cc Outdated

@thefourtheye thefourtheye Jul 16, 2015

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very minor nit. Is four spaces here, okay?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@thefourtheye thefourtheye Jul 16, 2015

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we really cast the result to uint8_t when the actual data is int8_t?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@bnoordhuis bnoordhuis Jul 16, 2015

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The alternative was to change the type of the list elements to uint8_t but then I'd also have to change all the -1 to 255 to squelch compiler warnings. It would make the diff a lot noisier for no reason; conversion from signed to unsigned is well-defined.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@bnoordhuis bnoordhuis Jul 17, 2015

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is four spaces here, okay?

I don't think we really have a convention for that but I'll change it to two spaces before landing.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bnoordhuis
bnoordhuis force-pushed the optimize-base64-decode branch from 7f8acb6 to a5df468 Compare July 17, 2015 08:42
@bnoordhuis

bnoordhuis commented Jul 17, 2015

Copy link
Copy Markdown
Member Author

Incorporated feedback, PTAL.

@trevnorris Good suggestion about force-flattening the string first. I'm confident saying now that it's actually 50% faster. :-)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@trevnorris

trevnorris commented Jul 17, 2015

Copy link
Copy Markdown
Contributor

LGTM
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ronkorving

ronkorving commented Jul 21, 2015

Copy link
Copy Markdown
Contributor

@bnoordhuis then I guess you can rename this PR :) great job!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@indutny
indutny force-pushed the master branch from bffb204 to eb35968 Compare July 22, 2015 21:21
bnoordhuis added 2 commits July 25, 2015 19:07
@bnoordhuis
src: make base64 decoding 50% faster
8fd3ce1 
Make the inner loop execute fewer compare-and-branch executions per
processed byte, resulting in a 50% or more speedup.

This coincidentally fixes an out-of-bounds read:

    while (unbase64(*src) < 0 && src < srcEnd)

Should have read:

    while (src < srcEnd && unbase64(*src) < 0)

But this commit removes the offending code altogether.

Fixes: nodejs#2166
PR-URL: nodejs#2193
Reviewed-By: Trevor Norris <[email protected]>
@bnoordhuis
test: fix valgrind uninitialized memory warning
ac70bc8 
parallel/test-buffer called `Buffer.prototype.toString()` on a buffer
with uninitialized memory.  Call `Buffer.prototype.fill()` on it first.

PR-URL: nodejs#2193
Reviewed-By: Trevor Norris <[email protected]>
@bnoordhuis
bnoordhuis force-pushed the optimize-base64-decode branch from a5df468 to ac70bc8 Compare July 25, 2015 17:13
@bnoordhuis bnoordhuis closed this Jul 25, 2015
@bnoordhuis
bnoordhuis deleted the optimize-base64-decode branch July 25, 2015 17:13
@bnoordhuis
bnoordhuis merged commit ac70bc8 into nodejs:master Jul 25, 2015
@bnoordhuis

bnoordhuis commented Jul 25, 2015

Copy link
Copy Markdown
Member Author

Landed in 8fd3ce1 and ac70bc8 with hex values, thanks everyone.

I only added @trevnorris in the Reviewed-By because he was the only one to formally LGTM it.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@YurySolovyov

YurySolovyov commented Jul 25, 2015

Copy link
Copy Markdown

Is it going to be in 2.5.0, 3.0 or both?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@Fishrock123

Fishrock123 commented Jul 26, 2015

Copy link
Copy Markdown
Contributor

@YuriSolovyov 2.5.0+ (both) :)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
This was referenced Jul 27, 2015
Release Proposal: v2.5.0 #2239
Merged
Release proposal: v3.0.0 #2221
Closed
@aqrln aqrln mentioned this pull request Apr 4, 2017
buffer: optimize decoding wrapped base64 data #12146
Closed
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Invalid read in node::base64_decode<char>(char*, unsigned long, char const*, unsigned long)

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@bnoordhuis @trevnorris @ronkorving @YurySolovyov @Fishrock123 @ChALkeR @rvagg @thefourtheye @petkaantonov @mscdex
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.