GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-29427: allow unpadded input and ouput in base64 module by guillp · Pull Request #7072 · python/cpython

Conversation

@guillp

@guillp guillp commented May 23, 2018 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Allow base64.b64decode() to accept unpadded input instead of throwing a binascii.Error. Padding check is switchable using the new padded boolean argument, and is enabled by default to maintain backward compatibility.
In a symmetric fashion, allow base64.b64encode() to produce unpadded output, using the padded argument. Default output will include padding for backward compatibility.

Updated test_base64 unittests to include new tests for optional padding.

https://bugs.python.org/issue29427

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
mnme and pkong-ds reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
guillp added 2 commits May 23, 2018 17:10
@guillp
allow parsing and base64
d6d4d93 
Allow base64.b64decode() to accept unpadded input instead of throwing a binascii.Error. Padding check is switchable using the new `padded` boolean argument, and is enabled by default to maintain backward compatibility.
In a symmetric fashion, allow base64.b64encode() to produce unpadded output, using the `padded` argument. Default output will include padding for backward compatibility.
@guillp
updated base64 unittests for unpadded input/output
dd69e85 
Added some test cases for base64 module, to test optional padding in b64decode() and b64encode(), urlsafe_b64encode() and urlsafe_b64decode().
@the-knights-who-say-ni

the-knights-who-say-ni commented May 23, 2018

Copy link
Copy Markdown

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately our records indicate you have not signed the CLA. For legal reasons we need you to sign this before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

When your account is ready, please add a comment in this pull request
and a Python core developer will remove the CLA not signed label
to make the bot check again.

Thanks again to your contribution and we look forward to looking at it!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@the-knights-who-say-ni the-knights-who-say-ni added the CLA not signed label May 23, 2018
@bedevere-bot bedevere-bot added the awaiting review label May 23, 2018
@guillp
added NEWS.d entry for bpo-29427
698ec49
@guillp

guillp commented May 24, 2018

Copy link
Copy Markdown
Author

Good catch, this indeed breaks when the string to decode contains non-base64 characters, which would be ignored by binascii.a2b_decode().
I don't think we should reject strings that are incompletely padded, we should ignore the adding instead (just like it was a non-base64 character); unless validate=True, in that case we should reject strings that contain padding.

If you agree, I'll push a fix including all that.

BTW, I signed the CLA.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@guillp
fixed b64decode on non padded strings
8eb9823 
Fixes b64decode on non padded input if that input contains invalid characters.
Implements validate=True for non padded input.
@the-knights-who-say-ni the-knights-who-say-ni added CLA signed and removed CLA not signed labels May 24, 2018
guillp added 2 commits May 24, 2018 13:31
@guillp
more base64 test cases for unpadded input
df1102c
@guillp
fixed indentation
1b307ea
@FranklinYu

FranklinYu commented Aug 20, 2019

Copy link
Copy Markdown

Any news?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@djc

djc commented Sep 3, 2019

Copy link
Copy Markdown
Contributor

@guillp there seem to be some whitespace issues (according to Travis). Could you fix those up? I'd love to have this feature!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
sir-sigurd
sir-sigurd reviewed Sep 3, 2019
View reviewed changes

@sir-sigurd sir-sigurd left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that documentation should be updated.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Misc/NEWS.d/next/Library/2018-05-23-18-02-03.bpo-29427.82cb18.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/base64.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/base64.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@bitti

bitti commented Oct 6, 2019

Copy link
Copy Markdown

What's the holdup? Can't the maintainer edit the PR if @guillp seems to have forgotten about it?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
guillp and others added 2 commits October 8, 2019 14:57
@guillp @sir-sigurd
Update Misc/NEWS.d/next/Library/2018-05-23-18-02-03.bpo-29427.82cb18.rst
77618ad 
updated commit description

Co-Authored-By: Sergey Fedoseev <[email protected]>
@guillp @sir-sigurd
typos
df3d326 
Co-Authored-By: Sergey Fedoseev <[email protected]>
@guillp

guillp commented Oct 8, 2019

Copy link
Copy Markdown
Author

Very sorry about that. git push to github.com is not allowed from my workplace, so I add to commit manually from github.com UI which is a pain. Thanks to @sir-sigurd for the review.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@dfrankow

dfrankow commented Feb 26, 2021

Copy link
Copy Markdown

I hope this is not dead, it would've been useful for me.
fasih reacted with eyes emoji
All reactions
  • 👀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
dfrankow
dfrankow reviewed Feb 26, 2021
View reviewed changes
Comment thread Lib/base64.py
if not padded:
if validate and not re.match(b'^[A-Za-z0-9+/]*$', s):
raise binascii.Error('Padding found in supposedly non-padded input')
s += b'=='

@dfrankow dfrankow Feb 26, 2021

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why two ==s is always the right padding. I may be missing something.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@bitti bitti Feb 27, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably depends on the base64 flavor of what is accepted as "right" padding, but as far as I understand the a2b_base64 implementation extra padding characters are just ignored, so always appending == should be safe in this context.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@dfrankow dfrankow Feb 27, 2021

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks. Maybe worth putting that in a comment in the code.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@fasih fasih Mar 18, 2021

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am using util from django.utils.http.urlsafe_base64_encode

https://docs.djangoproject.com/en/3.1/ref/utils/#django.utils.http.urlsafe_base64_encode

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
gofr
gofr reviewed Apr 27, 2021
View reviewed changes
Comment thread Lib/base64.py
_urlsafe_decode_translation = bytes.maketrans(b'-_', b'+/')

def urlsafe_b64encode(s):
def urlsafe_b64encode(s, validate=False, padded=True):

@gofr gofr Apr 27, 2021

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validate arg isn't used. And if it's going to be added it should be added to urlsafe_b64decode instead.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@Thorney Thorney mannequin mentioned this pull request Jul 2, 2022
Option to skip padding for base64 urlsafe encoding/decoding #73613
Closed
@ezio-melotti ezio-melotti removed the CLA signed label Jul 13, 2022
@Elkasitu

Elkasitu commented Apr 10, 2024

Copy link
Copy Markdown

Is there anything that can be done to push this change forward? Should a new PR be submitted if the original contributor has lost interest?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@guillp guillp closed this by deleting the head repository May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

5 more reviewers
@dfrankow dfrankow dfrankow left review comments
@bitti bitti bitti left review comments
@sir-sigurd sir-sigurd sir-sigurd left review comments
@fasih fasih fasih left review comments
@gofr gofr gofr left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

13 participants

@guillp @the-knights-who-say-ni @FranklinYu @djc @bitti @dfrankow @Elkasitu @nneonneo @sir-sigurd @fasih @gofr @ezio-melotti @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.