GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter by matthieucan · Pull Request #136877 · python/cpython

Conversation

@matthieucan

@matthieucan matthieucan commented Jul 20, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

With those changes, the MIME parameter parser discards parameters with an invalid section number that uses a digit not convertible to integer such as super-script "²" or "𐩃" (Kharosthi number).

For backwards compatibility, keep accepting non-ASCII digits that can be converted to integers, such as NKO digits.

Before:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> reg('Content-Disposition', 'inline; foo*0=bar')
'inline; foo="bar"'
>>> reg('Content-Disposition', 'inline; foo*X=bar')
'inline;'
>>> reg('Content-Disposition', 'inline; foo*0=bar; foo*߁=baz')
'inline; foo="barbaz"'
>>> reg('Content-Disposition', 'inline; foo*²=bar')
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    reg('Content-Disposition', 'inline; foo*²=bar')
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 604, in __call__
    return self[name](name, value)
           ~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 192, in __new__
    cls.parse(value, kwds)
    ~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 448, in parse
    kwds['parse_tree'] = parse_tree = cls.value_parser(value)
                                      ~~~~~~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2736, in parse_content_disposition_header
    disp_header.append(parse_mime_parameters(value[1:]))
                       ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2601, in parse_mime_parameters
    token, value = get_parameter(value)
                   ~~~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2464, in get_parameter
    token, value = get_section(value)
                   ~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2417, in get_section
    section.number = int(digits)
                     ~~~^^^^^^^^
ValueError: invalid literal for int() with base 10: '²'

After:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> reg('Content-Disposition', 'inline; foo*0=bar')
'inline; foo="bar"'
>>> reg('Content-Disposition', 'inline; foo*X=bar')
'inline;'
>>> reg('Content-Disposition', 'inline; foo*0=bar; foo*߁=baz')
'inline; foo="barbaz"'
>>> reg('Content-Disposition', 'inline; foo*²=bar')
'inline;'
  • Issue: email: ValueError in get_section when parsing header with non-ASCII digit #87112

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
matthieucan added 2 commits July 20, 2025 17:43
@matthieucan
Only allow 0-9 digits in MIME parameter section numbers
ff4ca5c
@matthieucan
Use gh bug nubmer
656b6fb
@matthieucan
matthieucan requested a review from a team as a code owner July 20, 2025 15:59
@bedevere-app bedevere-app Bot added the awaiting review label Jul 20, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Jul 20, 2025
email: ValueError in get_section when parsing header with non-ASCII digit #87112
Open
@matthieucan
Add news entry
75636a5
StanFromIreland
StanFromIreland reviewed Jul 20, 2025
View reviewed changes
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@picnixz picnixz changed the title gh-87112: Do not fail when non 0-9 digit is used as section number in MIME header parameter gh-87112: Ensure that only ASCII digits are accepted as section number in MIME header parameter Jul 20, 2025
picnixz
picnixz reviewed Jul 20, 2025
View reviewed changes
Comment thread Lib/test/test_email/test__header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
matthieucan added 2 commits July 20, 2025 19:17
@matthieucan
Comments from PR review, remove inner function
b15c404
@matthieucan
Rephrase News entry
7cba42f
StanFromIreland
StanFromIreland reviewed Jul 20, 2025
View reviewed changes
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_email/test__header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
picnixz
picnixz reviewed Jul 20, 2025
View reviewed changes
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/email/_header_value_parser.py Outdated
"found {}".format(value))
digits = ''
while value and value[0].isdigit():
while value and '0' <= value[0] <= '9':

@picnixz picnixz Jul 20, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
while value and '0' <= value[0] <= '9':
while value and ('0' <= value[0] <= '9'):

It will a bit clearer. Or you can still use a separate function to make it even cleareer. The bottleneck won't be the function call IMO.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@matthieucan matthieucan Jul 20, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that, but not the separate function. It was my understanding that @StanFromIreland was leaning towards not having an inner function

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@StanFromIreland StanFromIreland Jul 20, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, I was against the function to check if it is in a dictionary.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@matthieucan matthieucan Jul 20, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Just moved it to a separate function for extra-clarity

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
matthieucan added 2 commits July 20, 2025 19:35
@matthieucan
Comments from PR review, improve phrasing
d2d59ee
@matthieucan
Use separate function
54bbbb7
@AA-Turner AA-Turner added the sprint label Jul 20, 2025
@encukou

encukou commented Jul 22, 2025

Copy link
Copy Markdown
Member

Could you also test with, for example, ߅ (NKO DIGIT FIVE), which int accepts?
Given that some values worked before, IMO the HeaderParseError should only be raised if int() fails; for other non-ASCII numbers this should append an InvalidHeaderDefect.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@matthieucan

matthieucan commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

Could you also test with, for example, ߅ (NKO DIGIT FIVE), which int accepts? Given that some values worked before, IMO the HeaderParseError should only be raised if int() fails; for other non-ASCII numbers this should append an InvalidHeaderDefect.

Thank you for looking into this.

In my understanding, those are the possible scenarios:

  • The section number is made of ASCII digits ('0' <= d <= '9')
    • Valid, remains unchanged with this PR
  • The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE)
    • This was accepted before, without a defect
    • This PR, in its current state, rejects those (they are ignored and trigger a defect)
    • ➡️ ❓ @encukou Do I understand correctly that they should be accepted, and not raise a defect (even if they are not RFC-valid)? Or should they be accepted and raise a defect?
  • The section number is made of digits (isdigit(d)) which cannot be converted to integers (not int(d)), e.g. ²
    • This used to fail with a raised exception
    • This PR, in its current state, rejects those (they are ignored and trigger a defect)
  • The section number is not made of digits (not isdigit(d))
    • Invalid, remains unchanged with this PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@encukou

encukou commented Jul 23, 2025

Copy link
Copy Markdown
Member

The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE). This was accepted before, without a defect

IMO, they should be accepted and raise a defect.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
matthieucan added 2 commits July 27, 2025 14:16
@matthieucan
Accept digits that are convertible to int for backwards compatibility
f12e424
@matthieucan
Update news message
b364339
@matthieucan matthieucan changed the title gh-87112: Ensure that only ASCII digits are accepted as section number in MIME header parameter gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter Jul 27, 2025
@matthieucan

matthieucan commented Jul 27, 2025

Copy link
Copy Markdown
Contributor Author

The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE). This was accepted before, without a defect

IMO, they should be accepted and raise a defect.

Thank you, I agree. This is now implemented.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions

github-actions Bot commented Apr 27, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 27, 2026
@github-project-automation github-project-automation Bot added this to Sprint Apr 27, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Sprint Apr 27, 2026
@bitdancer

bitdancer commented Apr 27, 2026

Copy link
Copy Markdown
Member

I think that what we should to do here is change the incorrect isdigit to isdecimal and add as_ew_allowed=False to MIMEVersion. If I'm reading rfc 6532 correctly (though it is quite possible I'm not) then non-ascii decimal digits are not a defect unless you are serializing to ascii. (They should already show up as non-ascii defects when parsing with utf8=False.) If we make those changes that should eliminate the exception, and the new code proposed in #122540 should raise an error if the header is serialized with utf8=False.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label May 15, 2026
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 90 days with no activity.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@picnixz picnixz picnixz left review comments
@StanFromIreland StanFromIreland StanFromIreland left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review sprint stale Stale PR or inactive for long period of time.

Projects

Sprint
Status: Todo

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@matthieucan @encukou @bitdancer @picnixz @StanFromIreland @AA-Turner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.