Description

sethmlarson

opened on Jul 2, 2024

Issue body actions

Bug description:

Today the tarfile module parsing of header values allows for backtracking when parsing header values. Headers have a well-known format that doesn't require backtracking to parse reliably, the new method of parsing will only require a single pass over a byte stream.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-121285: Remove backtracking when parsing tarfile headers #121286
• [3.13] gh-121285: Remove backtracking when parsing tarfile headers (GH-121286) #123542
• [3.12] gh-121285: Remove backtracking when parsing tarfile headers (GH-121286) #123543
• [3.11] gh-121285: Remove backtracking when parsing tarfile headers (GH-121286) #123639
• [3.10] gh-121285: Remove backtracking when parsing tarfile headers (GH-121286) #123640
• [3.9] gh-121285: Remove backtracking when parsing tarfile headers (GH-121286) #123641
• [3.8] gh-121285: Remove backtracking when parsing tarfile headers (GH-121286) #123642

Reacted by Gregory P. Smith

Activity

1. sethmlarson

   added

   type-securityA security issueA security issue

   on Jul 2, 2024
2. gpshead

   added a commit that references this issue on Aug 31, 2024

   gh-121285: Remove backtracking when parsing tarfile headers (GH-121286)

   34ddb64
3. miss-islington

   added 2 commits that reference this issue on Aug 31, 2024

   pythongh-121285: Remove backtracking when parsing tarfile headers (py…

   c47341f

   pythongh-121285: Remove backtracking when parsing tarfile headers (py…

   aef7f11
4. gpshead

   assigned

   gpshead

   and

   sethmlarson

   on Aug 31, 2024
5. gpshead commented on Aug 31, 2024

   

   gpshead

   on Aug 31, 2024

   Member

   More actions

   main version merged. I'll let you work out any non-automatic backports.

   Reacted by Seth Larson
6. gpshead

   added a commit that references this issue on Aug 31, 2024

   [3.12] gh-121285: Remove backtracking when parsing tarfile headers (G…

   4eaf489
7. picnixz

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   3.11only security fixesonly security fixes

   3.10 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   3.13only security fixesonly security fixes

   on Sep 1, 2024
8. picnixz

   added this to Tarfile issues 🗄️on Sep 1, 2024
9. 4 remaining items
10. sethmlarson

    added 3 commits that reference this issue on Sep 3, 2024

    [3.10] pythongh-121285: Remove backtracking when parsing tarfile head…

    3ea4396

    [3.9] pythongh-121285: Remove backtracking when parsing tarfile heade…

    daf223e

    [3.8] pythongh-121285: Remove backtracking when parsing tarfile heade…

    80b31d4
11. pablogsal

    added 2 commits that reference this issue on Sep 3, 2024

    [3.11] gh-121285: Remove backtracking when parsing tarfile headers (G…

    d449caf

    [3.10] gh-121285: Remove backtracking when parsing tarfile headers (G…

    743acbe
12. frenzymadness commented on Sep 4, 2024

    

    frenzymadness

    on Sep 4, 2024

    Contributor

    More actions

    The CVE description contains:

    Regular expressions that allowed excessive backtracking during tarfile.TarFile header parsing are vulnerable to ReDoS via specifically-crafted tar archives.

    but the test added to verify the fix does not cause/test the ReDoS. I mean, when you run the test with unpatched Python, it raises tarfile.ReadError as well just with a different message, for example:

    tarfile.ReadError: file could not be opened successfully:
    - method gz: ReadError('not a gzip file')
    - method bz2: ReadError('not a bzip2 file')
    - method xz: ReadError('not an lzma file')
    - method tar: ReadError('bad checksum')
    

    Could you please help me understand how one of the regexes (fox example: br"(\d+) ([^=]+)=") can cause a regex backtracking? There is no combination of greedy nested quatifiers, greedy quantifiers are limited by the character class and I don't see a need for the regex algorithm to go back and consider another path if the match is not found.

    Reacted by Miro Hrončok and Pastea
13. sethmlarson commented on Sep 4, 2024

    

    sethmlarson

    on Sep 4, 2024

    ContributorAuthor

    More actions

    @frenzymadness Apologies for the confusion, not all of the regular expressions in the PR have issues with backtracking. The regex you mention was replaced to process the entire tar header block in one pass rather than multiple passes.
14. ambv

    added 2 commits that reference this issue on Sep 4, 2024

    [3.8] gh-121285: Remove backtracking when parsing tarfile headers (GH…

    7d1f50c

    [3.9] gh-121285: Remove backtracking when parsing tarfile headers (GH…

    b4225ca
15. ambv

    closed this as completedon Sep 4, 2024
16. github-project-automation

    moved this from In Progress to Done in Tarfile issues 🗄️on Sep 4, 2024
17. frenzymadness

    added 2 commits that reference this issue on Sep 5, 2024

    00437: [3.8] pythongh-121285: Remove backtracking when parsing tarfil…

    182d6ca

    00437: [3.8] pythongh-121285: Remove backtracking when parsing tarfil…

    3209abb
18. mcepl commented on Sep 18, 2024

    

    mcepl

    on Sep 18, 2024

    Contributor

    More actions

    Patch for Python 3.8 applies more or less painlessly on the 3.6 branch as well.
19. mcepl

    added a commit that references this issue on Oct 9, 2024

    [CVE-2024-6232] Remove backtracking when parsing tarfile headers

    bfc7d56

Issue actions