Bug report

If the PAX header of a tar file contains an invalid value for the keyword size, such as size=xyz, the current behaviour of tarfile is to set it to 0 (see here). In this case it results in incomplete extraction, the extracted file will be of 0 size even the data is intact, and it stops extracting subsequent records in the tar file.

Let say we have 3 files foo, bar and quux:

-rw-r--r--@ 1 user  group    71K  6 Nov 16:03 bar
-rw-r--r--@ 1 user  group   211K  6 Nov 16:02 foo
-rw-r--r--@ 1 user  group    25K  6 Nov 16:03 quux

Using GNU tar to create a tar file with invalid PAX header size=xyz:

$ tar cf invalid_size.tar --format=pax --pax-option='size=xyz' foo bar quux

Now extract that tar file using tarfile:

import tarfile
with tarfile.open("invalid_size.tar") as tf:
    tf.extractall()

It will only produce foo with size 0. This is not consistent with the behaviour of both GNU tar and bsdtar:

# using GNU tar (1.35)
$ tar xvf invalid_size.tar
tar: Malformed extended header: invalid size=xyz
foo
tar: Malformed extended header: invalid size=xyz
bar
tar: Malformed extended header: invalid size=xyz
quux
tar: Exiting with failure status due to previous errors

# using bsdtar (3.5.3)
$ tar xvf invalid_size.tar
x foo
x bar
x quux

Despite complaining, GNU tar still extracts the 3 files in full and bsdtar doesn't seem to care at all, it just extracts the 3 files in full without any warning message. (used diff to check the content with the original files and they are identical)

To fix this, one approach is to raise an exception when it fails to parse size (found this PR when looking up this issue), but this will break existing scripts. Instead I propose tarfile should follow what GNU tar and bsdtar are doing, ignore any invalid value of size (i.e. do not override with size=0) and proceed to extract all the records in the tar archive. This won't break any existing scripts, preserves data to end users and is consistent with popular tools.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-161588: Ignore invalid PAX header 'size' in tarfile to extract files properly #161589