Currently ZipFile.writestr writes the local time into the ZipFile.
(depends on both current time and local timezone)
See
|
zinfo = ZipInfo(filename=zinfo_or_arcname, |
|
date_time=time.localtime(time.time())[:6]) |
This makes pip installing a package generate non-reproducible build artifacts.
Specifically, Scripts/*.exe files (created for packages that define entry_points/console_scripts) are not reproducible on Windows when installed by pip. This also leaks into the *.dist-info/RECORD files.
For example, after running pip install wheel or pip install pyinstaller,
in wheel-0.37.1.dist-info/RECORD, I have this line:
../../Scripts/wheel.exe,sha256=u9TbPw2XNs_F9uy7y2zwumuzAZDbOSB7BXjLHZ0tTHg,97103
in pyinstaller-4.10.dist-info/RECORD, I have these lines:
../../Scripts/pyi-archive_viewer.exe,sha256=nC-9muPlIhUC1qvFkXHpyKJyRQqXISXxbUPXQ1XVOiM,97133
../../Scripts/pyi-bindepend.exe,sha256=udFHiAdndPpSwaIqmhmLEy36IUs1cNNoNQznSEnLJQQ,97128
../../Scripts/pyi-grab_version.exe,sha256=3ET9E841tFWujFL99aG4frzgwlP9f9pAkMgE0k2UGK0,97131
../../Scripts/pyi-makespec.exe,sha256=dJkfmITdLJhyPngmqziqqj5tH9qqfeQc5BTubeoXWUs,97127
../../Scripts/pyi-set_version.exe,sha256=sWmcOVS93fUY-wbdoz6ixBCvjy1tC4Aaw30DMmrmo-0,97130
../../Scripts/pyinstaller.exe,sha256=haInbhH0pImJn24cW4v917oUZmzXZj8OE89KFh4MO2Y,97112
Upon comparing multiple Scripts/wheel.exe files, I've found that the only difference is due to the above-mentioned timestamp embedded inside the exe (or rather, same timestamp embedded twice).
The exe files get created by distlib (vendored by pip).
Here is a traceback with an artificial exception to illustrate the codepath:
(env) PS C:\tmp> pip install --no-build-isolation pyinstaller
Collecting pyinstaller
Using cached pyinstaller-4.10-py3-none-win_amd64.whl (2.0 MB)
Requirement already satisfied: setuptools in c:\tmp\env\lib\site-packages (from pyinstaller) (61.0.0)
Requirement already satisfied: pyinstaller-hooks-contrib>=2020.6 in c:\tmp\env\lib\site-packages (from pyinstaller) (2022.3)
Requirement already satisfied: altgraph in c:\tmp\env\lib\site-packages (from pyinstaller) (0.17.2)
Requirement already satisfied: pefile>=2017.8.1 in c:\tmp\env\lib\site-packages (from pyinstaller) (2021.9.3)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\tmp\env\lib\site-packages (from pyinstaller) (0.2.0)
Requirement already satisfied: future in c:\tmp\env\lib\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)
Installing collected packages: pyinstaller
ERROR: Exception:
Traceback (most recent call last):
File "C:\tmp\env\lib\site-packages\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "C:\tmp\env\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper
return func(self, options, args)
File "C:\tmp\env\lib\site-packages\pip\_internal\commands\install.py", line 405, in run
installed = install_given_reqs(
File "C:\tmp\env\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs
requirement.install(
File "C:\tmp\env\lib\site-packages\pip\_internal\req\req_install.py", line 769, in install
install_wheel(
File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 729, in install_wheel
_install_wheel(
File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 646, in _install_wheel
generated_console_scripts = maker.make_multiple(scripts_to_generate)
File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 440, in make_multiple
filenames.extend(self.make(specification, options))
File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 427, in make
return super().make(specification, options)
File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 429, in make
self._make_script(entry, filenames, options=options)
File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 329, in _make_script
self._write_script(scriptnames, shebang, script, filenames, ext)
File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 263, in _write_script
raise Exception(f"heyheyhey2. {sha256(launcher)=}. {sha256(shebang)=}. {sha256(zip_data)=}. " +
Exception: heyheyhey2. sha256(launcher)='a00a877acefc'. sha256(shebang)='58628e924f22'. sha256(zip_data)='a423496a0482'. ('SOURCE_DATE_EPOCH' in os.environ)=True
The interesting code is here:
https://github.com/pypa/distlib/blob/d0e3f49df5d1aeb9daeaaabf0391c9e13e4a6562/distlib/scripts.py#L251-L252
This calls into the cpython standard library, where time.time() gets written into the file:
|
zinfo = ZipInfo(filename=zinfo_or_arcname, |
|
date_time=time.localtime(time.time())[:6]) |
Ideally, either distlib or the stdlib zipfile module should be changed to respect SOURCE_DATE_EPOCH, but it's not entirely clear to me which...
The attached patch changes ZipFile.writestr to respect SOURCE_DATE_EPOCH if set.
related pypa/distlib#164
related spesmilo/electrum#7739