GeezerVison is showing github.com — Unvendor `libmpdec` sources · Issue #115119 · python/cpython
Unvendor libmpdec sources #115119Closed 1 / 11 of 1 issue completedClosed 1 / 11 of 1 issue completed DescriptionTo facilitate cleaner updates of the externally-maintained libmpdec required by the _decimal module, we should migrate away from the bundled copy in Modules/_decimal/libmpdec and towards an "external" in cpython-source-deps for Windows and --with-system-libmpdec by default elsewhere.
My tentative plan is as follows:
- In 3.13:
- In 3.15:
- In 3.16:
Linked PRs
Reacted by Erlend E. Aasland, Kirill Podoprigora, Ned Deily, Sam James and John Activityerlend-aasland commented on Feb 7, 2024
[...] with a fallback to the bundled copy (maybe with a warning?)
+1 for an autoconf warning. ned-deily commented on Feb 8, 2024 Adding an external libmpdec to the macOS installer build shouldn't be a big issue. Assign it to me if you proceed with this. Reacted by Zachary Ware and Erlend E. Aasland erlend-aasland commented on Feb 8, 2024 FTR: the source deps repo is now updated. erlend-aasland commented on Feb 8, 2024 @ned-deily: for build-installer.py, is there more to do than adding a configuration for mpdecimal? I've used the following configure options: ['--disable-cxx', 'MACHINE=universal'] ned-deily commented on Feb 8, 2024 Likely yes, I'll take care of it. Thanks. Reacted by Erlend E. Aasland ned-deily commented on Feb 13, 2024 I took a quick look at this and be aware that this change proposal will likely impact everyone building Python 3.13 on macOS. macOS does not ship a copy of libmpdec so, if the bundled version is removed, anyone building Python on macOS will now need to provide a local copy, either building from source or using a version from one of the widely-used third-party package managers, like Homebrew or MacPorts. Both currently have mpdecimal packages at the 4.0.0 level. (Recall that, unlike Windows builds, we do not currently provide non-vendored third-party library binaries for macOS users, other than those built for and linked into the Pythons provided by python.org installers for macOS.) In a quick test, it appeared that with ./configure --with-system-libmpdec today, pkg-config info for libmpdec was not being used. If so, that should be corrected. And, in any case, the devguide instructions for installing dependencies will need to be updated to cover libmpdec. We should add all this and get this working before moving on to changing the installer. Reacted by Erlend E. Aasland erlend-aasland commented on Feb 13, 2024
In a quick test, it appeared that with ./configure --with-system-libmpdec today, pkg-config info for libmpdec was not being used. If so, that should be corrected.
Last time I checked, libmpdec did not provide pkgconfig files:
$ ls $(brew --prefix libmpdec)/lib/pkgconfig
ls: /opt/homebrew/opt/mpdecimal/lib/pkgconfig: No such file or directory
$ cd mpdecimal-2.5.1; find . | grep pkg
$ ned-deily commented on Feb 13, 2024 With the current 4.0.0 version, it appears both Homebrew and MacPorts ship them:
$ pkg-config libmpdec --cflags
-I/opt/homebrew/Cellar/mpdecimal/4.0.0/include
$ pkg-config libmpdec --libs
-L/opt/homebrew/Cellar/mpdecimal/4.0.0/lib -lmpdec -lm
...
$ pkg-config libmpdec --libs
-L/opt/macports/lib -lmpdec -lm Reacted by Erlend E. Aasland erlend-aasland commented on Feb 13, 2024 Yes, the 4.0.0 version does indeed provide pkg-config configuration files. I'll work on an Autoconf patch. zware commented on May 13, 2026 I consider the disappearance of libmpdec{,-dev} in Ubuntu 24.04 to be a bug in that version of Ubuntu; lobby them to fix it. asottile commented on May 13, 2026 it's an LTS, they don't add new packages. I've tried many times asottile commented on May 13, 2026 it's also not present in debian:bookworm vstinner commented on May 13, 2026 Can't you add libmpdev in deadsnakes PPA? Reacted by Petr Viktorin and Zachary Ware eli-schwartz commented on May 13, 2026
@eli-schwartz I don't control the ubuntu repositories just the deadsnakes ones. unless you're suggesting "backporting" via revendoring in a patch?
I quite literally mean backport the *.deb packaging format package for a dpkg package named "libmpdec". If I'd meant reverting a patch to revendor the source directory I'd say "forward port the libmpdec sources from older python", which is the opposite of a backport.
The repository that you control is deadsnakes, hence backport from Ubuntu 26.04 to deadsnakes. skirpichev commented on May 14, 2026
(noticing this because of nightly builds for deadsnakes) -- removing the bundled copy makes python unbuildable on ubuntu 24.04 because there is no system libmpdec there
@asottile, could you please point to build logs or elaborate a bit exact reason of failure? Missing libmpdec shouldn't prevent CPython 3.16 from build:
$ ./configure -q
configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module
$ make -s
Written build/lib.linux-x86_64-3.16/_sysconfigdata__linux_x86_64-linux-gnu.py
Written build/lib.linux-x86_64-3.16/_sysconfig_vars__linux_x86_64-linux-gnu.json
The necessary bits to build these optional modules were not found:
_decimal
To find the necessary bits, look in configure.ac and config.log.
Checked 116 modules (37 built-in, 77 shared, 1 n/a on linux-x86_64, 0 disabled, 1 missing, 0 failed on import)
$ ./python -m test test_decimal
Using random seed: 1990606034
0:00:00 load avg: 2.12 Run 1 test sequentially in a single process
0:00:00 load avg: 2.12 [1/1] test_decimal
C tests skipped: no module named _decimal.
0:00:15 load avg: 2.24 [1/1] test_decimal passed
== Tests result: SUCCESS ==
1 test OK.
Total duration: 15.6 sec
Total tests: run=579 skipped=198
Total test files: run=1/1
Result: SUCCESS
It might be that differences of pure-Python decimal module wrt the C extension break some your CI tests. In this case, please open issues.
I consider the disappearance of libmpdec{,-dev} in Ubuntu 24.04 to be a bug in that version of Ubuntu; lobby them to fix it.
They blindly follow unprofessional actions of Debian maintainers: https://bugs.debian.org/1056785.
BTW, this is not a problem for ubuntu 22.04 or 26.04. Tentative schedule for new GA images: actions/runner-images#13964 (comment)
Edit:
I would prefer waiting to remove the bundled copy until 24.04 is end-of-lifed
That means April 2034, i.e ~10 years. Any good reasons for this huge delay, given that the libmpdec is available in PPA and it's trivial to build it from sources? added a commit that references this issue on Jun 1, 2026gh-115119: Remove superfluous TEST_COVERAGE private macro from _decim… added a commit that references this issue on Jun 2, 2026gh-115119: Fix libmpdec detection when pkg-config files are missing (#… edmorley commented on Sep 3, 2026 Reacted by Victor Stinner edmorley commented on Sep 22, 2026 Reacted by Victor Stinner Reacted by Victor Stinner added a commit that references this issue on Oct 12, 2026gh-115119: Remove deprecated --with-system-mpdec option from iOS buil…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|