GeezerVison is showing github.com — test_mmap.test_access_parameter failing on macOS Sonoma 14.0 beta on Apple Silicon · Issue #107888 · python/cpython
test_mmap.test_access_parameter failing on macOS Sonoma 14.0 beta on Apple Silicon #107888DescriptionBug report
The other day I decided to give the MacOS Sonoma beta a go on my M1 MBP. test_mmap is now consistently failing with a PermissionError on main and 3.12 branches.
A clear and concise description of the bug
I happened to have an old MacBook Air laying about which is still running MacOS Monterey. I verified that all tests pass there.
Here's the output from the failing test:
test_access_parameter (test.test_mmap.MmapTests.test_access_parameter) ... ERROR
======================================================================
ERROR: test_access_parameter (test.test_mmap.MmapTests.test_access_parameter)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/skip/src/python/cpython/Lib/test/test_mmap.py", line 258, in test_access_parameter
m = mmap.mmap(f.fileno(), mapsize, prot=prot)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 1] Operation not permitted
Your environment
- CPython versions tested on:
main and 3.12 branches
- Operating system and architecture: MacBook Pro M1 Pro, MacOS Sonoma 14.0 beta 5
Linked PRs
Activitysmontanaro commented on Aug 12, 2023 ContributorAuthor More actions Failing on the 3.11 branch as well. I don't know if this is important enough to elevate for 3.12.0rc1, but Sonoma is supposed to be released in the fall. Fixing this issue now might be worth it. smontanaro commented on Aug 12, 2023 ContributorAuthor More actions ronaldoussoren commented on Aug 12, 2023 MacOS 14 is still in beta and contains bugs. We generally don't add workarounds for bugs in beta versions of an OS.
Is this something that can be reproduced in C? ronaldoussoren commented on Aug 12, 2023 For me the test appears to pass just fine on the current beta (14.0, 23A5312d):
$ python3.12 -m unittest test.test_mmap.MmapTests.test_access_parameter
.
----------------------------------------------------------------------
Ran 1 test in 0.153s
OK
Likewise with 3.11. Both are the latest release installed using the python.org installers, and likewise while running the test_mmap tests using python3.12 -m test test.test_mmap -v.
EDIT: This is in an x86_64 VM, I don't have an arm64 vm/machine running the 14.0 beta. smontanaro commented on Aug 12, 2023 ContributorAuthor More actions That's fine. I wasn't sure what three policy was, and thought it better to err on the side of caution.
In case it's a problem with the Sonoma beta, I provided feedback to Apple with a reference to this issue as well. changed the title [-]test_mmap.test_access_parameter failing on MacOS Sonoma 14.0 beta[/-] [+]test_mmap.test_access_parameter failing on macOS Sonoma 14.0 beta[/+] on Aug 12, 2023ned-deily commented on Aug 12, 2023 I can also reproduce the failure on an arm64 Sonoma beta. While it seems to be a change in OS behavior, the test is supposed to fail anyway; it now unexpectedly fails earlier.
# Try writing with PROT_EXEC and without PROT_WRITE
prot = mmap.PROT_READ | getattr(mmap, 'PROT_EXEC', 0)
with open(TESTFN, "r+b") as f:
m = mmap.mmap(f.fileno(), mapsize, prot=prot)
self.assertRaises(TypeError, m.write, b"abcdef")
self.assertRaises(TypeError, m.write_byte, 0)
m.close()
With the Sonoma beta on arm64, the mmap.mmap() call is failing before it gets to try the Write calls that should fail. Dunno how significant a change that is or whether it might be an ill-defined test case. ronaldoussoren commented on Aug 18, 2023 The test itself looks sane.
Is this something that can be reproduced in a small C program?
The following code should be equivalent to the call to mmap.mmap in the test and works on my M1 laptop running 13.5:
#include <sys/mman.h>
#include <stdio.h>
#include <fcntl.h>
int main(void)
{
int fd = open("/dev/zero", O_RDONLY);
if (fd == -1) {
perror("open");
return 1;
}
void* m = mmap(NULL, 10, PROT_READ|PROT_EXEC, 0, fd, 0);
if (m == NULL) {
perror("mmap");
return 1;
}
printf("done\n");
return 0;
}If this does work, does code signing the binary with the same entitlements as CPython also work? Likewise code signing with a JIT entitlement. smontanaro commented on Aug 18, 2023 ContributorAuthor More actions Thanks, Ronald. That works. I have no idea what you're talking about "code signing" though. ronaldoussoren commented on Aug 19, 2023
Thanks, Ronald. That works. I have no idea what you're talking about "code signing" though.
Code signing as documented by Apple
It is also possible to test this the other way:
$ cp /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python Python.backup
$ codesign --remove-signature /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
$ codesign -s - /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
$ /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python -m test test.test_mmap -v
$ cp Python.backup /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python (The dollar signs at the start of lines represent the shell prompt and shouldn't be copied when executing this).
This makes a copy of the interpreter in the framework, then replaces the code signing and entitlements by a trivial code signature that's required on Apple Silicon, then runs the test and finally restores the original signed versions of the interpreter.
Doing this tells is if the error is related to changes in the hardened runtime on macOS, or is a general behaviour change that affects the mmap extension module. ned-deily commented on Aug 24, 2023
Doing this tells is if the error is related to changes in the hardened runtime on macOS, or is a general behaviour change that affects the mmap extension module.
FWIW, I was able to test this with the most recent macOS 14 Sonoma Developer Preview on both an Intel (x86_64) VM and an Apple Silicon (arm64) VM. test_mmap appears to only fail on Apple Silicon systems running Sonoma and it fails regardless of code signing and hardened runtime usage. That is, the test_mmap failures are seen when running from uninstalled developer builds (./configure && make && ./python.exe -m test test_mmap) as well as running from the latest python.org installer binaries (which are signed and use the hardened runtime). changed the title [-]test_mmap.test_access_parameter failing on macOS Sonoma 14.0 beta[/-] [+]test_mmap.test_access_parameter failing on macOS Sonoma 14.0 beta on Apple Silicon[/+] on Aug 24, 2023
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|