GeezerVison

Strictly testing system calls against this list is too specific, this is just a recorded existing behavior of an existing implementation the insides of which are somewhat beyond our control (these come from an underlying libc kernel interface). we don't want to guarantee that these stay the same.

assertEqual against a strict list isn't what is desired, it will be a maintenance burden. Make this a less fragile, more specific, regression test for specific behaviors. Such as not having multiple fstat calls, or having at most two read calls.

I agree this is a a potentially fragile point, and lots could lead to a test failure even though the cpython implementation is as desired (ex. cpython _io, cpython _pyio, glibc, and musl differ around Buffered I/O). There are a lot of system calls which could happen as part of the Python standard file opening process which I want to validate don't ideally. Just saying "one stat", "two reads" "no ioctl" doesn't quite feel like it could help catch these changes.

Motivating examples I would like to be able to write behavior tests around:

1. CPython's open() builtin currently calls isatty to check if the file it just opened is "interactive" to determine buffered mode. If buffered, a BufferedIO is constructed which additionally wants to have an accurate position inside the file, and calls lseek, sometimes at __init__ (inside open), but sometimes when other I/O operations happen (read, write). The buffering behavior is visible, and documented, around stdout (https://docs.python.org/3/library/sys.html#sys.stdout) with a special environment variable to change it PYTHONUNBUFFERED. stderr has a different behavior than stdout. For this PR, I needed to do a flush=True in print() because the test python has its output redirected to a file which changes the behavior of the standard library making there be one write() system call at interpreter shutdown instead of calls when the print() statements are run. The isatty python check results in an ioctl system call (full stack at [0]).
2. gh-121313: Limit the reading size from pipes to their default buffer size on Unix systems #121315 in the initial implementation added a fstat to every read call to test "is this a pipe" (108e65b). That would fairly substantially reduce read performance. If I just test for at most one fstat over lifetime, could catch. But what if it was an attribute which ioctl is used to get? The performance slowdown was happening because of lots large allocation + de-allocation (Limit the reading size from pipes to their default buffer size on Unix systems #121313), which isn't a FD system call.

In some ways this feels like it interacts a bit with "what is a supported platform" (https://peps.python.org/pep-0011/). I can defer for now and count specific system calls (1 fstat over life of file, no ioctls, etc.) but worry it leaves some substantial gaps. I'm really not sure what the right set of tradeoffs is for long term maintenance + platform porting and needing to update tests like this one.

[0]

1. ioctl(TCGETS) (linux)
2. tcgettattr (glibc)
3. isatty (glibc)
4. io.isatty (cpython)
5. FileIO.isatty (cpython)
6. open() (cpython)

include a link to the specific gh cpython issue where additional calls were a problem.

It seems wrong to remove this subTest as part of this PR which is focused on making the strace test helper concept generic.

Agreed, order of operations vs. pr builds finishing / passing. I suspect this PR will take longer than the _USE_VFORK removal, but will separate out if we get close to it

gh-121383 / _USE_VFORK removal has been merged

move the assignment to the global _strace_working into this function so that the is None test and code assigning it to something other than that all lives together in one scope. (have _can_strace() return a boolean to be assigned perhaps)

this gets used, so perhaps don't make it private. Add a comment describing what exactly it is for. for the most part it looks like it'll be the same as stderr.

don't use str for child process output. The process emits binary output and no codec is declared or guaranteed.

It is okay to decode names once parsed out for high level use in StraceEvent.syscall, but the process itself shouldn't be expected to produce output matching any encoding.

(1) Always specify a codec on decode. BUT... (2) Instead of decoding here, it'd be better to leave output as bytes. There will be processes that do not emit valid encoded data.

FYI - modern typing does not need these two. use dict[] and list[] directly. (but read below, we aren't ready to accept non-trivial annotations in test.support)

This isn't a valid annotation. Use -> dict[str, list[StraceEvent]]... But it'd better to just not annotate this file at all. Mention what gets returned in doc strings instead.

BUT This is a good example of why we generally do not have type annotations anywhere within CPython. We are not setup to run multiple type checkers over the stdlib to validate them. You can see what little is configured in .github/workflows/mypy.yml - but do not try to use this PR to enable it for test.support. it isn't ready.

PEP 649 being implemented is the only reason this was not a Python syntax error.

why ._raw_events rather than .stderr?

My theory was trying to encourage people to use the parsed results / .events for their test cases. It's useful to have the underlying un-parsed strace output (ex. vfork anywhere in args or function names should pass) but cases like #121315 mainly just care "these calls don't happen in this case", and for mine the overall set of calls are roughly what is expected.

nit: i would just stick a functools.cache on this

Updated to functools.cache, definitely simplified a bit. Initially had been worried about adding a lot more dependencies to the I/O tests, but also realized when things like read are broken, interpreter build breaks (can't load modules, etc.)