GeezerVison is showing github.com — Support python -Xcpu_count=<n> feature for container environment. · Issue #109595 · python/cpython
Support python -Xcpu_count=<n> feature for container environment. #109595DescriptionFeature or enhancement
As #80235, there are requests for isolating CPU count in k8s or container environment, and this is a very important feature these days. (Practically my corp, a lot of workloads are running under container environments, and controlling CPU count is very important to resolve busy neighborhood issues)
There were a lot of discussions, and following the cgroup spec requires a lot of complexity and performance issues (due to fallback).
JDK 21 chooses not to depend on CPU Shares to compute active processor count and they choose to use -XX:ActiveProcessorCount=<n>.
see: https://bugs.openjdk.org/browse/JDK-8281571
I think that this strategy will be worth using from the CPython side too.
So if the user executes the python with -Xcpu_count=3option os.cpu_count will return 3, instead of the actual CPU count that was calculated from os.cpu_count.
cc @vstinner @indygreg
Linked PRs
Reacted by LYK and Ed Morley Activityvstinner commented on Sep 20, 2023 I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account. But we should provide the two values: taking affinity in account, and "total" number of CPUs. It might be tricky to change the default :-( corona10 commented on Sep 20, 2023
It might be tricky to change the default :-(
Providing -Xcpu_count will not touch any default behavior until the user explicitly set the value. So it will be the expected side-effect from the user side.
I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account
We need to understand the actual user's usecase. AFAIK k8s users or container users never use something like taskset to isolate CPU resources. They just write a Docker file and set the resources to k8s pods.
Even if we update os.cpu_count to reflect sched_getaffinity, it will not be directly helpful to actual container people.
I think that that's why JDK still provides -XX:ActiveProcessorCount=<n>.
Until we do not provide a practical way to limit CPU resources, there is no way to reflect os.cpu_count to follow their limitation.
And as you know, container application is veeeery important use case in modern applications. vstinner commented on Sep 21, 2023 os.cpu_count() is used in other use cases than containers, where CPU affinity is used.
If we modify cpu_count() to take affinity in account and/or if your -X option is implemented, we should add an option to get "total number of CPUs". indygreg commented on Sep 21, 2023
AFAIK k8s users or container users never use something like taskset to isolate CPU resources.
I have used taskset in Kubernetes. It is useful to pin processes to specific CPU cores without using something more advanced/modern like the CPU Manager feature.
One of the things I used it for was testing the behavior of a JVM when passing -XX:ActiveProcessorCount= to a different value than the number of CPUs being allowed to schedule on.
There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.) Reacted by Donghee Na and LYK vstinner commented on Sep 21, 2023 I created issue #109649: os.cpu_count(): add "affinity" parameter to get the number of "usable" CPUs.
UPDATE: I renamed the usable parameter to affinity. corona10 commented on Sep 21, 2023
There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)
Yeah, this is exactly what I want to say :) Thank you for emphasizing! corona10 commented on Sep 22, 2023 The reason I don't support cgroup directly is the following reasons.
docker run ... --cpu-shares=512 java .... ==> os::active_processor_count() = 1
docker run ... --cpu-shares=1024 java ... ==> os::active_processor_count() = 32 (total CPUs on this system)
docker run ... --cpu-shares=2048 java ... ==> os::active_processor_count() = 2
- It will increase overhead itself to os.cpu_count itself, if we decide to parse cgroup, we need to parse cgroup anyway by default, AFAIK there is no way to avoid this path if we decide to support cgroup as seamless approach to what people really want.
- Supporting cgroupv2 is much harder than cgroupv1, it will increase the maintenance cost for cpython codebase, and we need to follow up new specifications if cgroupv3 is released.
- Let's not follow what JDK already tried and failed: https://bugs.openjdk.org/browse/JDK-8281571
- Adding a new flag to os.cpu_count() will not propagate to all libraries that the program depends on but this solution will do.
indygreg commented on Sep 22, 2023 Is there an appetite for allowing non-integer / special values to -Xcpu_count= that define the mechanism to derive the CPU count? e.g.
-Xcpu_count=system-processor-count would use the equivalent of os.cpu_count() today.-Xcpu_count=cgroups-shares-1024 would read from cgroups cpu.shares and divide by 1024.-Xcpu_count=sched-affinity would compute CPU count from scheduler affinity.
Obviously we would need to bikeshed the names and behavior bit. But I see this as a potential elegant solution that not only allows forceful count overrides but also allows dynamic derivation using well-defined semantics. It also gives CPython more flexibility to introduce new variants/behavior without breaking backwards compatibility. It seemingly placates all parties. vstinner commented on Sep 22, 2023
-Xcpu_count=sched-affinity would compute CPU count from scheduler affinity.
As the author of PR #109649, I'm interested by this mode 😁 vstinner commented on Sep 22, 2023
It seemingly placates all parties.
It seems like there is no good default behavior fitting all use cases. Giving more choices should please more people. vstinner commented on Sep 22, 2023 Since you added PYTHONCPUCOUNT env var, the effect of the env var is wider than a single process, child processes are affected as well.
Can you please add a -X cpu_count value to ignore PYTHONCPUCOUNT: in short, get Python 3.12 behavior (total number of CPUs) when it is really what I want? Maybe it can just be:-X cpu_count=default.
See my example for a concrete use case: #109652 (comment)
In the UTF-8 mode, I did something similar: PYTHONUTF8=1 python -X utf8=0 ignores PYTHONUTF8 env var and ensures that the UTF-8 Mode is not enabled to this command. gpshead commented on Sep 29, 2023
I propose to:
- Add os.process_cpu_count(): PR gh-109649: Add os.process_cpu_count() function #109907
- Add
PYTHONCPUCOUNT=value env var and -X cpu_count=value which overrides os.cpu_count() and os.process_cpu_count() with a number.
- For use case (B), display the number of system CPUS, I would like to support
-X cpu_count=default (affect current process) which overrides PYTHONCPUCOUNT (system wide).
I like these proposals, they are along the lines of what we can actually accomplish and provide clear concrete results. We're exposing raw os information and allowing the user to override that when they deem appropriate for practical purposes.
Answering deeper questions such as things pertaining to efficiency vs performance, throughput, latency, Linux cgroup shares, other container configs, cpu hotplug, or what VM cores actually are vs what an underlying hypervisor may be configured to actually schedule "core" processes on... are IMNSHO better left to continually evolving PyPI libraries. Because those are pretty fluid concepts.
For example, for practical reasons, software may choose to adopt higher level concept libraries such as https://pypi.org/project/threadpoolctl/ which @mdboom mentioned on Discord. Because application processes and libraries are often not isolated and work best if they coordinate which fractions of available compute resources they each use with the other cross-language libraries and processes all being used at once for their common goal. A raw numbers of available logical cores alone can't accomplish that.
I like most of your "Later, we would consider" proposals (as followup work) - except for the cgroups one: Per the above, I view that as too ill-defined of a concept for us to dictate any mapping of that to "cores" as a stdlib API. vstinner commented on Sep 29, 2023
I like most of your "Later, we would consider" proposals (as followup work) - except for the cgroups one: Per the above, I view that as too ill-defined of a concept for us to dictate any mapping of that to "cores" as a stdlib API.
I have mixed feeling about reading cgroups in Python stdlib. What I care the most here is to make sure that with the proposed designed, tomorrow, we can still change our mind, and read cgroups.
I'm saying that because my first proposition was to add an affinity parameter to os.cpu_count(). It's simple. It's easy. Why not? Well. If tomorrow, we read cgroups, what does it mean? We have to add a second cgroups parameters? Now what? The correct would look like:
kwargs = {}
if sys.version_info >= (3, 13):
kwargs['affinity'] = True
if sys.version_info >= (3, 14):
kwargs['cgroups'] = True
cpu_count = os.cpu_count(**kwargs)Hu! That's not convenient.
I prefer a separated function which has no parameter: process_cpu_count(). It gets CPU affinity if available, or not. If tomorrow we consider reading cgroups, again, it would still fit into "return the number of CPU usable by the current process". corona10 commented on Sep 30, 2023 Yeah, I like the proposal #109595 (comment), it will makes better situation for exceptional environment of Python users :) corona10 commented on Oct 10, 2023 FYI, I am preparing -Xcpu_count=process option with the separate PR. Reacted by Erlend E. Aasland vstinner commented on Oct 10, 2023 Would you mind to create a separated PR for -Xcpu_count=process? vstinner commented on Oct 10, 2023 Follow-up issue: #110649: Add -Xcpu_count=process cmdline mode to redirect os.cpu_count as os.process_cpu_count. blackliner commented on May 23, 2024 did the PYTHONCPUCOUNT env var survive to the main branch? If not, how can I control this behavior from the environment, without touching existing code? (Hoping os.cpu_count() will return the reduced number of cores) vstinner commented on May 23, 2024 Reacted by Florian Berchtold
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|