Description

vstinner

opened on Sep 21, 2023

Issue body actions

Feature or enhancement

os.cpu_count() returns the total number of CPUs that the current machine has. On Unix, sched_setaffinity() can reduce the number of CPU "usable" by a process. Currently, os.cpu_count() docstring contains a recipe how to get the number of usable CPUs:

>>> import os; help(os.cpu_count)
cpu_count()
    Return the number of CPUs in the system; return None if indeterminable.
    
    This number is not equivalent to the number of CPUs the current process can
    use.  The number of usable CPUs can be obtained with
    ``len(os.sched_getaffinity(0))``

I propose to add an affinity optional keyword-only parameter to os.cpu_count() which would return len(os.sched_getaffinity(0)).st

UPDATE: Rename usable parameter to affinity.

Linked PRs

• gh-109649: Add affinity parameter to os.cpu_count() #109652
• gh-109649: Add os.process_cpu_count() function #109907
• gh-109649: Use os.process_cpu_count() #110165
• [3.12] gh-109649: Enhance os.cpu_count() documentation #110169
• [3.11] [3.12] gh-109649: Enhance os.cpu_count() documentation (GH-110169) #110226
• gh-109649: Fix test_os.test_process_cpu_count_affinity() #111689

Activity

1. vstinner

   added

   type-featureA feature request or enhancementA feature request or enhancement

   on Sep 21, 2023
2. vstinner commented on Sep 21, 2023

   

   vstinner

   on Sep 21, 2023

   MemberAuthor

   More actions

   os.cpu_count() issues (open and closed):

   • issue add os.cpu_count() #62114: add os.cpu_count()
   • issue cpu_count() unreliable on Windows #67226: cpu_count() unreliable on Windows
   • issue os.cpu_count is problematic on sparc/solaris #73451: os.cpu_count is problematic on sparc/solaris
   • issue os.cpu_count() returns wrong number of processors on system with > 64 logical processors #74766: Windows: os.cpu_count() returns wrong number of processors on system with > 64 logical processors
   • issue os.cpu_count() should return a count assigned to a container or that the process is restricted to #80235: On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)
   • issue Support python -Xcpu_count=<n> feature for container environment. #109595: Support python -Xcpu_count= feature for container environment

   Previously, the function was known as multiprocessing.cpu_count():

   • issue multiprocessing.cpu_count() should use hw.availcpu on Mac OS X #61646: multiprocessing.cpu_count() should use hw.availcpu on Mac OS X
3. vstinner

   added 3 commits that reference this issue on Sep 21, 2023

   pythongh-109649: Add usable parameter to os.cpu_count()

   477d57b

   pythongh-109649: Add usable parameter to os.cpu_count()

   0f769ec

   pythongh-109649: Add usable parameter to os.cpu_count()

   df887a8
4. vstinner commented on Sep 21, 2023

   

   vstinner

   on Sep 21, 2023

   MemberAuthor

   More actions

   On Windows, maybe it can be implemented with GetProcessGroupAffinity(), GetProcessAffinityMask(), GetActiveProcessorGroupCount() and GetActiveProcessorCount().

   GetProcessGroupAffinity():
   https://learn.microsoft.com/en-us/windows/win32/api/processtopologyapi/nf-processtopologyapi-getprocessgroupaffinity
5. vstinner commented on Sep 21, 2023

   

   vstinner

   on Sep 21, 2023

   MemberAuthor

   More actions

   On Windows, a CPU affinity can be run with start /affinity command. Example to only run Python on the firstCPU:

   start /affinity 0x0000000000000001 python.exe
   
6. vstinner

   added a commit that references this issue on Sep 21, 2023

   pythongh-109649: Add affinity parameter to os.cpu_count()

   6872c18
7. vstinner

   changed the title [-]os.cpu_count(): add "usable" parameter to get the number of "usable" CPUs[/-] [+]os.cpu_count(): add "affinity" parameter to get the number of logical CPUs usable by the current process[/+] on Sep 21, 2023
8. vstinner

   added 6 commits that reference this issue on Sep 21, 2023

   pythongh-109649: Add affinity parameter to os.cpu_count()

   38ba1c3

   pythongh-109649: Add affinity parameter to os.cpu_count()

   ae114ac

   pythongh-109649: Add affinity parameter to os.cpu_count()

   2762144

   pythongh-109649: Add os.process_cpu_count() function

   48c68d2

   pythongh-109649: Add os.process_cpu_count() function

   e3554fa

   pythongh-109649: Add os.process_cpu_count() function

   69c8bf4
9. 5 remaining items
10. vstinner

    added 6 commits that reference this issue on Sep 30, 2023

    pythongh-109649: Use os.process_cpu_count()

    5a6d043

    pythongh-109649: Use os.process_cpu_count()

    c1c59bf

    pythongh-109649: Use os.process_cpu_count()

    6b15e90

    pythongh-109649: Enhance os.cpu_count() documentation

    208ef27

    pythongh-109649: Enhance os.cpu_count() documentation

    b370b73

    gh-109649: Use os.process_cpu_count() (#110165)

    a46e960
11. Yhg1s

    added a commit that references this issue on Oct 2, 2023

    [3.12] gh-109649: Enhance os.cpu_count() documentation (#110169)

    5245b97
12. miss-islington

    added a commit that references this issue on Oct 2, 2023

    [3.12] pythongh-109649: Enhance os.cpu_count() documentation (pythonG…

    cfc236c
13. gpshead

    added a commit that references this issue on Oct 3, 2023

    [3.11] [3.12] gh-109649: Enhance os.cpu_count() documentation (GH-110169

    bf6843e
14. vstinner

    added 2 commits that reference this issue on Nov 3, 2023

    pythongh-109649: Fix test_os.test_process_cpu_count_affinity()

    2db7d4f

    gh-109649: Fix test_os.test_process_cpu_count_affinity() (#111689)

    244e37b
15. aisk

    added a commit that references this issue on Feb 11, 2024

    pythongh-109649: Fix test_os.test_process_cpu_count_affinity() (pytho…

    36895e1
16. Glyphack

    added 3 commits that reference this issue on Sep 2, 2024

    pythongh-109649: Add os.process_cpu_count() function (python#109907)

    7662bfc

    pythongh-109649: Use os.process_cpu_count() (python#110165)

    c3bc035

    pythongh-109649: Fix test_os.test_process_cpu_count_affinity() (pytho…

    dcd6d00

Issue actions