Description

mattip

opened on Aug 14, 2023

Issue body actions

Bug report

Testing out 3.12.0rc1 on NumPy, we came across numpy/numpy#24399. I have distilled this down to a reproducer that fails on 3.12.0rc1, but passes on previous versions.

Checklist

• I am confident this is a bug in CPython, not a bug in a third-party project
• I have searched the CPython issue tracker, and am confident this bug has not been reported before

A clear and concise description of the bug

from ctypes import Structure, pointer, c_int16, cast, POINTER

class Struct(Structure):
    _fields_ = [('a', c_int16)]

Struct3 = 3 * Struct

c_array = (2 * Struct3)(
    Struct3(Struct(a=1), Struct(a=2), Struct(a=3)),
    Struct3(Struct(a=4), Struct(a=5), Struct(a=6))
)

assert c_array[0][0].a == 1

p_obj = cast(pointer(c_array), POINTER(pointer(c_array)._type_))
obj = p_obj.contents
# Works on 3.12, obj is 2d
assert obj[0][0].a == 1

p_obj = cast(pointer(c_array[0]), POINTER(pointer(c_array)._type_))
obj = p_obj.contents
# Fails on 3.12.0rc1, but works on 3.11 and earlier: obj is not 2d
assert obj[0][0].a == 1

Your environment

• CPython versions tested on: 3.12rc1
• Operating system and architecture: ubuntu 22.04

Edit: 3.12rc1 -> 3.12.0rc1

Linked PRs

• gh-107940: Only reuse the pointer object when it matches the _type_ of the container #108222

Activity

1. mattip

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Aug 14, 2023
2. Eclips4

   added

   topic-ctypes

   on Aug 14, 2023
3. mattip

   changed the title [-]incompatible behaviour between 3.12rc1 and previous versions in ctypes[/-] [+]incompatible behaviour between 3.12.0rc1 and previous versions in ctypes[/+] on Aug 14, 2023
4. mattip commented on Aug 14, 2023

   

   mattip

   on Aug 14, 2023

   ContributorAuthor

   More actions

   I see #107131 touched Pointer_get_contents to solve #46376. Perhaps that PR did not take cast into account when checking the equivalency of pointers?

   Reacted by Łukasz Langa, Erlend E. Aasland, Eric Wieser and Konstantin
5. erlend-aasland

   added

   release-blocker

   on Aug 14, 2023
6. ezio-melotti

   added this to Release and Deferred blockers 🚫on Aug 14, 2023
7. github-project-automation

   moved this to Todo in Release and Deferred blockers 🚫on Aug 14, 2023
8. kumaraditya303

   assigned

   ambv

   on Aug 20, 2023
9. kumaraditya303

   added

   3.12only security fixesonly security fixes

   3.13only security fixesonly security fixes

   on Aug 20, 2023
10. ambv

    added 3 commits that reference this issue on Aug 21, 2023

    Only reuse the pointer object when it matches the `_type_` of the con…

    This issue will close once commit 3136d4f is merged into the 'main' branch.

    3136d4f

    Only reuse the pointer object when it matches the `_type_` of the con…

    This issue will close once commit e3c5b02 is merged into the 'main' branch.

    e3c5b02

    [3.11] Revert "gh-46376: Return existing pointer when possible in cty…

    This issue will close once commit c3d129f is merged into the 'main' branch.

    c3d129f
11. vstinner commented on Aug 30, 2023

    

    vstinner

    on Aug 30, 2023

    Member

    More actions

    In the 3.12 branch, I confirm that the commit 54aaaad introduced the regression: issue #46376 of PR #107487 (backport of PR #107131).
12. vstinner commented on Aug 30, 2023

    

    vstinner

    on Aug 30, 2023

    Member

    More actions

    It seems like the PCP project is affected by this regression, see the Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=2231751
13. vstinner commented on Aug 30, 2023

    

    vstinner

    on Aug 30, 2023

    Member

    More actions

    I proposed a revert until the regression is understood and fixed: #108688
14. ambv

    added a commit that references this issue on Sep 4, 2023

    Only reuse the pointer object when it matches the `_type_` of the con…

    This issue will close once commit 16d39d2 is merged into the 'main' branch.

    16d39d2
15. ambv commented on Sep 4, 2023

    

    ambv

    on Sep 4, 2023

    Contributor

    More actions

    Revert for main merged. Revert for 3.12 up: #108864
16. ambv commented on Sep 4, 2023

    

    ambv

    on Sep 4, 2023

    Contributor

    More actions

    3.12 revert landed as well.
17. ambv commented on Sep 4, 2023

    

    ambv

    on Sep 4, 2023

    Contributor

    More actions

    3.11 was reverted before 3.11.5 in GH-108412.
18. ambv commented on Sep 4, 2023

    

    ambv

    on Sep 4, 2023

    Contributor

    More actions

    We'll close this and continue search for a good solution in the original issue.
19. ambv

    closed this as completedon Sep 4, 2023
20. github-project-automation

    moved this from Todo to Done in Release and Deferred blockers 🚫on Sep 4, 2023
21. vstinner commented on Sep 4, 2023

    

    vstinner

    on Sep 4, 2023

    Member

    More actions

    We'll close this and continue search for a good solution in the original issue.

    Thanks. IMO it's a reasonable practical solution to this complicated problem.

    If someone comes up with a new fix, please ping me, I will try to review it! But I'm not available to investigate the issue and attempt writing a better ctypes fix.

Issue actions