GeezerVison is showing github.com — Expose _PyInterpreterFrame_GetLine in the private API · Issue #96803 · python/cpython
Expose _PyInterpreterFrame_GetLine in the private API #96803Labels 3.11only security fixesonly security fixes3.12only security fixesonly security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DescriptionIn 3.11 we are using now an opaque pointer to _PyInterpreterFrame in the eval function. The problem is that for PEP 523's APIs, we changed the type of _PyFrameEvalFunction to be PyObject *(*_PyFrameEvalFunction)(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag) but there is no way to do anything with _PyInterpreterFrame because is opaque. Even if the extension defines PY_BUILD_CORE the symbols are not exposed in the executable/libpython, which means that including the headers is insufficient.
This is a problem because it really limits what users of PEP 523 can do with the frame object that they receive, even if is opaque. Most profilers using PEP 523 API do it for speed reasons and the only thing they need is to get the line number, but that is now impossible because the pointer is opaque and we don't offer any exposed APIs.
Linked PRs
Reacted by Gregory P. Smith Activitypablogsal commented on Sep 13, 2022 ericsnowcurrently commented on Sep 13, 2022 markshannon commented on Sep 14, 2022 I see your point, but I don't think that _PyInterpreterFrame_GetLine() is the right API.
I think the pair of _PyInterpreterFrame_GetCode() and _PyInterpreterFrame_GetLasti() would be better.
Getting the offset is much cheaper than computing the line number.
The line number can be computed, if needed.
Other information to be extracted from the code object, such as the filename and function name. Reacted by Pablo Galindo Salgado pablogsal commented on Sep 15, 2022 I started exposing _PyInterpreterFrame_GetCode and _PyInterpreterFrame_GetLasti but then I realized that neither _PyInterpreterFrame_GetCode and _PyInterpreterFrame_GetLasti are in 3.12 so exposing those is going to be more chaotic than just exposing _PyInterpreterFrame_GetLine. Notice that if the application defines PY_BUILD_CORE it can already access f_code and f_lasti manually from the struct.
As there can be arbitrary logic in _PyInterpreterFrame_GetLine I still think is a better idea to export _PyInterpreterFrame_GetLine. markshannon commented on Sep 15, 2022 It still think that the pair of _PyInterpreterFrame_GetCode and _PyInterpreterFrame_GetLasti is a better interface.
If you think that doing that would be too disruptive for 3.11, we can add them for 3.12, and just expose _PyInterpreterFrame_GetLine for 3.11. pablogsal commented on Sep 15, 2022
If you think that doing that would be too disruptive for 3.11, we can add them for 3.12, and just expose
In 3.12 they are already exposed if the user calls PY_BUILD_CORE because one is a macro and the other can be basically substituted by frame->f_code because the struct definition is available if PY_BUILD_CORE is defined. The problem here is having to re-implement _PyInterpreterFrame_GetLine.
Exposing _PyInterpreterFrame_GetCode and _PyInterpreterFrame_GetLasti doesn't give the user anything that cannot be achieved by just calling frame->f_code and frame->f_lasti after defining PY_BUILD_CORE markshannon commented on Sep 15, 2022 _PyInterpreterFrame_GetLine is quite simple once we have the other two functions. There is no harm in exposing it as well.
the struct definition is available if PY_BUILD_CORE is defined.
All that does is expose the existence of a field call f_code. It doesn't say anything about its meaning or when it is valid.
In general, frame->f_code cannot be substituted for _PyInterpreterFrame_GetCode() as the meaning of frame->f_code is not defined. E.g. we might make it such that f_code is only valid for some frames faster-cpython/ideas#111 pablogsal commented on Sep 15, 2022 Ok, I am convinced. I will expose _PyInterpreterFrame_GetCode and _PyInterpreterFrame_GetLasti added a commit that references this issue on May 5, 2023GH-96803: Add three C-API functions to make _PyInterpreterFrame less … erlend-aasland commented on May 5, 2023 gh-96849 added new APIs, but no documentation was added. Mentioning it here, so we don't forget before we close this issue. encukou commented on May 9, 2023 Note that even unstable API needs tests as well as documentation. Reacted by Erlend E. Aasland added a commit that references this issue on May 18, 2023GH-96803: Document and test new unstable internal frame API functions (…
MetadataMetadata3.11only security fixesonly security fixes3.12only security fixesonly security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DevelopmentNo branches or pull requestsIssue actions
|