GeezerVison is showing github.com — _Py_wfopen no longer exported · Issue #127350 · python/cpython
_Py_wfopen no longer exported #127350DescriptionBug report
Bug description:
_Py_wfopen in no longer exported since 3.13.
I'm using embed version and I can not use fopen or _wfopen.
Please reconsider decision to remove _Py_wfopen since it is only way to open file when used in embed mode and fopen/_wfopen is not available. Without it PyRun_FileExFlags is useless to me and my application can no longer call external scripts.
CPython versions tested on:
3.13
Operating systems tested on:
Windows
Linked PRs
Activityskirpichev commented on Nov 28, 2024 vstinner commented on Nov 28, 2024 Hi,
I'm using embed version and I can not use fopen or _wfopen.
Why can't you use fopen() or _wfopen()? PlanetCNC commented on Nov 28, 2024 In my case PyRun_FileExFlags crashes if I use FILE* from fopen() or _wfopen().
I suspect this is because different toolset versions are used to compile python313.dll and our software. That is why we don't mix them.
With _Py_wfopen everything works flawlessly.
We integrated Python into our CNC machine control software and it was working great for lots of users until now. Here is an example which is no longer possible to run:
https://www.youtube.com/watch?v=MruGmi1JoCc
Here is our module documentation:
https://cnc.zone/sdk/python/python vstinner commented on Nov 28, 2024
In my case PyRun_FileExFlags crashes if I use FILE* from fopen() or _wfopen(). I suspect this is because different toolset versions are used to compile python313.dll and our software. That is why we don't mix them.
The FILE* is not NULL? That's strange.
You may give a try to _Py_fopen_obj() function which is the last "open" function which returns a FILE* in the Python C API. It's private, you're not supposed to use it. PlanetCNC commented on Nov 28, 2024 No, it is not NULL.
I tried _Py_fopen_obj() but I was unable to make it work. It is "private" anyway so I expect to be removed in next version.
How about making new, nonprivate function with same implementation as _Py_wfopen ? PlanetCNC commented on Nov 28, 2024 I managed to make _Py_fopen_obj() work. Please do not remove my last option to open file. ZeroIntensity commented on Nov 28, 2024
In my case PyRun_FileExFlags crashes if I use FILE* from fopen() or _wfopen().
That sounds like a bug, could you file a new issue with a reproducer?
But overall, I think we just need a public API if this is the only way to do something. You shouldn't ever rely on anything prefixed with _Py as a user. vstinner commented on Nov 29, 2024 Yeah, you should be able to call fopen() and pass the result to a Python API which expect a FILE* file. PlanetCNC commented on Nov 29, 2024 asvetlov commented on Nov 29, 2024 I read the article as:
- 10 years ago you should be careful and use the same Visual Studio version and the same multithreaded (
/MT vs /MD) mode as it was used for compiling Python itself. It was always a good advice. Now CRT is cross-compatible which helps people a lot; good to know.
- Release the memory by the same dynamic library that was used for allocation. For example, call
fclose() in your dll for files opened with fopen(). Also it sounds reasonable.
ZeroIntensity commented on Nov 29, 2024 As far as I can tell, _Py_wfopen is just a wrapper over fopen and _wfopen anyway. It shouldn't matter where the FILE * comes from. PlanetCNC commented on Nov 29, 2024
As far as I can tell, _Py_wfopen is just a wrapper over fopen and _wfopen anyway. It shouldn't matter where the FILE * comes from.
This is not true. Microsoft clearly states that passing passing CRT objects across DLL boundaries causes potential errors.
I spend a lot of time trying to "fix" this issue and is simply not possible. And I'm not a novice - I have more than 30 years experience in programming very complex stuff that runs on all popular operating systems. It is not that I don't know how to correctly use 'fopen' and '_wfopen'.
If there is such a problem keeping '_Py_wfopen' then functions that accept FILE* should also expect filename as PyObject and then internally call _Py_fopen_obj(). This change should be fairly easy to make. ZeroIntensity commented on Nov 29, 2024 I'm not doubting your expertise, but note that the PyRun APIs have existed for years; if they weren't usable without the private API on Windows, I would hope that we would have noticed by now. Could this be a matter of some incorrect linker flags?
cc @zooba PlanetCNC commented on Dec 3, 2024 All PyRun_... functions that accept a FILE* parameter already include a corresponding filename parameter. Currently, if the FILE* is NULL, these functions crash. A potential solution could be to modify the implementation so that when FILE* is NULL, the filename is used internally to call _Py_fopen or _Py_wfopen. This approach would resolve the issue without requiring any changes to the existing API definitions. Reacted by Petr Viktorin and Steve Dower zooba commented on Dec 3, 2024
A potential solution could be to modify the implementation so that when FILE* is NULL, the filename is used internally to call _Py_fopen or _Py_wfopen.
Sounds good to me.
Anyone want to open a PR? vstinner commented on Dec 11, 2024 The C API has many functions which expect a FILE*, not only PyRun_FileExFlags():
- Py_UniversalNewlineFgets()
- Py_FdIsInteractive()
- PyObject_Print()
- PyMarshal_ReadLastObjectFromFile()
- PyMarshal_ReadLongFromFile()
- PyMarshal_ReadObjectFromFile()
- PyMarshal_ReadShortFromFile()
- PyMarshal_WriteLongToFile()
- PyMarshal_WriteObjectToFile()
- PyRun_AnyFileExFlags()
- PyRun_SimpleFileExFlags()
- PyRun_InteractiveOneFlags()
- PyRun_InteractiveOneObject()
- PyRun_InteractiveLoopFlags()
- PyRun_AnyFile()
- PyRun_AnyFileEx()
- PyRun_AnyFileFlags()
- PyRun_File()
- PyRun_FileEx()
- PyRun_FileFlags()
- PyRun_InteractiveLoop()
- PyRun_InteractiveOne()
- PyRun_SimpleFile()
- PyRun_SimpleFileEx()
- PyRun_SimpleString()
So I propose adding Py_fopen() instead: I wrote PR gh-127821 for that. vstinner commented on Dec 12, 2024 @PlanetCNC:
I managed to make _Py_fopen_obj() work. Please do not remove my last option to open file.
So you don't need a "Py_fclose()" function, right? You just call fclose() directly? vstinner commented on Jan 6, 2025 I added Py_fopen() and Py_fclose() functions to Python 3.14.
I added these functions to pythoncapi-compat for Python 3.13 and older: python/pythoncapi-compat#127
Thanks @PlanetCNC for your bug report. I suggest you to use pythoncapi-compat, Py_fopen() and Py_fclose().
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|