GeezerVison is showing github.com — Document `PyCF_ALLOW_INCOMPLETE_INPUT` · Issue #142372 · python/cpython
Document PyCF_ALLOW_INCOMPLETE_INPUT #142372DescriptionOriginal title: Should PyCF_ALLOW_INCOMPLETE_INPUT remain undocumented?
Since _IncompleteInputError is undocumented on purpose -- according to GH-119521 -- should then PyCF_ALLOW_INCOMPLETE_INPUT remain an undocumented macro in the C API as well? I'm coming from GH-141004.
CC @pablogsal @JelleZijlstra @encukou
CC @ZeroIntensity mentorship docs topic-C-API Activityjohnslavik commented on Dec 7, 2025 Mentioning the private exception would be essential to the documentation of this macro... unless I just say it's a "private exception handled by the insiders 🕵 Who knows, knows" ZeroIntensity commented on Dec 7, 2025 Personally, I'm not a fan of pretending that something is private by not documenting it. People love to read CPython's source to find what kind of things we're hiding from them (C API users especially), so it would not surprise me if PyCF_ALLOW_INCOMPLETE_INPUT is already in use somewhere. If we don't want people to use it, we should go through the formal deprecation process. johnslavik commented on Dec 7, 2025
it would not surprise me if PyCF_ALLOW_INCOMPLETE_INPUT is already in use somewhere
It is. pablogsal commented on Dec 7, 2025
Since _IncompleteInputError is undocumented on purpose -- according to GH-119521 -- should then PyCF_ALLOW_INCOMPLETE_INPUT remain an undocumented macro in the C API as well? I'm coming from GH-141004.
Yes, it can change at any point and is an implementation detail for some internal modules. So either we explicitly state that or we leave it undocumented.
If we don't want people to use it, we should go through the formal deprecation process.
I disagree. If is not documented it's not supported (obviously reality is not that easy when a lot of critical projects decide to use it anyway). This is an implementation detail and always have been and we can remove it or change its semantics. That's how it was designed and it must be kept that way. Reacted by Bartosz Sławecki ZeroIntensity commented on Dec 7, 2025
This is an implementation detail and always have been and we can remove it or change its semantics. That's how it was designed and it must be kept that way.
Ah, I was unclear in my first message, sorry!
I'm not asking for this to be "supported". I just think we should add something to the docs that says "Hey, here's this cool thing, but don't you dare use it, use this other thing instead". I agree that this should stay an implementation detail. johnslavik commented on Dec 7, 2025
"Hey, here's this cool thing, but don't you dare use it, use this other thing instead".
I think that the convention of not documenting unsupported things is just to (1) save unnecessary maintenance elsewhere than in the code itself and (2) not present the reader with something that for sure is irrelevant and unactionable to them.
I see why it makes sense to document something only if we intend people to use it at some point in time. There for sure are private parts of the standard library which are undocumented, even though they have been around for a very long time and some critical projects do rely on it, e.g. typing._eval_type.
We do not intend anybody to use PyCF_ALLOW_INCOMPLETE_INPUT, so if we ever remove it in the code, the documentation won't have to be additionally maintained to reflect that internal change, which was never really a relevant thing any C API user. johnslavik commented on Dec 7, 2025 I guess that the bad part of this is that there is no obvious clue in the name of PyCF_ALLOW_INCOMPLETE_INPUT that it should not (or even must not) be used, such as in _IncompleteInputError. _IncompleteInputError is "private" by convention. Perhaps a different naming convention would be more helpful in the future. ZeroIntensity commented on Dec 7, 2025
I think that the convention of not documenting unsupported things is just to (1) save unnecessary maintenance elsewhere than in the code itself and (2) not present the reader with something that for sure is irrelevant and unactionable to them.
It's usually more complicated than that. Again, people love using our internal APIs, especially when they don't realize they're internal. The documentation would primarily be for existing codebases that found this in our source code a while ago and began using it. Through the docs, we can ask them not to use it and tell them what to use instead. If we don't do anything, they'll keep using it and complain to us when we break it someday.
Perhaps a different naming convention would be more helpful in the future.
Yeah, it should have definitely been _PyCF. I added a CI job preventing this from happening in the future. johnslavik commented on Dec 8, 2025 Okay, I think that the general conclusion is that we should not document this at the moment. I've got the guidance I needed -- thank y'all for the quick and thoughtful responses!
Please reopen if you think that this needs further discussion. ZeroIntensity commented on Dec 8, 2025 I think this needs more discussion. I am strongly -1 on leaving C APIs undocumented to try to hide them; for APIs without a _Py prefix, omitting documentation is not enough to prevent people from using them.
I'd be fine with taking an approach similar to #141634, where the flag is documented as "do not use this, use this other thing". That said, I don't see a great alternative for this flag, so if this is supposed to be private, what should people use instead? Reacted by Bartosz Sławecki pablogsal commented on Dec 8, 2025
I think this needs more discussion. I am strongly -1 on leaving C APIs undocumented to try to hide them; for APIs without a _Py prefix, omitting documentation is not enough to prevent people from using them.
I am happy if you change the name to have a _ prefix name if that works for everyone.
what should people use instead?
Nothing, is not supported functionality. We know is as generic as we need but there is no testing outside our own usage and is not envisioned to be generic for users. ZeroIntensity commented on Dec 8, 2025 Ok, that sounds great to me. We need to deprecate the old name before adding a _ prefix, but I'm okay with leaving it undocumented if it's (hard) deprecated. Is that something you'd like to work on, @johnslavik? johnslavik commented on Dec 8, 2025
Is that something you'd like to work on, @johnslavik?
Sure! changed the title [-]Should `PyCF_ALLOW_INCOMPLETE_INPUT` remain undocumented?[/-] [+]Deprecate `PyCF_ALLOW_INCOMPLETE_INPUT`[/+] on Jan 17, 2026changed the title [-]Deprecate `PyCF_ALLOW_INCOMPLETE_INPUT`[/-] [+]Document `PyCF_ALLOW_INCOMPLETE_INPUT`[/+] on Jan 17, 2026ZeroIntensity commented on Jul 18, 2026 Discussed with Pablo at the sprint. He agreed to document these internal flags on the condition that LLMs can see they are internal and not for general use. Let's move future discussion over to #153958. Reacted by Bartosz Sławecki
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|