GeezerVison is showing github.com — Argument Clinic's `[python input]` blocks do not support empty lines in comments · Issue #128152 · python/cpython
Argument Clinic's [python input] blocks do not support empty lines in comments #128152DescriptionBug report
Bug description:
The following is not a valid Python input clinic but should be:
/*[python input]
# Line 1
#
# Line 3
[python start generated code]*/ This results in an assertion error:
$ python ./Tools/clinic/clinic.py --force --make --exclude Lib/test/clinic.test.c --srcdir .
Traceback (most recent call last):
File "/lib/python/cpython/./Tools/clinic/clinic.py", line 11, in <module>
main()
File "/lib/python/cpython/Tools/clinic/libclinic/cli.py", line 226, in main
run_clinic(parser, args)
File "/lib/python/cpython/Tools/clinic/libclinic/cli.py", line 205, in run_clinic
parse_file(path,
File "/lib/python/cpython/Tools/clinic/libclinic/cli.py", line 84, in parse_file
cooked = clinic.parse(raw)
^^^^^^^^^^^^^^^^^
File "/lib/python/cpython/Tools/clinic/libclinic/app.py", line 186, in parse
for block in self.block_parser:
^^^^^^^^^^^^^^^^^
File "/lib/python/cpython/Tools/clinic/libclinic/block_parser.py", line 132, in __next__
return_value = self.parse_clinic_block(self.dsl_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lib/python/cpython/Tools/clinic/libclinic/block_parser.py", line 194, in parse_clinic_block
line = self._line()
^^^^^^^^^^^^
File "/lib/python/cpython/Tools/clinic/libclinic/block_parser.py", line 155, in _line
self.language.parse_line(line)
File "/lib/python/cpython/Tools/clinic/libclinic/clanguage.py", line 70, in parse_line
self.cpp.writeline(line)
File "/lib/python/cpython/Tools/clinic/libclinic/cpp.py", line 139, in writeline
assert line
^^^^
AssertionError
make: *** [Makefile:1055: clinic] Error 1CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs
Activitychanged the title [-]Argument Clinic's `[python input]` blocks do not supported nested C comments[/-] [+]Argument Clinic's `[python input]` blocks do not support empty lines in comments[/+] on Dec 21, 2024picnixz commented on Dec 21, 2024 I've had a quick look at the failure and this happens before we've detected the [python input] block. Similarly, #128153 faces the same issue, namely Clinic assumes that we're still processing a C file and doesn't check for the desired DSL.
@erlend-aasland I don't really know how to fix this and it's not really an issue since I can just write some fake characters to simulate an empty line (e.g., # ::) but I'd like to know if first detecting the DSL before attempting to parse it would be a good alternative or if the current design of AC is just too intricate for this feature. Reacted by Erlend E. Aasland StanFromIreland commented on Dec 21, 2024 I've opened a PR for this issue. I think the problem was that empty lines were breaking the logic so if they don't reach it there should be no more error. picnixz commented on Dec 21, 2024 As I said on the PR, I'm not sure this is the best approach. The "empty lines" that are detected at this stage are actually lines of the form <space>#</space>, so it's not entirely clear whether they should be ignored at this point. The reason is that we first parse a file before deciding whether this new line is part of a python input block. If this is the case, we should switch to another parsing method (for instance, I haven't actually checked what happens if I write #if defined(...) inside the Python input clinic block). erlend-aasland commented on Jan 3, 2025 This has nothing to do with [python input]; you'll get the same error with [clinic input]:
[clinic input] repro
$ cat > buggy.c
/*[clinic input]
# comment
#
[clinic start generated code]*/
$ ./Tools/clinic/clinic.py buggy.c
Traceback (most recent call last):
File "/Users/erlend.aasland/src/python/main/./Tools/clinic/clinic.py", line 11, in <module>
main()
~~~~^^
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/cli.py", line 226, in main
run_clinic(parser, args)
~~~~~~~~~~^^^^^^^^^^^^^^
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/cli.py", line 218, in run_clinic
parse_file(filename, output=ns.output,
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
verify=not ns.force, limited_capi=ns.limited_capi)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/cli.py", line 84, in parse_file
cooked = clinic.parse(raw)
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/app.py", line 186, in parse
for block in self.block_parser:
^^^^^^^^^^^^^^^^^
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/block_parser.py", line 132, in __next__
return_value = self.parse_clinic_block(self.dsl_name)
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/block_parser.py", line 194, in parse_clinic_block
line = self._line()
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/block_parser.py", line 155, in _line
self.language.parse_line(line)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/clanguage.py", line 70, in parse_line
self.cpp.writeline(line)
~~~~~~~~~~~~~~~~~~^^^^^^
File "/Users/erlend.aasland/src/python/main/Tools/clinic/libclinic/cpp.py", line 139, in writeline
assert line
^^^^
AssertionError picnixz commented on Jan 3, 2025 Yes, that's why I'm asking if this is possible to pre-determine the DSL before parsing the C file (a two-pass parsing) so that python input blocks are parsed using the Python parser (namely, if we're in a python input block, we skip the special handling of # which assumes that we're parsing pre-processor guards). Because the error happens before this (I'm not sure it's possible) erlend-aasland commented on Jan 3, 2025 The problem seems to be in cpp.py's incorrect parsing of comments: it tries to parse directives inside comments. The fix is straightforward: ignore CPP directives inside C comments. Reacted by Bénédikt Tran and Stan Ulbrych added a commit that references this issue on Jan 4, 2025gh-128152: Argument Clinic: ignore pre-processor directives inside C …
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|