GeezerVison is showing github.com — Remove the old parser · Issue #85111 · python/cpython
Remove the old parser #85111Description
| BPO |
40939 |
| Nosy |
@gvanrossum, @terryjreedy, @vstinner, @ned-deily, @encukou, @nnemkin, @hroncok, @lysnikolaou, @pablogsal, @miss-islington, @brandtbucher |
| PRs |
bpo-40939: Remove the old parser #20768bpo-40939: Generate keyword.py using pegen #20800[3.9] bpo-40939: Generate keyword.py using the new parser (GH-20800) #20801bpo-40939: Remove PEG parser easter egg #20802[3.9] bpo-40939: Fix test_keyword for the old parser #20814bpo-40939: Remove some extra references to PYTHONOLDPARSER #20815bpo-40939: Clean and adapt the peg_generator directory after deleting the old parser #20822bpo-40939: Remove the old parser (Part 2) #21005[3.9] bpo-40939: Deprecate the PyParser_SimpleParse* functions #21012bpo-40939: Rename PyPegen* functions to PyParser* #21016[3.9] bpo-40939: Do not emit deprecation warnings inside CPython for old parser APIs #21025[3.9] bpo-40939: Deprecate PyNode_Compile #21036bpo-40939: run autoreconf to fix configure{.ac,} disparity #21152bpo-40939: Delete remaining references to Grammar/Grammar #21624bpo-40939: Add the new grammar to the grammar specification documentation #19969[3.9] bpo-40939: Use the new grammar for the grammar specification documentation (GH-19969) #21641bpo-40939: Remove even more references to the old parser #21642bpo-41808: Add What's New 3.9 entry missing from master #22294bpo-40939: Document removal of the old parser in 3.10 whatsnew #23321bpo-40939: Restore some stable API functions incorrectly deleted #23606bpo-40939: Remove documentation for PyParser_* & add porting notes #26855[3.10] bpo-40939: Remove documentation for PyParser_* & add porting notes (GH-26855) #26898 |
| Files |
py_limited_api_example.zip: Example of using Py_CompileString with Py_LIMITED_APIpy3.txt: Imports from python3.dll |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/pablogsal'
closed_at = <Date 2020-12-02.05:17:21.232>
created_at = <Date 2020-06-10.11:25:17.640>
labels = ['interpreter-core', '3.10']
title = 'Remove the old parser'
updated_at = <Date 2021-06-28.09:36:20.843>
user = 'https://github.com/pablogsal' bugs.python.org fields:
activity = <Date 2021-06-28.09:36:20.843>
actor = 'petr.viktorin'
assignee = 'pablogsal'
closed = True
closed_date = <Date 2020-12-02.05:17:21.232>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2020-06-10.11:25:17.640>
creator = 'pablogsal'
dependencies = []
files = ['49642', '49647']
hgrepos = []
issue_num = 40939
keywords = ['patch']
message_count = 69.0
messages = ['371182', '371264', '371265', '371266', '371267', '371268', '371270', '371271', '371288', '371289', '371290', '371291', '371292', '371301', '371307', '371308', '371310', '371311', '371316', '371327', '371929', '371940', '371942', '371953', '371977', '372023', '372025', '372026', '372027', '372370', '374282', '374346', '374410', '374413', '374416', '374417', '377083', '380826', '380828', '380855', '380863', '381095', '381103', '381112', '381136', '381137', '381140', '381172', '381176', '381182', '381227', '381232', '382129', '382130', '382134', '382285', '382287', '382289', '382293', '382299', '382323', '382324', '382325', '382329', '382337', '387017', '396348', '396488', '396622']
nosy_count = 12.0
nosy_names = ['gvanrossum', 'terry.reedy', 'vstinner', 'ned.deily', 'petr.viktorin', 'nnemkin', 'Igor.Skochinsky', 'hroncok', 'lys.nikolaou', 'pablogsal', 'miss-islington', 'brandtbucher']
pr_nums = ['20768', '20800', '20801', '20802', '20814', '20815', '20822', '21005', '21012', '21016', '21025', '21036', '21152', '21624', '19969', '21641', '21642', '22294', '23321', '23606', '26855', '26898']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40939'
versions = ['Python 3.10'] Activitypablogsal commented on Jun 10, 2020 As stated in PEP-617, the old parser will be removed in Python 3.10 pablogsal commented on Jun 11, 2020 vstinner commented on Jun 11, 2020 >>> __new_parser__
File "<stdin>", line 1
__new_parser__
^
SyntaxError: You found it!"new", "ex" or "ng" are not really future proof names. Can we rename the keyword to "__peg_parser__"? lysnikolaou commented on Jun 11, 2020
Can we rename the keyword to "__peg_parser__"?
I guess we could just remove this, as soon as the old parser is out. We were only using this to differentiate between the two parsers, when we were testing enabling/disabling the old one. I could get a PR ready to be merged after #64967 is there. pablogsal commented on Jun 11, 2020
I guess we could just remove this, as soon as the old parser is out. We were only using this to differentiate between the two parsers, when we were testing enabling/disabling the old one. I could get a PR ready to be merged after #64967 is there.
I would personally would like to keep the easter egg, but I assume is better to rename it to "__peg_parser__". miss-islington commented on Jun 11, 2020 lysnikolaou commented on Jun 11, 2020
I would personally would like to keep the easter egg, but I assume is better to rename it to "__peg_parser__".
Ok then! On it. vstinner commented on Jun 11, 2020 I did'nt ask to remove the easter egg. I'm just asking to avoid the "new" name. In my experience, each time that a "new" thing happens, later we have to use "new extended", "new_v2" or worse name :-)
Oh, if the name changes, please change it in 3.9 as well.
Look at this amazing names of the 5 flavors of functions parsing a string:
PyParser_ParseString()
PyParser_ParseStringFlags()
PyParser_ParseStringFlagsFilename()
PyParser_ParseStringFlagsFilenameEx() <= public!
PyParser_ParseStringObject() Same for parsing a file:
PyParser_ParseFile()
PyParser_ParseFileFlags()
PyParser_ParseFileFlagsEx() <= public!
PyParser_ParseFileObject() Or PyRun functions:
PyRun_String()
PyRun_AnyFile()
PyRun_AnyFileEx() <= public!
PyRun_AnyFileFlags()
PyRun_SimpleString()
PyRun_SimpleFile()
PyRun_SimpleFileEx() <= public!
PyRun_InteractiveOne()
PyRun_InteractiveLoop()
PyRun_File()
PyRun_FileEx() <= public!
PyRun_FileFlags() ceval.c:
PyEval_EvalCode()
PyEval_EvalCodeEx() <= public!
_PyEval_EvalCodeWithName()
_PyEval_EvalCode() I cannot count the number of "Ex" functions that we have :-)
Py_Finalize() -> :Py_FinalizeEx() <= public!
PyErr_Print() -> PyErr_PrintEx() <= public!
PySys_SetArgv() -> PySys_SetArgvEx() <= public!
PyErr_WarnEx() <= public!
_PyBytes_FormatEx()
_PyDict_MergeEx()
_Py_DecodeLocaleEx(), _Py_EncodeLocaleEx()
struct PyMemAllocatorEx
_Py_DecodeUTF8Ex(), _Py_EncodeUTF8Ex()
etc. gvanrossum commented on Jun 11, 2020 Honestly I see no reason to keep that easter egg. Can we remove it please? pablogsal commented on Dec 2, 2020 Seems that commit 1ed83ad was deleting some functions that were not correctly covered by redirection macros. I have opened PR 23606 to restore those as they were. pablogsal commented on Dec 2, 2020 IgorSkochinsky commented on Dec 2, 2020 Thanks for the fix! I'd like to submit a test to avoid this and similar issues in future. Are there any guidelines for this? Sorry if this is a wrong place to ask. pablogsal commented on Dec 2, 2020 I am currently working on test that checks that the stable API symbols are correctly exported. Unfortunately there is no official maintained list if those symbols, so is taking a while IgorSkochinsky commented on Dec 2, 2020
I am currently working on test that checks that the stable API symbols are correctly exported.
Thank you very much! For added motivation, the 3.8.0 release was unusable thanks to bpo-37633 which was somewhat similar (also in pythonrun). For reference, I've attached a list of symbols we currently use (a few more we have to import dynamically since they're not in the stable ABI but we'd like to keep that list as short as possible). nnemkin commented on Dec 2, 2020
Unfortunately there is no official maintained list if those symbols, so is taking a while
On Windows there is PC/python3dll.c. pablogsal commented on Dec 2, 2020
On Windows there is PC/python3dll.c.
Even that is severely out of date, unfortunately. There are many functions that were removed in 3.8 and 3.7 that are still listed there. pablogsal commented on Dec 2, 2020 vstinner commented on Feb 15, 2021 encukou commented on Jun 22, 2021
-
The removed functions are still listed in the documentation; this is confusing.
-
struct _node is not very usable in the C API, but what I saw in mod_wsgi was giving gave PyParser_* output to PyNode_Compile. I tried to write porting notes for this usage. It should also help with the "compile to check for syntax errors" case in unbound.
encukou commented on Jun 24, 2021 encukou commented on Jun 28, 2021
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|