GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-43950: Initial base implementation for PEP 657 by pablogsal · Pull Request #26955 · python/cpython

Conversation

@pablogsal

@pablogsal pablogsal commented Jun 29, 2021 •
edited by isidentical
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

⚠️ This is not the full feature and contains no optimizations for the time being, just the base implementation to avoid merge conflicts in the future ⚠️

All the internal structures, and data representation are subjected to change in future PRs as we add optimizations

This PR is part of PEP 657 and augments the compiler to emit ending
line numbers as well as starting and ending columns from the AST
into compiled code objects. This allows bytecodes to be correlated
to the exact source code ranges that generated them.

This information is made available through the following public APIs:

  • The co_positions method on code objects.
  • The C API function PyCode_Addr2Location.

Co-authored-by: Pablo Galindo [email protected]
Co-authored-by: Batuhan Taskaya [email protected]
Co-authored-by: Ammar Asker [email protected]

https://bugs.python.org/issue43950

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@pablogsal
pablogsal requested review from brettcannon, encukou, ericsnowcurrently, markshannon, ncoghlan, terryjreedy, tiran and warsaw as code owners June 29, 2021 15:06
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jun 29, 2021
@bedevere-bot bedevere-bot added the awaiting core review label Jun 29, 2021
@pablogsal pablogsal added the skip news label Jun 29, 2021
@pablogsal
pablogsal removed request for brettcannon, encukou, ericsnowcurrently, markshannon, ncoghlan, terryjreedy, tiran and warsaw June 29, 2021 16:25
ammaraskar
ammaraskar reviewed Jun 29, 2021
View reviewed changes
Comment thread Objects/typeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

ammaraskar
ammaraskar reviewed Jun 29, 2021
View reviewed changes
Comment thread Python/traceback.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@isidentical

isidentical commented Jun 29, 2021

Copy link
Copy Markdown
Member

A couple doctest failures FYI: https://travis-ci.com/github/python/cpython/jobs/520228793

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ammaraskar
ammaraskar force-pushed the bpo-43950 branch 2 times, most recently from b53b276 to 0673116 Compare June 30, 2021 04:49
@pablogsal pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jun 30, 2021
@bedevere-bot

bedevere-bot commented Jun 30, 2021

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 0673116a6fa30547c1138fd3f96059412d99a979 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jun 30, 2021
isidentical
isidentical approved these changes Jun 30, 2021
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Jun 30, 2021
Fidget-Spinner
Fidget-Spinner reviewed Jun 30, 2021
View reviewed changes
Comment thread Include/cpython/code.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Jun 30, 2021

Copy link
Copy Markdown
Member

Instead of adding three additional location fields to each instruction, and continually update the equivalent fields in the scope struct, why not change i_lineno to i_ast_node to hold the AST node with the matching location?

Doesn't PEP 657 require that the locations match those of the AST exactly?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Jun 30, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Instead of adding three additional location fields to each instruction, and continually update the equivalent fields in the scope struct, why not change i_lineno to i_ast_node to hold the AST node with the matching location?

That would be basically the same because you are modifiying these fields in the compilation and assembler steps. For example:

...
b->b_instr[i].i_lineno = prev_lineno;
...

that means that if we use a i_ast_node pointer all instructions will share the same ast node, so changes to the lineno of one instruction will affect others, which is a big problem. So either we start adding a bunch of macros to get and set to handle the default case a NULL pointer for i_ast_node or we would end copiying the ast node location structure, which is almost exactly what we are doing currently.

Doesn't PEP 657 require that the locations match those of the AST exactly?

No, the PEP just states what is the source of the information, but it doesn't explain how is propagated. So we have some freedom to select from which nodes do we pick the information or to "fix it" if we ever need to.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Jun 30, 2021

Copy link
Copy Markdown
Member Author

@isidentical By the way, it seems that there are more places where we are setting lineno = -1 that we didn't update to also account for the new fields

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal
pablogsal force-pushed the bpo-43950 branch 2 times, most recently from 9ca2578 to 2498168 Compare July 1, 2021 00:12
@ammaraskar

ammaraskar commented Jul 1, 2021

Copy link
Copy Markdown
Member

@terryjreedy Adding you as a reviewer as requested. You might not be too interested in this part, it's just exposing the lower level offset data but the tests might give you sort of an idea on how they can be used from Python. The follow up PRs will add the traceback.py changes that you might be more interested in.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ammaraskar
ammaraskar requested a review from terryjreedy July 1, 2021 17:09
@isidentical
isidentical force-pushed the bpo-43950 branch from 2d37745 to 58d4b8b Compare July 1, 2021 19:54
@isidentical isidentical added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 1, 2021
@bedevere-bot

bedevere-bot commented Jul 1, 2021

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @isidentical for commit ba64180d3f61ee75b2b624ae2055ec22a2fb27fc 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 1, 2021
terryjreedy
terryjreedy reviewed Jul 1, 2021
View reviewed changes

@terryjreedy terryjreedy left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand, the only officially visible change is the addition of code.co_positions, which returns an iterable of (start_line, end_line, start_column, end_column) tuples, where column numbers are 1-based. Suggested title after bpo tag: "Add code.co_positions (PEP 657)" or "Add PEP-657 code.co_positions".

I presume a doc change will come. I will review it for readability if pinged.

I read the tests and have two suggestions.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_compile.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_compile.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@ammaraskar @isidentical @pablogsal
bpo-43950: Add code.co_positions (PEP 657)
d989cd1
This PR is part of PEP 657 and augments the compiler to emit ending
line numbers as well as starting and ending columns from the AST
into compiled code objects. This allows bytecodes to be correlated
to the exact source code ranges that generated them.

This information is made available through the following public APIs:

* The `co_positions` method on code objects.
* The C API function `PyCode_Addr2Location`.

Co-authored-by: Batuhan Taskaya <[email protected]>
Co-authored-by: Pablo Galindo <[email protected]>
@ammaraskar
ammaraskar force-pushed the bpo-43950 branch from 8f88710 to d989cd1 Compare July 2, 2021 01:00
@ammaraskar

ammaraskar commented Jul 2, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Amended the commit message as per Terry's recommendation and rebased #26958 on top of the latest changes, tests for the traceback changes are still passing.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal
pablogsal merged commit 98eee94 into python:main Jul 2, 2021
@bedevere-bot bedevere-bot removed the awaiting merge label Jul 2, 2021
@pablogsal
pablogsal deleted the bpo-43950 branch July 2, 2021 14:10
@pablogsal pablogsal mentioned this pull request Apr 13, 2022
Include column offsets for bytecode instructions #88116
Closed
@sunmy2019 sunmy2019 mentioned this pull request Sep 9, 2023
Exceptions slow in 3.11, depending on location #109181
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou encukou left review comments
@ammaraskar ammaraskar ammaraskar left review comments
@terryjreedy terryjreedy terryjreedy left review comments
@Fidget-Spinner Fidget-Spinner Fidget-Spinner left review comments
@isidentical isidentical isidentical approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

9 participants

@pablogsal @isidentical @bedevere-bot @markshannon @ammaraskar @encukou @terryjreedy @Fidget-Spinner @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.