GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-148991: Add colour to `tokenize` CLI output by hugovk · Pull Request #148992 · python/cpython

Conversation

@hugovk

@hugovk hugovk commented Apr 25, 2026 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

For example, given this script which has a bit of everything:

Details
# A comment

x = 42
y = 3.14
z = 2 + 3j
name = "hello"
raw = b"bytes"
f_str = f"value: {x}"
t_str = t"template: {x}"

match x:
    case 1:
        pass

for i in range(10):
    if i % 2 == 0:
        print(i)

def greet(name: str = "world") -> None:
    ...

class Foo:
    @staticmethod
    def bar(a, b=True, *args, **kwargs):
        return a + b

lambda x: x * 2

result = [i**2 for i in range(5)]
mapping = {"key": "value", 1: None}
s = {1, 2, 3}
t = (1,)

try:
    1 / 0
except ZeroDivisionError as e:
    raise ValueError("oops") from e

async def main():
    await something()
    async for item in aiter():
        yield item

a = x if x > 0 else -x
b = x | y & z ^ ~x
c = x << 1 >> 2
d: int
e += 1  # type: ignore
image
  • Issue: Add colour to tokenize CLI output #148991

📚 Documentation preview 📚: https://cpython-previews--148992.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@hugovk
Add colour to tokenize CLI output
7d11c7a
@hugovk
hugovk requested review from AA-Turner, lysnikolaou and pablogsal as code owners April 25, 2026 15:16
@bedevere-app bedevere-app Bot added the awaiting core review label Apr 25, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Apr 25, 2026
Add colour to tokenize CLI output #148991
Closed
@StanFromIreland
Move to _print_tokens()
1a0039a
@hugovk hugovk mentioned this pull request Apr 25, 2026
Move tokenizer color to _print_tokens() hugovk/cpython#82
Merged
StanFromIreland and others added 2 commits April 25, 2026 17:38
@StanFromIreland
Yield instead
1dda979
@hugovk
Move tokenizer color to _print_tokens() (#82)
49b7f38
StanFromIreland
StanFromIreland reviewed Apr 26, 2026
View reviewed changes
Comment thread Lib/tokenize.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@hugovk @StanFromIreland
Update Lib/tokenize.py
60a24c6
Co-authored-by: Stan Ulbrych <[email protected]>
StanFromIreland
StanFromIreland approved these changes Apr 26, 2026
View reviewed changes

@StanFromIreland StanFromIreland left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk reacted with rocket emoji
All reactions
  • 🚀 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Apr 26, 2026
@hugovk
hugovk merged commit 1e7dfbc into python:main Apr 26, 2026
150 of 152 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
hugovk deleted the 3.15-tokenize-colour branch April 26, 2026 19:14
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@StanFromIreland StanFromIreland StanFromIreland approved these changes
@pablogsal pablogsal Awaiting requested review from pablogsal pablogsal is a code owner
@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@hugovk @StanFromIreland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.