GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-121970: Extract ``pydoc_topics`` into a new extension by AA-Turner · Pull Request #129116 · python/cpython

Conversation

@AA-Turner

@AA-Turner AA-Turner commented Jan 21, 2025 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This also simplifies the pydoc-topics builder. Grouping the topic labels by docname improves the speed of topics generation from ~68s to ~57s (19% faster) from a cold state, and from ~13s to ~3.4s (3.8x faster) when re-using the pickled documents.

The representation of topics.py also changes from the default pprint.pformat output of:

topics = {'assert': 'The "assert" statement\n'
           '**********************\n'
           '\n'
           'Assert statements are a convenient way to insert debugging '
           'assertions\n'
           'into a program:\n'

to a simpler representation using triple single quotes (save for when ''' appears in the body):

topics = {
    'assert': r'''The "assert" statement
**********************

Assert statements are a convenient way to insert debugging assertions
into a program:
'''

This representation is both nicer to read and is 63% of the file size of the current topics.py (518KB vs 830KB). Line count also decreases from 17,486 to 12,782.

Tested by running:

>>> import runpy
>>> topics_old = runpy.run_path("Doc/build/topics_old.py")['topics']
>>> topics_new = runpy.run_path("Doc/build/topics_new.py")['topics']
>>> assert list(topics_old) == list(topics_new) # check order
>>> [k for k in topics_old if topics_old[k] != topics_new[k]]
['debugger', 'formatstrings']

The 'formatstrings' change is trailing whitespace on the >>> for num in range(5,12): line:

>>> fs_old_stripped = '\n'.join(map(str.rstrip, topics_old['formatstrings'].splitlines()))
>>> fs_new_stripped = '\n'.join(map(str.rstrip, topics_new['formatstrings'].splitlines()))
>>> assert fs_old_stripped == fs_new_stripped

The 'debugger' change is "Ctrl-C" to "Ctrl"-"C", but I'm not sure what caused this:

>>> print('\n'.join(difflib.unified_diff(topics_old['debugger'].splitlines(), topics_new['debugger'].splitlines())))
--- 
+++ 
@@ -186,9 +186,9 @@
    originate in a module that matches one of these patterns. [1]
 
    By default, Pdb sets a handler for the SIGINT signal (which is sent
-   when the user presses "Ctrl-C" on the console) when you give a
+   when the user presses "Ctrl"-"C" on the console) when you give a
    "continue" command. This allows you to break into the debugger
-   again by pressing "Ctrl-C".  If you want Pdb not to touch the
+   again by pressing "Ctrl"-"C".  If you want Pdb not to touch the
    SIGINT handler, set *nosigint* to true.

cc @hugovk as 3.14 release manager as this does change the format of Lib/pydoc_data/topics.py. I'm happy when doing backports to preserve the current format (pformat) if release managers would prefer.

A

  • Issue: Improve and reorganise pyspecific #121970

📚 Documentation preview 📚: https://cpython-previews--129116.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
AA-Turner added 8 commits January 21, 2025 00:19
@AA-Turner
Extract pydoc_topics into a new extension
192eac8
@AA-Turner
Initial refactor (format, lint, type annotations, pathlib)
d44ad65
@AA-Turner
Second stage refactor
c78578b
@AA-Turner
Third stage refactor (use TextTranslator directly)
fe0a5dc
@AA-Turner
Fourth stage refactor (aggregate documents)
371755d
@AA-Turner
double quotes
e27c60a
@AA-Turner
Use a triple single quote representation
1d278ae
@AA-Turner
Update Lib/pydoc_data/topics.py
5ac0ac4
@AA-Turner
AA-Turner requested a review from hugovk as a code owner January 21, 2025 03:52
@bedevere-app bedevere-app Bot mentioned this pull request Jan 21, 2025
Improve and reorganise pyspecific #121970
Open
@bedevere-app bedevere-app Bot added the awaiting core review label Jan 21, 2025
@AA-Turner AA-Turner added docs Documentation in the Doc dir skip news needs backport to 3.12 only security fixes needs backport to 3.13 only security fixes labels Jan 21, 2025
@hugovk

hugovk commented Jan 21, 2025

Copy link
Copy Markdown
Member

On macOS, make -C Doc pydoc-topics goes from 14s (cold) and 7s (warm) to 9s and 2s.

The file also reduced from 830 KB to 519 KB.

However, I don't have any ['debugger', 'formatstrings'] differences:

Python 3.13.1 (v3.13.1:06714517797, Dec  3 2024, 14:00:22) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import runpy
>>> topics_old = runpy.run_path("topics_old.py")['topics']
>>> topics_new = runpy.run_path("topics_new.py")['topics']
>>> assert list(topics_old) == list(topics_new) # check order
>>> [k for k in topics_old if topics_old[k] != topics_new[k]]
[]
>>>

topics_old_and_new.zip

AA-Turner reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AA-Turner

AA-Turner commented Jan 21, 2025

Copy link
Copy Markdown
Member Author

Wonderful! I think I may have been testing with Sphinx 8.1 vs 8.2 (unreleased), as I made a change to how the :kbd: role is implemented, which would explain the Ctrl-C change.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk
hugovk approved these changes Jan 21, 2025
View reviewed changes

@hugovk hugovk left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @hugovk as 3.14 release manager as this does change the format of Lib/pydoc_data/topics.py. I'm happy when doing backports to preserve the current format (pformat) if release managers would prefer.

I'm okay with it, but then there are no backports to 3.14 🙃

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 21, 2025
@AA-Turner

AA-Turner commented Jan 21, 2025

Copy link
Copy Markdown
Member Author

Ok, let's merge this and then ask @Yhg1s what he thinks about backporting to 3.13 and 3.12. I'm assuming the backports will fail due to the change to Lib/pydoc_data/topics.py

A

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AA-Turner
AA-Turner merged commit 01bcf13 into python:main Jan 21, 2025
@miss-islington-app

miss-islington-app Bot commented Jan 21, 2025

Copy link
Copy Markdown

Thanks @AA-Turner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 21, 2025
@miss-islington-app

miss-islington-app Bot commented Jan 21, 2025

Copy link
Copy Markdown

Sorry, @AA-Turner, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 01bcf13a1c5bfca5124cf2e0679c9d1b25b04708 3.13
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app miss-islington-app Bot assigned AA-Turner Jan 21, 2025
@miss-islington-app

miss-islington-app Bot commented Jan 21, 2025

Copy link
Copy Markdown

Sorry, @AA-Turner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 01bcf13a1c5bfca5124cf2e0679c9d1b25b04708 3.12
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Feb 11, 2025
Remove trailing whitespace in Lib/pydoc_data/topics.py #130014
Merged
gh-127833: Fix grammar snippet formatting for help() output #129692
Closed
@AA-Turner
AA-Turner deleted the docs/pydoc-topics branch February 12, 2025 19:35
@terryjreedy

terryjreedy commented Feb 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Did you decide not to backport? Is so, please remove labels.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AA-Turner

AA-Turner commented Feb 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I would like to backport; waiting on RM approval (see above).

A

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AA-Turner added a commit to AA-Turner/cpython that referenced this pull request Feb 22, 2025
@AA-Turner
pythonGH-121970: Extract pydoc_topics into a new extension (pytho…
d439c38
…n#129116)

(cherry picked from commit 01bcf13)
AA-Turner added a commit to AA-Turner/cpython that referenced this pull request Feb 22, 2025
@AA-Turner
[3.13] pythonGH-121970: Extract pydoc_topics into a new extension (…
5169c01
…pythonGH-129116)

(cherry picked from commit 01bcf13)

Co-authored-by: Adam Turner <[email protected]>
@bedevere-app

bedevere-app Bot commented Feb 22, 2025

Copy link
Copy Markdown

GH-130441 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Feb 22, 2025
AA-Turner added a commit to AA-Turner/cpython that referenced this pull request Feb 22, 2025
@AA-Turner
[3.12] pythonGH-121970: Extract pydoc_topics into a new extension (…
3205dce
…python#129116)

(cherry picked from commit 01bcf13)
AA-Turner added a commit to AA-Turner/cpython that referenced this pull request Feb 22, 2025
@AA-Turner
[3.12] pythonGH-121970: Extract pydoc_topics into a new extension (…
9ac56b8
…pythonGH-129116)

(cherry picked from commit 01bcf13)

Co-authored-by: Adam Turner <[email protected]>
@bedevere-app

bedevere-app Bot commented Feb 22, 2025

Copy link
Copy Markdown

GH-130443 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Feb 22, 2025
AA-Turner added a commit that referenced this pull request Feb 27, 2025
@AA-Turner
[3.13] GH-121970: Extract pydoc_topics into a new extension (#129116)…
93b2bcb
… (#130441)

Co-authored-by: Adam Turner <[email protected]>
AA-Turner added a commit that referenced this pull request Feb 27, 2025
@AA-Turner
[3.12] GH-121970: Extract pydoc_topics into a new extension (#129116)…
fb073b5
… (#130443)

Co-authored-by: Adam Turner <[email protected]>
hugovk added a commit to hugovk/cpython that referenced this pull request Mar 14, 2025
@hugovk
Revert "pythonGH-121970: Extract pydoc_topics into a new extension (
59307a6
python#129116)"

This reverts commit 01bcf13.
hugovk added a commit that referenced this pull request Mar 14, 2025
@hugovk
gh-121970: Revert "GH-121970: Extract pydoc_topics into a new ext…
ca1bedc
…ension (#129116)" (#131245)

Revert "GH-121970: Extract ``pydoc_topics`` into a new extension (#129116)"

This reverts commit 01bcf13.
AA-Turner added a commit to AA-Turner/cpython that referenced this pull request Mar 14, 2025
@AA-Turner
Revert "pythongh-121970: Revert "pythonGH-121970: Extract ``pydoc_top…
5c8ef1d
…ics`` into a new extension (python#129116)" (python#131245)"

This reverts commit ca1bedc.
AA-Turner added a commit to AA-Turner/cpython that referenced this pull request Mar 14, 2025
@AA-Turner
Reapply "pythonGH-121970: Extract pydoc_topics into a new extensi…
29fccb6
…on (python#129116)"

This reverts commit ca1bedc.
plashchynski pushed a commit to plashchynski/cpython that referenced this pull request Mar 17, 2025
@hugovk @plashchynski
pythongh-121970: Revert "pythonGH-121970: Extract pydoc_topics in…
219fc85
…to a new extension (python#129116)" (python#131245)

Revert "pythonGH-121970: Extract ``pydoc_topics`` into a new extension (python#129116)"

This reverts commit 01bcf13.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hugovk hugovk hugovk approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@AA-Turner AA-Turner

Labels

docs Documentation in the Doc dir skip news

Projects

Docs PRs
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@AA-Turner @hugovk @terryjreedy
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.