GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-81283: compiler: remove indent from docstring by methane · Pull Request #106411 · python/cpython

Conversation

@methane

@methane methane commented Jul 4, 2023 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: Automatically dedent docstring constants by default #81283

📚 Documentation preview 📚: https://cpython-previews--106411.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@methane
methane requested review from iritkatriel and markshannon as code owners July 4, 2023 12:00
@bedevere-bot bedevere-bot added the awaiting core review label Jul 4, 2023
@bedevere-bot bedevere-bot mentioned this pull request Jul 4, 2023
Automatically dedent docstring constants by default #81283
Closed
@methane

methane commented Jul 4, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author 
# sqlalchemy/orm/__pycache__
# main:
ls -lS | head
-rw-r--r--  1 inada-n  staff  197484 Jul  4 18:06 session.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  169161 Jul  4 18:06 mapper.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  137441 Jul  4 18:06 events.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  130456 Jul  4 18:06 relationships.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  128389 Jul  4 18:06 query.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  104012 Jul  4 18:06 strategies.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  101628 Jul  4 18:06 _orm_constructors.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  100939 Jul  4 18:06 context.cpython-313.pyc
-rw-r--r--  1 inada-n  staff   98761 Jul  4 18:06 attributes.cpython-313.pyc

# c-cleandoc:
total 4432
-rw-r--r--  1 inada-n  staff  187185 Jul  4 21:04 session.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  163634 Jul  4 21:04 mapper.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  127659 Jul  4 21:04 relationships.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  124634 Jul  4 21:04 events.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  119689 Jul  4 21:04 query.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  103889 Jul  4 21:04 strategies.cpython-313.pyc
-rw-r--r--  1 inada-n  staff  100432 Jul  4 21:04 context.cpython-313.pyc
-rw-r--r--  1 inada-n  staff   97329 Jul  4 21:04 attributes.cpython-313.pyc
-rw-r--r--  1 inada-n  staff   96105 Jul  4 21:04 _orm_constructors.cpython-313.pyc

# file size of session.pyc
>>> a,b=197484,187185
>>> (a-b)*100/a
5.215106033906545
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
methane
methane commented Jul 4, 2023
View reviewed changes
Comment thread Lib/inspect.py
lines = doc.expandtabs().split('\n')
except UnicodeError:
return None
else:

@methane methane Jul 4, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this try-except-else block because Python 3 don't autodecode from bytes.
If doc is bytes, doc.split('\n') raises TypeError, not UnicodeError.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
methane
methane commented Jul 4, 2023
View reviewed changes
Comment thread Lib/test/test_doctest.py

>>> def f(x):
... '>>> print(1, 2, 3)\n 1 2\n 3'
... '\n>>> print(1, 2, 3)\n 1 2\n 3'

@methane methane Jul 4, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is needed to avoid dedenting output examples.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@methane
methane force-pushed the c-cleandoc branch from e43ac5f to 12490d9 Compare July 4, 2023 12:17
methane added 3 commits July 5, 2023 02:08
@methane
compile time cleandoc
469b3f7
@methane
fix test_doctest
6fc4456
@methane
Add NEWS and what's new entry
78ef10c
@methane
methane force-pushed the c-cleandoc branch from 12490d9 to 78ef10c Compare July 4, 2023 17:08
iritkatriel
iritkatriel reviewed Jul 5, 2023
View reviewed changes
Comment thread Python/compile.c

Py_DECREF(doc);
return PyUnicode_FromStringAndSize(buff, w - buff);
}

@iritkatriel iritkatriel Jul 5, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure the dedent logic belongs in compile.c.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@methane methane Jul 5, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this dedent logic is very specific for docstring.
We can not use this logic in other place except inspect.cleandoc.

If we reuse this logic in inspect.cleandoc, it would be:

doc = compiler.cleandoc(doc).strip('\n')

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@methane
Share more testdata between inspect and compiler cleandoc
34d0cbd
methane
methane commented Jul 5, 2023
View reviewed changes
Comment thread Lib/inspect.py
# Find minimum indentation of any non-blank lines after first line.
margin = sys.maxsize
for line in lines[1:]:
content = len(line.lstrip(' '))

@methane methane Jul 5, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed from line.lstrip() to line.lstrip(' ') for better compatibility between inspect.cleandoc and compiler.cleandoc.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
methane added 2 commits July 11, 2023 23:46
@methane
Merge branch 'main' into c-cleandoc
f61fe41
@methane
Merge branch 'main' into c-cleandoc
a16e5d8
@methane methane mentioned this pull request Jul 14, 2023
Add textwrap.dedent, .indent, as str methods. #62535
Open
merwok
merwok reviewed Jul 14, 2023
View reviewed changes
Comment thread Doc/whatsnew/3.13.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Doc/whatsnew/3.13.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
methane and others added 3 commits July 15, 2023 17:57
@methane @merwok
Apply suggestions from code review
64d0d2f 
Co-authored-by: Éric <[email protected]>
@methane
skip dedent if not needed
976a013
@methane
add link to glossary
028b5df
@methane
methane merged commit 2566b74 into python:main Jul 15, 2023
@methane
methane deleted the c-cleandoc branch July 15, 2023 10:33
@bedevere-bot bedevere-bot removed the awaiting core review label Jul 15, 2023
@Eclips4 Eclips4 mentioned this pull request Jul 17, 2023
test_doctest are leaked #106843
Closed
sunmy2019
sunmy2019 reviewed Jul 18, 2023
View reviewed changes
Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@Eclips4 Eclips4 mentioned this pull request Jul 20, 2023
Forgot to add error check? #106916
Closed
@merwok

merwok commented Jul 24, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

With the forum discussion ongoing, and the other discussion about requiring reviews, I think this could have waited a little bit to get a review approval. On the other hand it’s not a huge change!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@merwok merwok mentioned this pull request Jul 25, 2023
Auto dedent -c arguments #103997
Closed
Carreau added a commit to Carreau/ipython that referenced this pull request Dec 31, 2023
@Carreau
Fix docstring leading space.
d36b634 
Since python/cpython#106411, it look like
cpython is removing leading space and indent.

We therefore need conditional check, until we decide Wether this is a
problem we want to deal with or not.
@Carreau Carreau mentioned this pull request Dec 31, 2023
Fix docstring leading space. ipython/ipython#14279
Merged
Carreau added a commit to Carreau/ipython that referenced this pull request Dec 31, 2023
@Carreau
Fix docstring leading space.
51e92d7 
Since python/cpython#106411, it look like
cpython is removing leading space and indent.

We therefore need conditional check, until we decide Wether this is a
problem we want to deal with or not.
Carreau added a commit to ipython/ipython that referenced this pull request Dec 31, 2023
@Carreau
Fix docstring leading space. (#14279)
fd2cf18 
Since python/cpython#106411, it look like
cpython is removing leading space and indent.

We therefore need conditional check, until we decide Wether this is a
problem we want to deal with or not.
@MichaReiser MichaReiser mentioned this pull request Feb 8, 2024
Formatter: Uses space indentation in docstrings when using indent-style = "tab" astral-sh/ruff#8430
Closed
@Carreau Carreau mentioned this pull request Oct 31, 2025
review ipython.utils.text:dedent ipython/ipython#14280
Open
@decorator-factory decorator-factory mentioned this pull request Dec 8, 2025
Indicate that surrogate literals are not allowed in docstrings since Python 3.13 #142411
Closed
@StanFromIreland StanFromIreland mentioned this pull request May 3, 2026
compile() raises UnicodeEncodeError for docstrings with lone surrogates #149303
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead left review comments
@merwok merwok merwok left review comments
@iritkatriel iritkatriel iritkatriel left review comments
@sunmy2019 sunmy2019 sunmy2019 left review comments
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@methane @merwok @gpshead @iritkatriel @sunmy2019 @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.