GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-120417: Add #noqa to used imports in the stdlib by vstinner · Pull Request #120421 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Jun 12, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Tools such as ruff can ignore "imported but unused" warnings if a line ends with "# noqa". It avoids the temptation to remove an import which is used effectively.

  • Issue: Remove unused imports from the stdlib #120417

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
pythongh-120417: Add #noqa to used imports in the stdlib
8347c30
Tools such as ruff can ignore "imported but unused" warnings if a
line ends with "# noqa". It avoids the temptation to remove an import
which is used effectively.
@vstinner vstinner added the skip news label Jun 12, 2024
@vstinner
vstinner requested review from abalkin, gpshead, pganssle, rhettinger and tiran as code owners June 12, 2024 16:26
@bedevere-app bedevere-app Bot mentioned this pull request Jun 12, 2024
Remove unused imports from the stdlib #120417
Closed
@bedevere-app bedevere-app Bot added the awaiting core review label Jun 12, 2024
@vstinner
Add more # noqa for more fun
19ceb4d
@vstinner
vstinner requested review from berkerpeksag, brettcannon, ericsnowcurrently, erlend-aasland, ncoghlan and warsaw as code owners June 12, 2024 16:41
vstinner added 2 commits June 12, 2024 18:50
@vstinner
One more #noqa batch
9b97f47
@vstinner
Fix test_symtable
2189665
AlexWaygood
AlexWaygood reviewed Jun 12, 2024
View reviewed changes
Comment thread Lib/importlib/machinery.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

vstinner added 2 commits June 12, 2024 21:02
@vstinner
Revert
e31a9e2
@vstinner
Merge branch 'main' into import_noqa
7d93caa
AlexWaygood
AlexWaygood reviewed Jun 12, 2024
View reviewed changes

@AlexWaygood AlexWaygood left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks much better now, a couple of questions though:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/code.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/collections/__init__.py
Comment on lines 48 to 51
try:
from _collections import _deque_iterator
from _collections import _deque_iterator # noqa
except ImportError:
pass

@AlexWaygood AlexWaygood Jun 12, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_collections passes for me if I remove this import. Does it need to stay? It was added in 52f96d3

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 12, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erlend-aasland @kumaraditya303: Do you know/recall why this symbol is exposed?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jun 12, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, test_deque fails if the import is removed. I expected deque to be tested as part of test_collections, but apparently it has its own test file. Could you add a comment that it is (apparently) required to expose it in the collections module in order for deque iterators to be pickled?

======================================================================
ERROR: test_iterator_pickle (test.test_deque.TestBasic.test_iterator_pickle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/alexw/dev/cpython/Lib/test/test_deque.py", line 640, in test_iterator_pickle
    dump = pickle.dumps((itorg, orig), proto)
_pickle.PicklingError: Can't pickle <class 'collections._deque_iterator'>: attribute lookup _deque_iterator on collections failed

----------------------------------------------------------------------

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@vstinner vstinner Jun 12, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I added a comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood reacted with heart emoji
All reactions
  • ❤️ 1 reaction
Comment thread Lib/pydoc.py Outdated
Comment on lines +78 to +80
from _pyrepl.pager import (get_pager, pipe_pager,
plain_pager, tempfile_pager, tty_pager,
plain) # noqa

@AlexWaygood AlexWaygood Jun 12, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is the import being flagged as unused here? Why does it need to stay?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 12, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pydoc.plain() is an alias to _pyrepl.pager.plain(). It has to stay since it's part of pydoc API, and test_pydoc fails if you remove it.

# noqa only applies to the current line, no?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jun 12, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pydoc.plain() is an alias to _pyrepl.pager.plain(). It has to stay since it's part of pydoc API, and test_pydoc fails if you remove it.

I see, thanks. Maybe add a comment explaining that to the code?

# noqa only applies to the current line, no?

It depends on the tool, the rule and the AST node. For example, in this:

from collections import (  # noqa: F401
    deque,
    OrderedDict,
    ChainMap,
)

the single noqa comment will cause Ruff to ignore the three unused imports deque, OrderedDict and ChainMap, because they constitute a single ImportFrom node in the AST and the noqa comment is applied on the starting line number of the AST node.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@vstinner vstinner Jun 12, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the import to a separated statement and added a comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood reacted with heart emoji
All reactions
  • ❤️ 1 reaction
@AlexWaygood
AlexWaygood removed request for brettcannon, ericsnowcurrently, ncoghlan and warsaw June 12, 2024 19:30
@vstinner
Add comments
13efcc9
@gpshead

gpshead commented Jun 12, 2024

Copy link
Copy Markdown
Member

Instead of blanket everything # noqa comments, please make these specific to the thing being disabled. # noqa: unused-import for example. Except that ruff appears to not support human meaningful and thus maintainable suppression comments in favor of inhumane error code numbers in noqa comments? So that might read F401 today.

But at least we could fix those when ruff enters the modern era and names things to be understandable by code maintainers.

AlexWaygood, aisk, and erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead gpshead mentioned this pull request Jun 12, 2024
FR: Add human-friendly check/message ids. Numeric ids (E999 etc) are not human friendly - they communicate nothing on their own. astral-sh/ruff#11849
Closed
@vstinner

vstinner commented Jun 12, 2024

Copy link
Copy Markdown
Member Author

Instead of blanket everything # noqa comments, please make these specific to the thing being disabled.

On a line like import readline # noqa, it's clear that noqa suppress a warning about unused import, no?

So that might read F401 today.

So you suggest using # noqa: F401 instead?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jun 12, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Except that ruff appears to not support human meaningful and thus maintainable suppression comments in favor of inhumane error code numbers in noqa comments? So that might read F401 today.

But at least we could fix those when ruff enters the modern era and names things to be understandable by code maintainers.

it's on our to-do list :-)

Edit: and I see Zanie already said that on the Ruff issue tracker ;)

gpshead reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood
AlexWaygood approved these changes Jun 12, 2024
View reviewed changes

@AlexWaygood AlexWaygood left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also prefer that we ignore a specific linter error (F401) rather than all linter errors. But other than that, this LGTM now, thanks (though some tests seem to be failing)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jun 12, 2024
@gpshead

gpshead commented Jun 12, 2024

Copy link
Copy Markdown
Member

yep, always list the specific error(s) being ignored in the comment.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger
rhettinger removed their request for review June 12, 2024 22:20
pganssle
pganssle reviewed Jun 13, 2024
View reviewed changes
Comment thread Lib/operator.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka
serhiy-storchaka approved these changes Jun 13, 2024
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

How did you know that all these imports were necessary? Or did you just flag all the names for which you can't confirm redundant imports?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
serhiy-storchaka
serhiy-storchaka reviewed Jun 13, 2024
View reviewed changes
Comment thread Lib/pydoc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka
Update Lib/pydoc.py
648969c
@vstinner

vstinner commented Jun 13, 2024

Copy link
Copy Markdown
Member Author

How did you know that all these imports were necessary? Or did you just flag all the names for which you can't confirm redundant imports?

I'm checking for unused imports for 5 years, and every year, I meet the same imports :-) Over time, I learnt which imports are done to populate an API, and which ones can be removed.

You can try to remove them and see the test suite failing.

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented Jun 13, 2024

Copy link
Copy Markdown
Member

I agree with all this, I just wondering how did you find them, because the amount of the work is large, and many cases are not trivial. Removing imports unlit the code breaks is risky, not all can be covered by tests.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
Use "# noqa: F401"
0506006
@vstinner

vstinner commented Jun 13, 2024

Copy link
Copy Markdown
Member Author

I modified the PR to use # noqa: F401. Hopefully, it seems like warning numbers are standardized between linters.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 13, 2024

Copy link
Copy Markdown
Member Author

I agree with all this, I just wondering how did you find them, because the amount of the work is large, and many cases are not trivial. Removing imports unlit the code breaks is risky, not all can be covered by tests.

I also rely on code review to validate my work :-)

AlexWaygood reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Jun 13, 2024

Copy link
Copy Markdown
Member

yep, always list the specific error(s) being ignored in the comment.

There's a rule (PGH004 blanket-noqa) to prevent noqas that are missing the specific error. We have it in https://github.com/python/cpython/blob/main/Tools/clinic/.ruff.toml:

cpython/Tools/clinic/.ruff.toml

Line 9 in ca5108a

"PGH004", # Ban blanket `# noqa` comments (only ignore specific error codes)

Shall we add it to the other one, https://github.com/python/cpython/blob/main/Lib/test/.ruff.toml ?

And maybe add another .ruff.toml for Lib/ or other directories?


I'm checking for unused imports for 5 years, and every year, I meet the same imports :-) Over time, I learnt which imports are done to populate an API, and which ones can be removed.

If we added F401 unused-import to Ruff, these would get flagged by the CI, and we'd have to label the ones we really need with noqa: F401.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner merged commit 6ae254a into python:main Jun 13, 2024
@vstinner
vstinner deleted the import_noqa branch June 13, 2024 14:14
@bedevere-app bedevere-app Bot removed the awaiting merge label Jun 13, 2024
@vstinner

vstinner commented Jun 13, 2024

Copy link
Copy Markdown
Member Author

I created a second PR for the 7 files which requires tons of #noqa: #120454.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 13, 2024

Copy link
Copy Markdown
Member Author

Merged, thanks for reviews!

@hugovk: I'm interested by a CI checking for unused imports, it's appealing :-)

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner mentioned this pull request Jun 17, 2024
gh-120417: Add #noqa to load_tests() in tests #120626
Closed
@encukou

encukou commented Jun 17, 2024

Copy link
Copy Markdown
Member

For things that are re-exported for a public API, should add them to __all__ rather than add the linter directive?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jun 17, 2024

Copy link
Copy Markdown
Member

For things that are re-exported for a public API, should add them to __all__ rather than add the linter directive?

usually IMO, yes

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mrahtz pushed a commit to mrahtz/cpython that referenced this pull request Jun 30, 2024
@vstinner @mrahtz
pythongh-120417: Add #noqa to used imports in the stdlib (python#120421)
c04a01d
Tools such as ruff can ignore "imported but unused" warnings if a
line ends with "# noqa: F401". It avoids the temptation to remove
an import which is used effectively.
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
@vstinner @estyxx
pythongh-120417: Add #noqa to used imports in the stdlib (python#120421)
cd91fb4
Tools such as ruff can ignore "imported but unused" warnings if a
line ends with "# noqa: F401". It avoids the temptation to remove
an import which is used effectively.
@vstinner vstinner mentioned this pull request Mar 13, 2025
Remove unused imports - March 2025 Edition #131152
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pganssle pganssle pganssle left review comments
@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes
@AlexWaygood AlexWaygood AlexWaygood approved these changes
@gpshead gpshead Awaiting requested review from gpshead gpshead is a code owner
@abalkin abalkin Awaiting requested review from abalkin
@tiran tiran Awaiting requested review from tiran
@berkerpeksag berkerpeksag Awaiting requested review from berkerpeksag
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@vstinner @gpshead @AlexWaygood @serhiy-storchaka @hugovk @encukou @pganssle
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.