GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Remove, move or `# noqa` more `TypeAlias` declarations by AlexWaygood · Pull Request #8450 · python/typeshed

Conversation

@AlexWaygood

@AlexWaygood AlexWaygood commented Jul 31, 2022

Copy link
Copy Markdown
Member

Unblocks PyCQA/flake8-pyi#255

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@AlexWaygood
Remove, move or # noqa more TypeAlias declarations
571e375
Akuli
Akuli reviewed Jul 31, 2022
View reviewed changes
Comment thread stdlib/multiprocessing/__init__.pyi
__all__ += ["parent_process"]

# The following type aliases can be used to annotate the return values of
# the corresponding functions. They are not defined at runtime.

@Akuli Akuli Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were a bit controversial when @srittau added them, and he probably doesn't want them removed: #5346

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jul 31, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks for the reference.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jul 31, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other options are:

  • Special-case __init__.pyi files in the flake8-pyi check (what I did originally, but @JelleZijlstra doesn't like that).
  • Just # noqa all the ones in multiprocessing.
  • Abandon the flake8-pyi check entirely.
  • Something else?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Akuli Akuli Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is somewhat of a "true positive". These type aliases are just weird: named as if they were private, intended to be public, not documented, not discoverable. But again, @srittau will probably disagree. I think we should wait for his input.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@AlexWaygood AlexWaygood Jul 31, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But again, @srittau will probably disagree. I think we should wait for his input.

Yeah of course, not going to merge without it :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@github-actions

This comment has been minimized.

Sign in to view
srittau
srittau reviewed Jul 31, 2022
View reviewed changes
Comment thread stdlib/multiprocessing/__init__.pyi Outdated
Comment on lines +13 to +14
# Using them as annotations is deprecated. Use imports from multiprocessing.queues instead.
# See #4266 for discussion.

@srittau srittau Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is just outdated? As far as I can see, all these are classes in the implementation, at least in Python 3.7+. Was this maybe different in the past? In that case, I don't think we need the type aliases below.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Akuli Akuli Jul 31, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They became functions in the stub at some point, but it was reverted.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@Akuli Akuli Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow I missed that you asked about the implementation, not about the stub. These are functions (actually methods) in the implementation:

>>> from multiprocessing import Queue
>>> Queue
<bound method BaseContext.Queue of <multiprocessing.context.DefaultContext object at 0x7f66f26d73d0>>

There is also an undocumented class that is different from the public and documented Queue:

>>> from multiprocessing.queues import Queue
>>> Queue
<class 'multiprocessing.queues.Queue'>

There's similarly e.g. a method multiprocessing.Event and a class multiprocessing.synchronize.Event.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jul 31, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what happens at runtime:

>>> import multiprocessing
>>> multiprocessing.Queue
<bound method BaseContext.Queue of <multiprocessing.context.DefaultContext object at 0x000002F32AF7E5C0>>
>>> multiprocessing.Queue[int]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'method' object is not subscriptable

So the stubs are a bit of a lie at the moment. As @Akuli says, the stubs were changed in #4289 so that they could match the runtime more closely, but then that was reverted in #4314 after Jukka complained in #4313.

The thing is, either way, I don't think having these aliases helps the situation much. To work around these issues, a user can probably just put multiprocessing.Queue in quotes in their annotations, or use from __future__ import annotations.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@srittau srittau Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should try to make them into functions again in the stubs and see the fallout, now that we have mypy_primer. Annotating something with muliprocessing.Queue, which is a function, is not something we should support.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

This comment was marked as off-topic.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Akuli Akuli Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I marked my previous comment as off-topic, as it doesn't belong at all to this PR. I somehow didn't see other people's comments that were posted before I wrote mine.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jul 31, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the type aliases are connected to this, we should keep those that are alternatives to a function.

I don't really see how the aliases help, even the ones which are alternatives to a function.

Take multiprocessing.Lock -- multiprocessing.Lock is a method, but multiprocessing.synchronize.Lock is a class. If it's a choice between doing this:

from multiprocessing.synchronize import Lock
foo: Lock

or this:

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from multiprocessing import _LockType
foo: "_LockType"

Then, I find the first way much cleaner and more ergonomic. And, as @Akuli says in his minimized comment, we know none of the projects in mypy_primer are using these aliases.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@srittau srittau Jul 31, 2022

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for being unclear. Yes, if there is a valid type somewhere, like multiprocessing.synchronize.Lock, we don't need the alias. It's possible that this is true for all aliases, I didn't check.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@AlexWaygood AlexWaygood Jul 31, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_BarrierType, _BoundedSemaphoreType, _ConditionType, _EventType, _LockType, _RLockType and _SemaphoreType are all aliases to valid types in multiprocessing.synchronize.

_QueueType, _SimpleQueueType and _JoinableQueueType are all aliases to valid types in multiprocessing.queues.

So yeah, it's true for all the aliases :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

srittau reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@AlexWaygood

AlexWaygood commented Aug 5, 2022

Copy link
Copy Markdown
Member Author

So, are we happy for this to go in? :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood
AlexWaygood commented Aug 5, 2022
View reviewed changes
Comment thread stdlib/multiprocessing/__init__.pyi Resolved

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood and others added 2 commits August 5, 2022 12:36
@AlexWaygood
Improve comment
1424f10
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
afbf143
srittau
srittau approved these changes Aug 5, 2022
View reviewed changes
@github-actions

github-actions Bot commented Aug 5, 2022

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood
AlexWaygood merged commit 892796a into master Aug 5, 2022
@AlexWaygood
AlexWaygood deleted the more-aliases branch August 5, 2022 12:03
@sobolevn sobolevn mentioned this pull request Nov 15, 2022
gh-99509: Add __class_getitem__ to multiprocessing.Queue python/cpython#99511
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Akuli Akuli Akuli left review comments
@srittau srittau srittau approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@AlexWaygood @srittau @Akuli
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.