GeezerVisonThe World Wide Web, restored to 800 pixels of glory. |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites |
GeezerVison is showing mail.python.org — Mailman 3 July 2015 - Python-Dev - python.org Python-Dev
Threads by month
July 2015
I'm back, I've re-read the PEP, and I've re-read the long thread with "(no
subject)".
I think Georg Brandl nailed it:
"""
*I like the "sequence and dict flattening" part of the PEP, mostly because
itis consistent and should be easy to understand, but the comprehension
syntaxenhancements seem to be bad for readability and "comprehending" what
the codedoes.The call syntax part is a mixed bag on the one hand it is nice
to be consistent with the extended possibilities in literals (flattening),
but on the other hand there would be small but annoying inconsistencies
anyways (e.g. the duplicate kwarg case above).*
"""
Greg Ewing followed up explaining that the inconsistency between dict
flattening and call syntax is inherent in the pre-existing different rules
for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1,
a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1,
**{'a': 1}) is a TypeError.)
For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations
for function calls proposed by the PEP is an easy +1 -- it's a
straightforward extension of the existing pattern, and anybody who knows
what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d,
**e) means shouldn't be hard either. Understanding the edge case for
duplicate keys with f(**d, **e) is a little harder, but the error messages
are pretty clear, and it is not a new edge case.
The sequence and dict flattening syntax proposals are also clean and
logical -- we already have *-unpacking on the receiving side, so allowing
*x in tuple expressions reads pretty naturally (and the similarity with *a
in argument lists certainly helps). From here, having [a, *x, b, *y] is
also natural, and then the extension to other displays is natural: {a, *x,
b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me.
So that leaves comprehensions. IIRC, during the development of the patch we
realized that f(*x for x in xs) is sufficiently ambiguous that we decided
to disallow it -- note that f(x for x in xs) is already somewhat of a
special case because an argument can only be a "bare" generator expression
if it is the only argument. The same reasoning doesn't apply (in that form)
to list, set and dict comprehensions -- while f(x for x in xs) is identical
in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x
for x in xs)] (that's a list of one element, and the element is a generator
expression).
The basic premise of this part of the proposal is that if you have a few
iterables, the new proposal (without comprehensions) lets you create a list
or generator expression that iterates over all of them, essentially
flattening them:
>>> xs = [1, 2, 3]
>>> ys = ['abc', 'def']
>>> zs = [99]
>>> [*xs, *ys, *zs]
[1, 2, 3, 'abc', 'def', 99]
>>>
But now suppose you have a list of iterables:
>>> xss = [[1, 2, 3], ['abc', 'def'], [99]]
>>> [*xss[0], *xss[1], *xss[2]]
[1, 2, 3, 'abc', 'def', 99]
>>>
Wouldn't it be nice if you could write the latter using a comprehension?
>>> xss = [[1, 2, 3], ['abc', 'def'], [99]]
>>> [*xs for xs in xss]
[1, 2, 3, 'abc', 'def', 99]
>>>
This is somewhat seductive, and the following is even nicer: the *xs
position may be an expression, e.g.:
>>> xss = [[1, 2, 3], ['abc', 'def'], [99]]
>>> [*xs[:2] for xs in xss]
[1, 2, 'abc', 'def', 99]
>>>
On the other hand, I had to explore the possibilities here by experimenting
in the interpreter, and I discovered some odd edge cases (e.g. you can
parenthesize the starred expression, but that seems a syntactic accident).
All in all I am personally +0 on the comprehension part of the PEP, and I
like that it provides a way to "flatten" a sequence of sequences, but I
think very few people in the thread have supported this part. Therefore I
would like to ask Neil to update the PEP and the patch to take out the
comprehension part, so that the two "easy wins" can make it into Python 3.5
(basically, I am accepting two-thirds of the PEP :-). There is some time
yet until alpha 2.
I would also like code reviewers (Benjamin?) to start reviewing the patch
<http://bugs.python.org/issue2292>, taking into account that the
comprehension part needs to be removed.
--
--Guido van Rossum (python.org/~guido)
It has been a while since I posted a copy of PEP 1 to the mailing
lists and newsgroups. I've recently done some updating of a few
sections, so in the interest of gaining wider community participation
in the Python development process, I'm posting the latest revision of
PEP 1 here. A version of the PEP is always available on-line at
http://www.python.org/peps/pep-0001.html
Enjoy,
-Barry
-------------------- snip snip --------------------
PEP: 1
Title: PEP Purpose and Guidelines
Version: $Revision: 1.36 $
Last-Modified: $Date: 2002/07/29 18:34:59 $
Author: Barry A. Warsaw, Jeremy Hylton
Status: Active
Type: Informational
Created: 13-Jun-2000
Post-History: 21-Mar-2001, 29-Jul-2002
What is a PEP?
PEP stands for Python Enhancement Proposal. A PEP is a design
document providing information to the Python community, or
describing a new feature for Python. The PEP should provide a
concise technical specification of the feature and a rationale for
the feature.
We intend PEPs to be the primary mechanisms for proposing new
features, for collecting community input on an issue, and for
documenting the design decisions that have gone into Python. The
PEP author is responsible for building consensus within the
community and documenting dissenting opinions.
Because the PEPs are maintained as plain text files under CVS
control, their revision history is the historical record of the
feature proposal[1].
Kinds of PEPs
There are two kinds of PEPs. A standards track PEP describes a
new feature or implementation for Python. An informational PEP
describes a Python design issue, or provides general guidelines or
information to the Python community, but does not propose a new
feature. Informational PEPs do not necessarily represent a Python
community consensus or recommendation, so users and implementors
are free to ignore informational PEPs or follow their advice.
PEP Work Flow
The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers
for each PEP and changes its status.
The PEP process begins with a new idea for Python. It is highly
recommended that a single PEP contain a single key proposal or new
idea. The more focussed the PEP, the more successfully it tends
to be. The PEP editor reserves the right to reject PEP proposals
if they appear too unfocussed or too broad. If in doubt, split
your PEP into several well-focussed ones.
Each PEP must have a champion -- someone who writes the PEP using
the style and format described below, shepherds the discussions in
the appropriate forums, and attempts to build community consensus
around the idea. The PEP champion (a.k.a. Author) should first
attempt to ascertain whether the idea is PEP-able. Small
enhancements or patches often don't need a PEP and can be injected
into the Python development work flow with a patch submission to
the SourceForge patch manager[2] or feature request tracker[3].
The PEP champion then emails the PEP editor <peps(a)python.org> with
a proposed title and a rough, but fleshed out, draft of the PEP.
This draft must be written in PEP style as described below.
If the PEP editor approves, he will assign the PEP a number, label
it as standards track or informational, give it status 'draft',
and create and check-in the initial draft of the PEP. The PEP
editor will not unreasonably deny a PEP. Reasons for denying PEP
status include duplication of effort, being technically unsound,
not providing proper motivation or addressing backwards
compatibility, or not in keeping with the Python philosophy. The
BDFL (Benevolent Dictator for Life, Guido van Rossum) can be
consulted during the approval phase, and is the final arbitrator
of the draft's PEP-ability.
If a pre-PEP is rejected, the author may elect to take the pre-PEP
to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org
mailing list) to help flesh it out, gain feedback and consensus
from the community at large, and improve the PEP for
re-submission.
The author of the PEP is then responsible for posting the PEP to
the community forums, and marshaling community support for it. As
updates are necessary, the PEP author can check in new versions if
they have CVS commit permissions, or can email new PEP versions to
the PEP editor for committing.
Standards track PEPs consists of two parts, a design document and
a reference implementation. The PEP should be reviewed and
accepted before a reference implementation is begun, unless a
reference implementation will aid people in studying the PEP.
Standards Track PEPs must include an implementation - in the form
of code, patch, or URL to same - before it can be considered
Final.
PEP authors are responsible for collecting community feedback on a
PEP before submitting it for review. A PEP that has not been
discussed on python-list(a)python.org and/or python-dev(a)python.org
will not be accepted. However, wherever possible, long open-ended
discussions on public mailing lists should be avoided. Strategies
to keep the discussions efficient include, setting up a separate
SIG mailing list for the topic, having the PEP author accept
private comments in the early design phases, etc. PEP authors
should use their discretion here.
Once the authors have completed a PEP, they must inform the PEP
editor that it is ready for review. PEPs are reviewed by the BDFL
and his chosen consultants, who may accept or reject a PEP or send
it back to the author(s) for revision.
Once a PEP has been accepted, the reference implementation must be
completed. When the reference implementation is complete and
accepted by the BDFL, the status will be changed to `Final.'
A PEP can also be assigned status `Deferred.' The PEP author or
editor can assign the PEP this status when no progress is being
made on the PEP. Once a PEP is deferred, the PEP editor can
re-assign it to draft status.
A PEP can also be `Rejected'. Perhaps after all is said and done
it was not a good idea. It is still important to have a record of
this fact.
PEPs can also be replaced by a different PEP, rendering the
original obsolete. This is intended for Informational PEPs, where
version 2 of an API can replace version 1.
PEP work flow is as follows:
Draft -> Accepted -> Final -> Replaced
^
+----> Rejected
v
Deferred
Some informational PEPs may also have a status of `Active' if they
are never meant to be completed. E.g. PEP 1.
What belongs in a successful PEP?
Each PEP should have the following parts:
1. Preamble -- RFC822 style headers containing meta-data about the
PEP, including the PEP number, a short descriptive title
(limited to a maximum of 44 characters), the names, and
optionally the contact info for each author, etc.
2. Abstract -- a short (~200 word) description of the technical
issue being addressed.
3. Copyright/public domain -- Each PEP must either be explicitly
labelled as placed in the public domain (see this PEP as an
example) or licensed under the Open Publication License[4].
4. Specification -- The technical specification should describe
the syntax and semantics of any new language feature. The
specification should be detailed enough to allow competing,
interoperable implementations for any of the current Python
platforms (CPython, JPython, Python .NET).
5. Motivation -- The motivation is critical for PEPs that want to
change the Python language. It should clearly explain why the
existing language specification is inadequate to address the
problem that the PEP solves. PEP submissions without
sufficient motivation may be rejected outright.
6. Rationale -- The rationale fleshes out the specification by
describing what motivated the design and why particular design
decisions were made. It should describe alternate designs that
were considered and related work, e.g. how the feature is
supported in other languages.
The rationale should provide evidence of consensus within the
community and discuss important objections or concerns raised
during discussion.
7. Backwards Compatibility -- All PEPs that introduce backwards
incompatibilities must include a section describing these
incompatibilities and their severity. The PEP must explain how
the author proposes to deal with these incompatibilities. PEP
submissions without a sufficient backwards compatibility
treatise may be rejected outright.
8. Reference Implementation -- The reference implementation must
be completed before any PEP is given status 'Final,' but it
need not be completed before the PEP is accepted. It is better
to finish the specification and rationale first and reach
consensus on it before writing code.
The final implementation must include test code and
documentation appropriate for either the Python language
reference or the standard library reference.
PEP Template
PEPs are written in plain ASCII text, and should adhere to a
rigid style. There is a Python script that parses this style and
converts the plain text PEP to HTML for viewing on the web[5].
PEP 9 contains a boilerplate[7] template you can use to get
started writing your PEP.
Each PEP must begin with an RFC822 style header preamble. The
headers must appear in the following order. Headers marked with
`*' are optional and are described below. All other headers are
required.
PEP: <pep number>
Title: <pep title>
Version: <cvs version string>
Last-Modified: <cvs date string>
Author: <list of authors' real names and optionally, email addrs>
* Discussions-To: <email address>
Status: <Draft | Active | Accepted | Deferred | Final | Replaced>
Type: <Informational | Standards Track>
* Requires: <pep numbers>
Created: <date created on, in dd-mmm-yyyy format>
* Python-Version: <version number>
Post-History: <dates of postings to python-list and python-dev>
* Replaces: <pep number>
* Replaced-By: <pep number>
The Author: header lists the names and optionally, the email
addresses of all the authors/owners of the PEP. The format of the
author entry should be
address(a)dom.ain (Random J. User)
if the email address is included, and just
Random J. User
if the address is not given. If there are multiple authors, each
should be on a separate line following RFC 822 continuation line
conventions. Note that personal email addresses in PEPs will be
obscured as a defense against spam harvesters.
Standards track PEPs must have a Python-Version: header which
indicates the version of Python that the feature will be released
with. Informational PEPs do not need a Python-Version: header.
While a PEP is in private discussions (usually during the initial
Draft phase), a Discussions-To: header will indicate the mailing
list or URL where the PEP is being discussed. No Discussions-To:
header is necessary if the PEP is being discussed privately with
the author, or on the python-list or python-dev email mailing
lists. Note that email addresses in the Discussions-To: header
will not be obscured.
Created: records the date that the PEP was assigned a number,
while Post-History: is used to record the dates of when new
versions of the PEP are posted to python-list and/or python-dev.
Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001.
PEPs may have a Requires: header, indicating the PEP numbers that
this PEP depends on.
PEPs may also have a Replaced-By: header indicating that a PEP has
been rendered obsolete by a later document; the value is the
number of the PEP that replaces the current document. The newer
PEP must have a Replaces: header containing the number of the PEP
that it rendered obsolete.
PEP Formatting Requirements
PEP headings must begin in column zero and the initial letter of
each word must be capitalized as in book titles. Acronyms should
be in all capitals. The body of each section must be indented 4
spaces. Code samples inside body sections should be indented a
further 4 spaces, and other indentation can be used as required to
make the text readable. You must use two blank lines between the
last line of a section's body and the next section heading.
You must adhere to the Emacs convention of adding two spaces at
the end of every sentence. You should fill your paragraphs to
column 70, but under no circumstances should your lines extend
past column 79. If your code samples spill over column 79, you
should rewrite them.
Tab characters must never appear in the document at all. A PEP
should include the standard Emacs stanza included by example at
the bottom of this PEP.
A PEP must contain a Copyright section, and it is strongly
recommended to put the PEP in the public domain.
When referencing an external web page in the body of a PEP, you
should include the title of the page in the text, with a
footnote reference to the URL. Do not include the URL in the body
text of the PEP. E.g.
Refer to the Python Language web site [1] for more details.
...
[1] http://www.python.org
When referring to another PEP, include the PEP number in the body
text, such as "PEP 1". The title may optionally appear. Add a
footnote reference that includes the PEP's title and author. It
may optionally include the explicit URL on a separate line, but
only in the References section. Note that the pep2html.py script
will calculate URLs automatically, e.g.:
...
Refer to PEP 1 [7] for more information about PEP style
...
References
[7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton
http://www.python.org/peps/pep-0001.html
If you decide to provide an explicit URL for a PEP, please use
this as the URL template:
http://www.python.org/peps/pep-xxxx.html
PEP numbers in URLs must be padded with zeros from the left, so as
to be exactly 4 characters wide, however PEP numbers in text are
never padded.
Reporting PEP Bugs, or Submitting PEP Updates
How you report a bug, or submit a PEP update depends on several
factors, such as the maturity of the PEP, the preferences of the
PEP author, and the nature of your comments. For the early draft
stages of the PEP, it's probably best to send your comments and
changes directly to the PEP author. For more mature, or finished
PEPs you may want to submit corrections to the SourceForge bug
manager[6] or better yet, the SourceForge patch manager[2] so that
your changes don't get lost. If the PEP author is a SF developer,
assign the bug/patch to him, otherwise assign it to the PEP
editor.
When in doubt about where to send your changes, please check first
with the PEP author and/or PEP editor.
PEP authors who are also SF committers, can update the PEPs
themselves by using "cvs commit" to commit their changes.
Remember to also push the formatted PEP text out to the web by
doing the following:
% python pep2html.py -i NUM
where NUM is the number of the PEP you want to push out. See
% python pep2html.py --help
for details.
Transferring PEP Ownership
It occasionally becomes necessary to transfer ownership of PEPs to
a new champion. In general, we'd like to retain the original
author as a co-author of the transferred PEP, but that's really up
to the original author. A good reason to transfer ownership is
because the original author no longer has the time or interest in
updating it or following through with the PEP process, or has
fallen off the face of the 'net (i.e. is unreachable or not
responding to email). A bad reason to transfer ownership is
because you don't agree with the direction of the PEP. We try to
build consensus around a PEP, but if that's not possible, you can
always submit a competing PEP.
If you are interested assuming ownership of a PEP, send a message
asking to take over, addressed to both the original author and the
PEP editor <peps(a)python.org>. If the original author doesn't
respond to email in a timely manner, the PEP editor will make a
unilateral decision (it's not like such decisions can be
reversed. :).
References and Footnotes
[1] This historical record is available by the normal CVS commands
for retrieving older revisions. For those without direct access
to the CVS tree, you can browse the current and past PEP revisions
via the SourceForge web site at
http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=…
[2] http://sourceforge.net/tracker/?group_id=5470&atid=305470
[3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse
[4] http://www.opencontent.org/openpub/
[5] The script referred to here is pep2html.py, which lives in
the same directory in the CVS tree as the PEPs themselves.
Try "pep2html.py --help" for details.
The URL for viewing PEPs on the web is
http://www.python.org/peps/
[6] http://sourceforge.net/tracker/?group_id=5470&atid=305470
[7] PEP 9, Sample PEP Template
http://www.python.org/peps/pep-0009.html
Copyright
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:
Just so people aren't caught unawares, it is very unlikely that I will have
time to be the final editor on "What's New for 3.5" they way I was for 3.3 and
3.4. I've tried to encourage people to keep What's New up to date, but
*someone* should make a final editing pass. Ideally they'd do at least the
research Serhiy did last year on checking that there's a mention for all of the
versionadded and versionchanged 3.5's in the docs. Even better would be to
review the NEWS and/or commit history...but *that* is a really big job these
days....
--David
On 21 July 2015 at 19:40, Nick Coghlan <ncoghlan(a)gmail.com> wrote:
> All of this is why the chart that I believe should be worrying people
> is the topmost one on this page:
> http://bugs.python.org/issue?@template=stats
>
> Both the number of open issues and the number of open issues with
> patches are steadily trending upwards. That means the bottleneck in
> the current process *isn't* getting patches written in the first
> place, it's getting them up to the appropriate standards and applied.
> Yet the answer to the problem isn't a simple "recruit more core
> developers", as the existing core developers are *also* the bottleneck
> in the review and mentoring process for *new* core developers.
Those charts doesn't show patches in 'commit-review' -
http://bugs.python.org/issue?%40columns=title&%40columns=id&stage=5&%40colu…
There are only 45 of those patches.
AIUI - and I'm very new to core here - anyone in triagers can get
patches up to commit-review status.
I think we should set a goal to keep inventory low here - e.g. review
and either bounce back to patch review, or commit, in less than a
month. Now - a month isn't super low, but we have lots of stuff
greater than a month.
For my part, I'm going to pick up more or less one thing a day and
review it, but I think it would be great if other committers were to
also to do this: if we had 5 of us doing 1 a day, I think we'd burn
down this 45 patch backlog rapidly without significant individual
cost. At which point, we can fairly say to folk doing triage that
we're ready for patches :)
-Rob
--
Robert Collins <rbtcollins(a)hp.com>
Distinguished Technologist
HP Converged Cloud
Hi,
Another summer with another EuroPython, which means its time again to try to revive PEP 447…
I’ve just pushes a minor update to the PEP and would like to get some feedback on this, arguably fairly esoteric, PEP.
The PEP proposes to to replace direct access to the class __dict__ in object.__getattribute__ and super.__getattribute__ by calls to a new special method to give the metaclass more control over attribute lookup, especially for access using a super() object. This is needed for classes that cannot store (all) descriptors in the class dict for some reason, see the PEP text for a real-world example of that.
Regards,
Ronald
The PEP text (with an outdated section with benchmarks removed):
PEP: 447
Title: Add __getdescriptor__ method to metaclass
Version: $Revision$
Last-Modified: $Date$
Author: Ronald Oussoren <ronaldoussoren(a)mac.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Jun-2013
Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013, 22-Jul-2015
Abstract
========
Currently ``object.__getattribute__`` and ``super.__getattribute__`` peek
in the ``__dict__`` of classes on the MRO for a class when looking for
an attribute. This PEP adds an optional ``__getdescriptor__`` method to
a metaclass that replaces this behavior and gives more control over attribute
lookup, especially when using a `super`_ object.
That is, the MRO walking loop in ``_PyType_Lookup`` and
``super.__getattribute__`` gets changed from::
def lookup(mro_list, name):
for cls in mro_list:
if name in cls.__dict__:
return cls.__dict__
return NotFound
to::
def lookup(mro_list, name):
for cls in mro_list:
try:
return cls.__getdescriptor__(name)
except AttributeError:
pass
return NotFound
The default implemention of ``__getdescriptor__`` looks in the class
dictionary::
class type:
def __getdescriptor__(cls, name):
try:
return cls.__dict__[name]
except KeyError:
raise AttributeError(name) from None
Rationale
=========
It is currently not possible to influence how the `super class`_ looks
up attributes (that is, ``super.__getattribute__`` unconditionally
peeks in the class ``__dict__``), and that can be problematic for
dynamic classes that can grow new methods on demand.
The ``__getdescriptor__`` method makes it possible to dynamically add
attributes even when looking them up using the `super class`_.
The new method affects ``object.__getattribute__`` (and
`PyObject_GenericGetAttr`_) as well for consistency and to have a single
place to implement dynamic attribute resolution for classes.
Background
----------
The current behavior of ``super.__getattribute__`` causes problems for
classes that are dynamic proxies for other (non-Python) classes or types,
an example of which is `PyObjC`_. PyObjC creates a Python class for every
class in the Objective-C runtime, and looks up methods in the Objective-C
runtime when they are used. This works fine for normal access, but doesn't
work for access with `super`_ objects. Because of this PyObjC currently
includes a custom `super`_ that must be used with its classes, as well as
completely reimplementing `PyObject_GenericGetAttr`_ for normal attribute
access.
The API in this PEP makes it possible to remove the custom `super`_ and
simplifies the implementation because the custom lookup behavior can be
added in a central location.
.. note::
`PyObjC`_ cannot precalculate the contents of the class ``__dict__``
because Objective-C classes can grow new methods at runtime. Furthermore
Objective-C classes tend to contain a lot of methods while most Python
code will only use a small subset of them, this makes precalculating
unnecessarily expensive.
The superclass attribute lookup hook
====================================
Both ``super.__getattribute__`` and ``object.__getattribute__`` (or
`PyObject_GenericGetAttr`_ and in particular ``_PyType_Lookup`` in C code)
walk an object's MRO and currently peek in the class' ``__dict__`` to look up
attributes.
With this proposal both lookup methods no longer peek in the class ``__dict__``
but call the special method ``__getdescriptor__``, which is a slot defined
on the metaclass. The default implementation of that method looks
up the name the class ``__dict__``, which means that attribute lookup is
unchanged unless a metatype actually defines the new special method.
Aside: Attribute resolution algorithm in Python
-----------------------------------------------
The attribute resolution proces as implemented by ``object.__getattribute__``
(or PyObject_GenericGetAttr`` in CPython's implementation) is fairly
straightforward, but not entirely so without reading C code.
The current CPython implementation of object.__getattribute__ is basicly
equivalent to the following (pseudo-) Python code (excluding some house keeping and speed tricks)::
def _PyType_Lookup(tp, name):
mro = tp.mro()
assert isinstance(mro, tuple)
for base in mro:
assert isinstance(base, type)
# PEP 447 will change these lines:
try:
return base.__dict__[name]
except KeyError:
pass
return None
class object:
def __getattribute__(self, name):
assert isinstance(name, str)
tp = type(self)
descr = _PyType_Lookup(tp, name)
f = None
if descr is not None:
f = descr.__get__
if f is not None and descr.__set__ is not None:
# Data descriptor
return f(descr, self, type(self))
dict = self.__dict__
if dict is not None:
try:
return self.__dict__[name]
except KeyError:
pass
if f is not None:
# Non-data descriptor
return f(descr, self, type(self))
if descr is not None:
# Regular class attribute
return descr
raise AttributeError(name)
class super:
def __getattribute__(self, name):
assert isinstance(name, unicode)
if name != '__class__':
starttype = self.__self_type__
mro = startype.mro()
try:
idx = mro.index(self.__thisclass__)
except ValueError:
pass
else:
for base in mro[idx+1:]:
# PEP 447 will change these lines:
try:
descr = base.__dict__[name]
except KeyError:
continue
f = descr.__get__
if f is not None:
return f(descr,
None if (self.__self__ is self.__self_type__) else self.__self__,
starttype)
else:
return descr
return object.__getattribute__(self, name)
This PEP should change the dict lookup at the lines starting at "# PEP 447" with
a method call to perform the actual lookup, making is possible to affect that
lookup both for normal attribute access and access through the `super proxy`_.
Note that specific classes can already completely override the default
behaviour by implementing their own ``__getattribute__`` slot (with or without
calling the super class implementation).
In Python code
--------------
A meta type can define a method ``__getdescriptor__`` that is called during
attribute resolution by both ``super.__getattribute__``
and ``object.__getattribute``::
class MetaType(type):
def __getdescriptor__(cls, name):
try:
return cls.__dict__[name]
except KeyError:
raise AttributeError(name) from None
The ``__getdescriptor__`` method has as its arguments a class (which is an
instance of the meta type) and the name of the attribute that is looked up.
It should return the value of the attribute without invoking descriptors,
and should raise `AttributeError`_ when the name cannot be found.
The `type`_ class provides a default implementation for ``__getdescriptor__``,
that looks up the name in the class dictionary.
Example usage
.............
The code below implements a silly metaclass that redirects attribute lookup to
uppercase versions of names::
class UpperCaseAccess (type):
def __getdescriptor__(cls, name):
try:
return cls.__dict__[name.upper()]
except KeyError:
raise AttributeError(name) from None
class SillyObject (metaclass=UpperCaseAccess):
def m(self):
return 42
def M(self):
return "fourtytwo"
obj = SillyObject()
assert obj.m() == "fortytwo"
As mentioned earlier in this PEP a more realistic use case of this
functionallity is a ``__getdescriptor__`` method that dynamicly populates the
class ``__dict__`` based on attribute access, primarily when it is not
possible to reliably keep the class dict in sync with its source, for example
because the source used to populate ``__dict__`` is dynamic as well and does
not have triggers that can be used to detect changes to that source.
An example of that are the class bridges in PyObjC: the class bridge is a
Python object (class) that represents an Objective-C class and conceptually
has a Python method for every Objective-C method in the Objective-C class.
As with Python it is possible to add new methods to an Objective-C class, or
replace existing ones, and there are no callbacks that can be used to detect
this.
In C code
---------
A new slot ``tp_getdescriptor`` is added to the ``PyTypeObject`` struct, this
slot corresponds to the ``__getdescriptor__`` method on `type`_.
The slot has the following prototype::
PyObject* (*getdescriptorfunc)(PyTypeObject* cls, PyObject* name);
This method should lookup *name* in the namespace of *cls*, without looking at
superclasses, and should not invoke descriptors. The method returns ``NULL``
without setting an exception when the *name* cannot be found, and returns a
new reference otherwise (not a borrowed reference).
Use of this hook by the interpreter
-----------------------------------
The new method is required for metatypes and as such is defined on `type_`.
Both ``super.__getattribute__`` and
``object.__getattribute__``/`PyObject_GenericGetAttr`_
(through ``_PyType_Lookup``) use the this ``__getdescriptor__`` method when
walking the MRO.
Other changes to the implementation
-----------------------------------
The change for `PyObject_GenericGetAttr`_ will be done by changing the private
function ``_PyType_Lookup``. This currently returns a borrowed reference, but
must return a new reference when the ``__getdescriptor__`` method is present.
Because of this ``_PyType_Lookup`` will be renamed to ``_PyType_LookupName``,
this will cause compile-time errors for all out-of-tree users of this
private API.
The attribute lookup cache in ``Objects/typeobject.c`` is disabled for classes
that have a metaclass that overrides ``__getdescriptor__``, because using the
cache might not be valid for such classes.
Impact of this PEP on introspection
-----------------------------------
Use of the method introduced in this PEP can affect introspection of classes
with a metaclass that uses a custom ``__getdescriptor__`` method. This section
lists those changes.
The items listed below are only affected by custom ``__getdescriptor__``
methods, the default implementation for ``object`` won't cause problems
because that still only uses the class ``__dict__`` and won't cause visible
changes to the visible behaviour of the ``object.__getattribute__``.
* ``dir`` might not show all attributes
As with a custom ``__getattribute__`` method `dir()`_ might not see all
(instance) attributes when using the ``__getdescriptor__()`` method to
dynamicly resolve attributes.
The solution for that is quite simple: classes using ``__getdescriptor__``
should also implement `__dir__()`_ if they want full support for the builtin
`dir()`_ function.
* ``inspect.getattr_static`` might not show all attributes
The function ``inspect.getattr_static`` intentionally does not invoke
``__getattribute__`` and descriptors to avoid invoking user code during
introspection with this function. The ``__getdescriptor__`` method will also
be ignored and is another way in which the result of ``inspect.getattr_static``
can be different from that of ``builtin.getattr``.
* ``inspect.getmembers`` and ``inspect.get_class_attrs``
Both of these functions directly access the class __dict__ of classes along
the MRO, and hence can be affected by a custom ``__getdescriptor__`` method.
**TODO**: I haven't fully worked out what the impact of this is, and if there
are mitigations for those using either updates to these functions, or
additional methods that users should implement to be fully compatible with
these functions.
One possible mitigation is to have a custom ``__getattribute__`` for these
classes that fills ``__dict__`` before returning and and defers to the
default implementation for other attributes.
* Direct introspection of the class ``__dict__``
Any code that directly access the class ``__dict__`` for introspection
can be affected by a custom ``__getdescriptor__`` method.
Performance impact
------------------
**WARNING**: The benchmark results in this section are old, and will be updated
when I've ported the patch to the current trunk. I don't expect significant
changes to the results in this section.
[snipped]
Alternative proposals
---------------------
``__getattribute_super__``
..........................
An earlier version of this PEP used the following static method on classes::
def __getattribute_super__(cls, name, object, owner): pass
This method performed name lookup as well as invoking descriptors and was
necessarily limited to working only with ``super.__getattribute__``.
Reuse ``tp_getattro``
.....................
It would be nice to avoid adding a new slot, thus keeping the API simpler and
easier to understand. A comment on `Issue 18181`_ asked about reusing the
``tp_getattro`` slot, that is super could call the ``tp_getattro`` slot of all
methods along the MRO.
That won't work because ``tp_getattro`` will look in the instance
``__dict__`` before it tries to resolve attributes using classes in the MRO.
This would mean that using ``tp_getattro`` instead of peeking the class
dictionaries changes the semantics of the `super class`_.
Alternate placement of the new method
.....................................
This PEP proposes to add ``__getdescriptor__`` as a method on the metaclass.
An alternative would be to add it as a class method on the class itself
(simular to how ``__new__`` is a `staticmethod`_ of the class and not a method
of the metaclass).
The two are functionally equivalent, and there's something to be said about
not requiring the use of a meta class.
References
==========
* `Issue 18181`_ contains an out of date prototype implementation
Copyright
=========
This document has been placed in the public domain.
.. _`Issue 18181`: http://bugs.python.org/issue18181
.. _`super class`: http://docs.python.org/3/library/functions.html#super
.. _`super proxy`: http://docs.python.org/3/library/functions.html#super
.. _`super`: http://docs.python.org/3/library/functions.html#super
.. _`dir()`: http://docs.python.org/3/library/functions.html#dir
.. _`staticmethod`: http://docs.python.org/3/library/functions.html#staticmethod
.. _`__dir__()`: https://docs.python.org/3/reference/datamodel.html#object.__dir__
.. _`NotImplemented`: http://docs.python.org/3/library/constants.html#NotImplemented
.. _`PyObject_GenericGetAttr`: http://docs.python.org/3/c-api/object.html#PyObject_GenericGetAttr
.. _`type`: http://docs.python.org/3/library/functions.html#type
.. _`AttributeError`: http://docs.python.org/3/library/exceptions.html#AttributeError
.. _`PyObjC`: http://pyobjc.sourceforge.net/
.. _`classmethod`: http://docs.python.org/3/library/functions.html#classmethod
There are over 400 issues on the bug tracker that have not had a
response to the initial message, roughly half of these within the last
eight months alone. Is there a (relatively) simple way that we can
share these out between us to sort those that are likely to need dealing
with in the medium to longer term, from the simple short term ones, e.g
very easy typo fixes?
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
ACTIVITY SUMMARY (2015-07-24 - 2015-07-31)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open 4961 ( +4)
closed 31563 (+52)
total 36524 (+56)
Open issues with patches: 2246
Issues opened (36)
==================
#23447: Import fails when doing a circular import involving an `import
http://bugs.python.org/issue23447 reopened by rbcollins
#24476: Statically link vcruntime140.dll
http://bugs.python.org/issue24476 reopened by steve.dower
#24709: Unix build uses '-Wno-unused-result', which icc doesn't recogn
http://bugs.python.org/issue24709 opened by zach.ware
#24710: Class name hardcoded in TracebackException.from_exception()
http://bugs.python.org/issue24710 opened by berker.peksag
#24711: Document getpass.getpass behavior on ^C
http://bugs.python.org/issue24711 opened by untitaker
#24712: Docs page's sidebar vibrates on mouse wheel scroll on Chrome.
http://bugs.python.org/issue24712 opened by Biwin John
#24715: Sorting HOW TO: bad example for reverse sort stability
http://bugs.python.org/issue24715 opened by jwilk
#24718: Specify interpreter when running in IDLE
http://bugs.python.org/issue24718 opened by Daniel Pope
#24720: Python install help
http://bugs.python.org/issue24720 opened by node
#24724: Element.findall documentation misleading
http://bugs.python.org/issue24724 opened by Eric S
#24725: test_socket testFDPassEmpty fails on OS X 10.11 DP with "Canno
http://bugs.python.org/issue24725 opened by ned.deily
#24726: OrderedDict has strange behaviour when dict.__setitem__ is use
http://bugs.python.org/issue24726 opened by Mark.Shannon
#24727: Expand readline module
http://bugs.python.org/issue24727 opened by Barney Stratford
#24731: Incorrect assert in str_subtype_new
http://bugs.python.org/issue24731 opened by Kevin Modzelewski
#24732: 3.5.0b3 Windows accept() on unready non-blocking socket raises
http://bugs.python.org/issue24732 opened by bryangeneolson
#24733: Logically Dead Code
http://bugs.python.org/issue24733 opened by pankaj.s01
#24736: argparse add_mutually_exclusive_group do not print help
http://bugs.python.org/issue24736 opened by Alexandre.Badez
#24739: allow argparse.FileType to accept newline argument
http://bugs.python.org/issue24739 opened by garyp
#24740: make patchcheck doesn't detect changes if commit is done first
http://bugs.python.org/issue24740 opened by rbcollins
#24743: Make _PyTraceback_Add public
http://bugs.python.org/issue24743 opened by mic-e
#24744: Lack of type checks in pkgutil.walk_packages and friends
http://bugs.python.org/issue24744 opened by sleepycal
#24745: Better default font for editor
http://bugs.python.org/issue24745 opened by markroseman
#24746: doctest 'fancy diff' formats incorrectly strip trailing whites
http://bugs.python.org/issue24746 opened by r.david.murray
#24747: ctypes silently truncates ints larger than C int
http://bugs.python.org/issue24747 opened by encukou
#24748: Change of behavior for importlib between 3.4 and 3.5 with DLL
http://bugs.python.org/issue24748 opened by ebfortin
#24750: IDLE: Cosmetic improvements for main window
http://bugs.python.org/issue24750 opened by markroseman
#24751: regrtest/buildbot: test run marked as failure even when re-run
http://bugs.python.org/issue24751 opened by zach.ware
#24752: SystemError when importing from a non-package directory
http://bugs.python.org/issue24752 opened by fossilet
#24754: argparse add_argument with action="store_true", type=bool shou
http://bugs.python.org/issue24754 opened by dbagnall
#24755: asyncio.wrap_future undocumented
http://bugs.python.org/issue24755 opened by wodny
#24756: doctest run_docstring_examples does have an obvious utility
http://bugs.python.org/issue24756 opened by r.david.murray
#24757: Installing Py on Windows: Need to restart or logout for path t
http://bugs.python.org/issue24757 opened by John Palermo
#24758: unittest.mock.Mock's new "unsafe" feature needs a better error
http://bugs.python.org/issue24758 opened by Randy Syring
#24760: IDLE settings dialog shouldn't be modal
http://bugs.python.org/issue24760 opened by markroseman
#24761: ERROR: test_dh_params (test.test_ssl.ThreadedTests)
http://bugs.python.org/issue24761 opened by cloud2han9
#24762: Branchless, vectorizable frozen set hash
http://bugs.python.org/issue24762 opened by rhettinger
Most recent 15 issues with no replies (15)
==========================================
#24762: Branchless, vectorizable frozen set hash
http://bugs.python.org/issue24762
#24761: ERROR: test_dh_params (test.test_ssl.ThreadedTests)
http://bugs.python.org/issue24761
#24755: asyncio.wrap_future undocumented
http://bugs.python.org/issue24755
#24746: doctest 'fancy diff' formats incorrectly strip trailing whites
http://bugs.python.org/issue24746
#24743: Make _PyTraceback_Add public
http://bugs.python.org/issue24743
#24740: make patchcheck doesn't detect changes if commit is done first
http://bugs.python.org/issue24740
#24739: allow argparse.FileType to accept newline argument
http://bugs.python.org/issue24739
#24736: argparse add_mutually_exclusive_group do not print help
http://bugs.python.org/issue24736
#24733: Logically Dead Code
http://bugs.python.org/issue24733
#24731: Incorrect assert in str_subtype_new
http://bugs.python.org/issue24731
#24727: Expand readline module
http://bugs.python.org/issue24727
#24718: Specify interpreter when running in IDLE
http://bugs.python.org/issue24718
#24715: Sorting HOW TO: bad example for reverse sort stability
http://bugs.python.org/issue24715
#24711: Document getpass.getpass behavior on ^C
http://bugs.python.org/issue24711
#24709: Unix build uses '-Wno-unused-result', which icc doesn't recogn
http://bugs.python.org/issue24709
Most recent 15 issues waiting for review (15)
=============================================
#24762: Branchless, vectorizable frozen set hash
http://bugs.python.org/issue24762
#24756: doctest run_docstring_examples does have an obvious utility
http://bugs.python.org/issue24756
#24751: regrtest/buildbot: test run marked as failure even when re-run
http://bugs.python.org/issue24751
#24750: IDLE: Cosmetic improvements for main window
http://bugs.python.org/issue24750
#24746: doctest 'fancy diff' formats incorrectly strip trailing whites
http://bugs.python.org/issue24746
#24733: Logically Dead Code
http://bugs.python.org/issue24733
#24724: Element.findall documentation misleading
http://bugs.python.org/issue24724
#24715: Sorting HOW TO: bad example for reverse sort stability
http://bugs.python.org/issue24715
#24710: Class name hardcoded in TracebackException.from_exception()
http://bugs.python.org/issue24710
#24698: get_externals.bat script fails
http://bugs.python.org/issue24698
#24693: zipfile: change RuntimeError to more appropriate exception typ
http://bugs.python.org/issue24693
#24692: types.coroutines() idempotence documentation
http://bugs.python.org/issue24692
#24682: Add Quick Start: Communications section to devguide
http://bugs.python.org/issue24682
#24674: pyclbr not recursively showing classes in packages
http://bugs.python.org/issue24674
#24673: distutils/_msvccompiler does not remove /DLL during link(CComp
http://bugs.python.org/issue24673
Top 10 most discussed issues (10)
=================================
#24667: OrderedDict.popitem()/__str__() raises KeyError
http://bugs.python.org/issue24667 14 msgs
#24748: Change of behavior for importlib between 3.4 and 3.5 with DLL
http://bugs.python.org/issue24748 13 msgs
#24720: Python install help
http://bugs.python.org/issue24720 12 msgs
#24750: IDLE: Cosmetic improvements for main window
http://bugs.python.org/issue24750 12 msgs
#18378: locale.getdefaultlocale() fails on Mac OS X with default langu
http://bugs.python.org/issue18378 8 msgs
#19450: Bug in sqlite in Windows binaries
http://bugs.python.org/issue19450 7 msgs
#24732: 3.5.0b3 Windows accept() on unready non-blocking socket raises
http://bugs.python.org/issue24732 7 msgs
#20059: Inconsistent urlparse/urllib.parse handling of invalid port va
http://bugs.python.org/issue20059 6 msgs
#23496: Steps for Android Native Build of Python 3.4.2
http://bugs.python.org/issue23496 6 msgs
#24651: Mock.assert* API is in user namespace
http://bugs.python.org/issue24651 6 msgs
Issues closed (52)
==================
#2091: file accepts 'rU+' as a mode
http://bugs.python.org/issue2091 closed by rbcollins
#12160: codecs doc: what is StreamCodec?
http://bugs.python.org/issue12160 closed by berker.peksag
#13884: IDLE: Remove tear-off menu feature
http://bugs.python.org/issue13884 closed by terry.reedy
#17527: PATCH as valid request method in wsgiref.validator
http://bugs.python.org/issue17527 closed by rbcollins
#17991: ctypes.c_char gives a misleading error when passed a one-chara
http://bugs.python.org/issue17991 closed by Steven.Barker
#20051: PA-RISC buildbot: compiler cannot create executables
http://bugs.python.org/issue20051 closed by zach.ware
#20366: SQLite FTS (full text search)
http://bugs.python.org/issue20366 closed by mark
#20544: Use specific asserts in operator tests
http://bugs.python.org/issue20544 closed by python-dev
#20551: Use specific asserts in decimal tests
http://bugs.python.org/issue20551 closed by skrah
#21657: pip.get_installed_distributions() Does not return packages in
http://bugs.python.org/issue21657 closed by zach.ware
#21697: shutil.copytree() handles symbolic directory incorrectly
http://bugs.python.org/issue21697 closed by berker.peksag
#21750: mock_open data is visible only once for the life of the class
http://bugs.python.org/issue21750 closed by rbcollins
#23254: Document how to close the TCPServer listening socket
http://bugs.python.org/issue23254 closed by rbcollins
#23319: Missing SWAP_INT in I_set_sw
http://bugs.python.org/issue23319 closed by haypo
#23426: run_setup is broken in distutils
http://bugs.python.org/issue23426 closed by rbcollins
#23441: rlcompleter: tab on empty prefix => insert spaces
http://bugs.python.org/issue23441 closed by berker.peksag
#23589: Redundant sentence in FAQ
http://bugs.python.org/issue23589 closed by rbcollins
#23779: imaplib authenticate raises TypeError if authenticator tries t
http://bugs.python.org/issue23779 closed by rbcollins
#23899: HTTP regression in distutils uploads to chishop
http://bugs.python.org/issue23899 closed by jason.coombs
#23931: Update DevGuide link in Quickstart Step 1
http://bugs.python.org/issue23931 closed by berker.peksag
#24109: Documentation for difflib uses optparse
http://bugs.python.org/issue24109 closed by berker.peksag
#24279: Update test_base64 to use test.support.script_helper
http://bugs.python.org/issue24279 closed by berker.peksag
#24360: improve argparse.Namespace __repr__ for invalid identifiers.
http://bugs.python.org/issue24360 closed by berker.peksag
#24420: Documentation regressions from adding subprocess.run()
http://bugs.python.org/issue24420 closed by berker.peksag
#24603: Update OpenSSL to 1.0.2d in Windows and OS X installer
http://bugs.python.org/issue24603 closed by ned.deily
#24613: array.fromstring Use After Free
http://bugs.python.org/issue24613 closed by serhiy.storchaka
#24621: zipfile.BadZipFile: File is not a zip file
http://bugs.python.org/issue24621 closed by serhiy.storchaka
#24645: logging.handlers.QueueHandler should not lock when handling a
http://bugs.python.org/issue24645 closed by vinay.sajip
#24681: Put most likely test first in set_add_entry()
http://bugs.python.org/issue24681 closed by rhettinger
#24683: Type confusion in json encoding
http://bugs.python.org/issue24683 closed by serhiy.storchaka
#24708: strop.replace Integer Overflow
http://bugs.python.org/issue24708 closed by python-dev
#24713: Import docs reference the deprecated imp.reload
http://bugs.python.org/issue24713 closed by berker.peksag
#24714: Crash with string_at(None)
http://bugs.python.org/issue24714 closed by haypo
#24716: Multiple fdopen() on mkstemp() descriptor crashes py27 interpr
http://bugs.python.org/issue24716 closed by Thomas Krijnen
#24717: python logging handler not used when added after a process is
http://bugs.python.org/issue24717 closed by vinay.sajip
#24719: Resourc Leak in cPickle Module
http://bugs.python.org/issue24719 closed by python-dev
#24721: The result of calling dict.* methods on OrderedDict is undefin
http://bugs.python.org/issue24721 closed by rhettinger
#24722: Python install help
http://bugs.python.org/issue24722 closed by zach.ware
#24723: The new typing module is not documented :-(
http://bugs.python.org/issue24723 closed by berker.peksag
#24728: Build fails when threads are disabled
http://bugs.python.org/issue24728 closed by berker.peksag
#24729: Input and Output tutorial erroneously references default encod
http://bugs.python.org/issue24729 closed by jason.coombs
#24730: Returning a float via a method yields in an approximate value
http://bugs.python.org/issue24730 closed by zach.ware
#24734: Dereferencing a null returning value
http://bugs.python.org/issue24734 closed by python-dev
#24735: Invalid access in combinations_with_replacement()
http://bugs.python.org/issue24735 closed by rhettinger
#24737: Doc issue: Python 3.6 doesn't compile on Visual Studio 2010 an
http://bugs.python.org/issue24737 closed by steve.dower
#24738: os.link() issues on Windows & ReFS
http://bugs.python.org/issue24738 closed by r.david.murray
#24741: Hangs and errors while testing on Ubuntu/Intel
http://bugs.python.org/issue24741 closed by r.david.murray
#24742: type(dict with docstring) returns set instead of dict
http://bugs.python.org/issue24742 closed by serhiy.storchaka
#24749: Which fastest python-mysql connector?
http://bugs.python.org/issue24749 closed by haypo
#24753: function to get completion install location for setup.py?
http://bugs.python.org/issue24753 closed by r.david.murray
#24759: Idle: require 8.5 / ttk
http://bugs.python.org/issue24759 closed by terry.reedy
#24763: asyncio.BaseSubprocessTransport triggers an unavoidable Resour
http://bugs.python.org/issue24763 closed by haypo
On 15-04-15, Akira Li <4kir4.1i(a)gmail.com> wrote:
> Isaac Schwabacher <ischwabacher(a)wisc.edu> writes:
>
> > On 15-04-15, Akira Li <4kir4.1i(a)gmail.com> wrote:
> >> Isaac Schwabacher <ischwabacher(a)wisc.edu> writes:
> >> > ...
> >> >
> >> > I know that you can do datetime.now(tz), and you can do datetime(2013,
> >> > 11, 3, 1, 30, tzinfo=zoneinfo('America/Chicago')), but not being able
> >> > to add a time zone to an existing naive datetime is painful (and
> >> > strptime doesn't even let you pass in a time zone).
> >>
> >> `.now(tz)` is correct. `datetime(..., tzinfo=tz)`) is wrong: if tz is a
> >> pytz timezone then you may get a wrong tzinfo (LMT), you should use
> >> `tz.localize(naive_dt, is_dst=False|True|None)` instead.
> >
> > The whole point of this thread is to finalize PEP 431, which fixes the
> > problem for which `localize()` and `normalize()` are workarounds. When
> > this is done, `datetime(..., tzinfo=tz)` will be correct.
> >
> > ijs
>
> The input time is ambiguous. Even if we assume PEP 431 is implemented in
> some form, your code is still missing isdst parameter (or the
> analog). PEP 431 won't fix it; it can't resolve the ambiguity by
> itself. Notice is_dst paramter in the `tz.localize()` call (current
> API).
...yeah, I forgot to throw that in there. It was supposed to be there all along. Nothing to see here, move along.
> .now(tz) works even during end-of-DST transitions (current API) when the
> local time is ambiguous.
I know that. That's what I was complaining about-- I was trying to talk about how astimezone() was going to be inadequate even after the PEP was implemented because it couldn't turn naive datetimes into aware ones, and people were giving examples that started with aware datetimes generated by now(tz), which completely went around the point I was trying to make. But it looks like astimezone() is going to grow an is_dst parameter, and everything will be OK.
ijs
How do we tell if we're helping or hindering the core development process? (was Re: How far to go with user-friendliness)
by Nick Coghlan July 29, 2015 July 29, 2015
On 20 July 2015 at 22:34, Ben Finney <ben+python(a)benfinney.id.au> wrote:
> Paul Moore <p.f.moore(a)gmail.com> writes:
>
>> Again, I'm sorry to pick on one sentence out of context, but it cut
>> straight to my biggest fear when doing a commit (on any project) -
>> what if, after all the worrying and consideration I put into doing
>> this commit, people disagree with me (or worse still, I made a
>> mistake)? Will I be able to justify what I decided?
>
> That seems quite healthy to me. On a collaborative project with effects
> far beyond oneself, yes, any change *should* be able to be justified
> when challenged.
No, that's not how this works: if folks are thinking that being a
Python user, or even a CPython core developer, means that we're
entitled to micromanage core developers by demanding extensive
explanations for any arbitrary commit we choose, they're thoroughly
mistaken. Only Guido has that privilege, and one of the reasons he's
as respected as he is is his willingness to trust the experience and
expertise of others and only rarely exercise his absolute authority.
Folks are granted core committer privileges because we *trust their
instincts*. We trust them to know when they're acting within the
limits of their own expertise and experience, and we trust them to
know when it would be beneficial to seek feedback from a wider
audience before making up their minds.
There are *many* cases where we *don't know* up front what the right
answer is, so we'll actively seek consultation, whether that's through
a review request on the issue tracker, a focused python-dev
discussion, a more speculative discussion on python-ideas, or engaging
in the full Python Enhancement Proposal process.
There are also cases where we'll decide "it seems plausible that this
might be a good idea, so let's try it out and see what happens in
practice rather than continuing to speculate" - only ever doing things
that you're already 100% certain are a good idea is a recipe for
stagnation and decline (hence the "Now is better than never" line in
the Zen). A feedback cycle of a few years is a relatively short time
in programming language development, so if we discover with the
benefit of hindsight that something that seemed plausible really was
in fact a genuinely bad idea, that's why we have a deprecation process
(as well as the abilty to mark new APIs as provisional). If we want a
faster feedback cycle measured in days or weeks or months rather than
years, then we'll find a way to make the idea available for
experimentation via PyPI or at least as a cookbook recipe or in a
public source repo.
But the authority and responsibility to make changes, to decide what
constitutes a reasonable risk, to decide which updates are appropriate
to send out to tens of millions of Python users worldwide remains
*ours*.
Some of those decisions will be akin to deciding to paint a bikeshed
blue instead of yellow (or green, or red, or chartreuse, or ...).
Others will be about mitigating the observed negative consequences of
a previous design decision while retaining the positive aspects.
Those kinds of design decision are still more art than science -
that's why our solution to them is to attempt to recruit people with
the interest, ability and time needed to make them well, and then
largely leave them to it. Explaining design decisions after the fact
isn't about *defending* those decisions, it's about attempting to
convey useful design knowledge, in order to help build new core
contributors (and also because explaining something to someone else is
a good way to understand it better yourself).
All of this is why the chart that I believe should be worrying people
is the topmost one on this page:
http://bugs.python.org/issue?@template=stats
Both the number of open issues and the number of open issues with
patches are steadily trending upwards. That means the bottleneck in
the current process *isn't* getting patches written in the first
place, it's getting them up to the appropriate standards and applied.
Yet the answer to the problem isn't a simple "recruit more core
developers", as the existing core developers are *also* the bottleneck
in the review and mentoring process for *new* core developers.
In my view, those stats are a useful tool we can use to ask ourselves
"Am I actually helping with this contribution, or at the very least,
not causing harm?":
* participating in the core-mentorship list, and actively looking for
ways to improve the effectiveness of the mentorship program (that
don't just queue up more work behind the existing core developer
bottleneck) should help make those stats better
* sponsoring or volunteering to help with the upstream projects our
core workflow tools are based on (Roundup, Buildbot) or with updating
our specific installations of them should help make those stats better
(and also offers faster iteration cycles than core development itself)
* helping core developers that have time to work on "CPython in
general" rather than specific projects of interest to them to focus
their attention more effectively may help make those stats better (and
it would be even better if we could credit such triaging efforts
appropriately)
* exploring ways to extract contribution metrics from Roundup so we
can have a more reliable detection mechanism for active and consistent
contributors than the "Mark 1 core developer memory, aka the
notoriously unreliable human brain" may help make those stats better
* helping out the upstream projects for one or both of the
forge.python.org proposals (Kallithea, Phabricator) that are designed
(at least in part) to remove core developers from the critical path
for management of the support repos like the developer guide and
Python Enhancement Proposals may help make those stats better
* getting more core contributors into a position where at least some
of their work time can be spent on facilitating upstream contributions
may help make those stats better (shifting at least some core
contribution activity to paid roles is also one of the keys to
countering the "free time for volunteer activities is inequitably
distributed" problem)
* alienating veteran core developers and encouraging them to spend
their time elsewhere will contribute to making those stats worse
* discouraging newly minted core developers from exercising their
granted authority will contribute to making those stats worse
* discouraging veteran core developers from recruiting new core
developers by contributing to creating a hostile environment on the
core mailing lists will *definitely* contribute to making those stats
worse
Make no mistake, sustainable open source development is a *genuinely
hard problem*. We're in a much better position than most projects
(being one of the top 5 programming languages in the world has its
benefits), but we're still effectively running a giant multinational
collaborative software development project with close to zero formal
management support. While their are good examples we can (and are)
learning from, improving the support structures for an already wildly
successful open source project without alienating existing
contributors isn't a task that comes with an instruction manual :)
Regards,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
Hi Internals,
This is the first message from Intel's language optimization team.
We would like to provide the Python internals developer community
with a daily service which will monitor latest committed patches
performance regressions against well known workloads.
Our aim is to run a multitude of workloads as well as real-life scenarios
which the community considers relevant. The service will send daily bulletins
containing latest measurements for daily variations and variations against
latest stable release run on our Intel-enabled servers.
The community's feedback is very important for us. For any questions,
comments or suggestions you can also contact us on our mailing list
lp(a)lists.01.org. You can also check our website: https://www.01.org/lp
Results for project python_default-nightly, build date 2015-07-24 09:02:02
commit: 3bbd0cbfe836511dd3e05fcc30ffb5bdbfe686ea
revision date: 2015-07-24 07:43:44
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64
Note: Baseline results were generated using release v3.4.3, with hash
b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00
--------------------------------------------------------------------------------
benchmark unit change since change since
last run v3.4.3
--------------------------------------------------------------------------------
:-) django_v2 sec 1.12735% 7.47953%
:-( pybench sec -0.53822% -2.40216%
:-( regex_v8 sec 0.61774% -2.32010%
:-| nbody sec 1.75860% -0.76206%
:-) json_dump_v2 sec 2.13422% -0.56930%
Our lab does a nightly source pull and build of the Python project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.
Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration. No license (express or implied, by estoppel
or otherwise) to any intellectual property rights is granted by this document.
Intel disclaims all express and implied warranties, including without
limitation, the implied warranties of merchantability, fitness for a particular
purpose, and non-infringement, as well as any warranty arising from course of
performance, course of dealing, or usage in trade. This document may contain
information on products, services and/or processes in development. Contact your
Intel representative to obtain the latest forecast, schedule, specifications and
roadmaps. The products and services described may contain defects or errors
known as errata which may cause deviations from published specifications.
Current characterized errata are available on request.
(C) 2015 Intel Corporation.
|
| You are visitor 01402520 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |
