GeezerVison is showing github.com — Lint: Update headers and checks per current guidance & provide helpful feedback by CAM-Gerlach · Pull Request #2484 · python/peps
Conversation
CAM-Gerlach
commented
Mar 30, 2022
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Over the past >year, we've made significant progress in programmatically parsing the PEP headers, using them to intelligently display more useful, informative and readable output in the rendered PEPs, conforming them to our modern guidance, automatically checking their format and (particularly now with #2475) making it easy for other tools to consume and enrich them.
As a final step toward these overarching goals, this PR:
- Adds the previously-missing automatic checks for all the remaining headers, so errors in them that could affect rendering and human- and machine-readability no longer pass silently
- Updates the existing checks to reflect the current format specified by PEP 1/12 and expected by current and future tooling
- Conforms the modest number of remaining existing PEPs to ensure the headers render correctly and are machine-readable, and
- Provides much more helpful, specific and user-friendly descriptive check text.
Overall, this PRs enhancements to our existing checks:
- Helps PEP authors, by giving them near-instant, automatic and targeted feedback about any syntax issues, locally or on CI, without having to for a full rebuild or a human review
- Helps PEP editors by freeing them from the need to manually inspect and fix PEP headers
- Avoids any edge cases in our rendering pipeline
- Helps both readers and tools by ensuring the output is easy for both humans and machines to read, and
- Opens the door to more "smart" processing in the future (e.g. automatic Discussions-To generation) to reduce PEP author/editor workload while making the displayed output more useful and readable.
To note, this PR does not make any new headers required, invalidate any existing established header formats nor require any of the newer ones; the new checks only trigger on the formats.
In addition to the above, this resolves the linting side of #2402 and supports the improvements in #2475 and #2467 .
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
CAM-Gerlach
requested review from
JelleZijlstra,
Mariatta,
ambv,
brettcannon,
dhellmann,
dstufft,
encukou,
ericsnowcurrently,
gpshead,
gvanrossum,
iritkatriel,
jackjansen,
ncoghlan,
njsmith,
pablogsal,
pfmoore,
pitrou,
rhettinger,
stevendaprano,
taleinat,
tim-one,
tiran,
vstinner,
warsaw,
willingc and
zware
as code owners
March 30, 2022 06:41
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still have some concerns about adding email addresses without permission, and replacing user at example.com with [email protected], but everything else is fine, and I won't block merging over it. Thanks!
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
CAM-Gerlach
commented
Apr 16, 2022
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
I still have some concerns about adding email addresses without permission,
Well again, the only email addresses added in the current version of the PR were those for same core devs who already chose to include them the headers of their other PEPs, and which are all already listed in the Authors Index, so it is only making things consistent between PEPs. Also, I'd initially thought from PEP 12 that the email address was a required element and had the checks enforce that for new PEPs, which is why I'd initially made the change, but I'd then realized after cross-referencing PEP 1 that it wasn't, and updated the checks accordingly.
Much the same is true of user at example.com -> [email protected]—its already obfuscated automatically, having it in the source makes authors (like myself at time) think they had to do it manually, and it modestly complicates the processing and checks, but likewise I quickly realized a relatively small simplification of the code was not worth the potential for more trouble for authors, so I ensured the checks and processing code allow it.
In both cases, there is little benefit conforming the existing source headers when neither is actually required; its just that likewise, it would also be a fair amount of work for no clear benefit to reverting them either, since the emails are already listed and obfuscated. But as it seems its the one thing that's blocking this PR, I guess I'll just bite the bullet and spend the time doing it.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
All done—this PR now does not add any existing emails, nor does it conform them to the standard address format. Also fixed what would be a conflict with #2531 , clarified the check feedback a bit further, and a couple other minor refinements.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I appreciate all your work here, thank you!
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
CAM-Gerlach
commented
Apr 17, 2022
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
Actually, I noticed there's actually a significant problem as it stands allowing manual obfuscation—glancing at the _mask_email() source code as part of the trailing comma oversight you noted on #2531 and I then fixed in #2467 , I notice that the automatic obfuscation is actually a fair bit more effective than just replacing @ with a literal at; rather, it actually uses the raw HTML entity codes for some of the characters, significantly raising the bar for scrapers.
However, since emails with manual obfuscation applied don't get converted into reference nodes, the automatic obfuscation logic doesn't work, so these emails actually are substantially less obfuscated than they would otherwise be (which PEP authors are almost certainly unaware of, and wasn't even to me until I dug deep into the code and actually tested it). Picking on myself here, from PEP 0, compare these two:
<tr class="[row-odd]()"><td>Gerlach, C.A.M.</td>
<td>cam.gerlach at gerlach.cam</td>
</tr>
versus, e.g.
<tr class="[row-even]()"><td>van Rossum, Guido (GvR)</td>
<td>guido at python.org</td>
</tr>
I attempted to add support for also properly masking manually obfuscated emails, but after some testing realized it would require some pretty significant code changes due to how things are currently structured, as well as some hacky and potentially unreliable heuristics. Furthermore, I realized this also makes the not showing the email addresses all, or only as abbreviations, as discussed in #2514 more complicated, since whether or not the email is actually processed as an email changes the doctree structure and node types.
Therefore, I conclude it would be best to re-revert the part of the previous change to conform the small minority of emails that were manually obfuscated to use standard email syntax and restore the linter check for such, so they are correctly processed and masked by the header transform code (and anything else that needs to mask/obfuscate/elide them), in order to ensure that the various automatic measures to protect authors' emails work consistently.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Actually, upon giving this more thought, making the author-emails abbrs instead of literal text as discussed in #2514 still requires doing fairly involved string-munging anyway due to having to parse and transform the older Email (author) format, so while it does make things a little more complicated, its not that much worse than parsing a while different syntax.
As such, I suggest we just go ahead and merge this PR as-is with the manual obfuscation still untouched, and then I can address properly masking manually "obfuscated" emails along with formatting them consistently as abbrs in a followup PR. It would also be pretty easy to improve the obfuscation further by choosing different Unicode lookalike characters for the spaces and letters, which coupled with being embedded in the abbr and using raw character codes should make it virtually impossible for spam harvesting, far more so than the common and well known replacement of @ with at.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Since it seems I've satisfied the immediate concerns that were raised and the two PEP editors that previously reviewed have ✔️ ed, this PEP has been open for a while and it seems there aren't any further objections, and it is blocking some further discussed and agreed changes (abbr for emails, making Content-Type optional, etc.,), I'll go ahead and finally merge this now.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
hugovk
hugovk approved these changes
AA-Turner
Awaiting requested review from AA-Turner
+2 more reviewers
Reviewers whose approvals may not affect merge requirements
Uh oh!
There was an error while loading. Please reload this page.
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|