GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-119400: make_ssl_certs: update reference test data automatically, pass in expiration dates as parameters #119400 by kanavin · Pull Request #119401 · python/cpython

Conversation

kanavin commented May 22, 2024 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

As discussed here:
#107594

make_ssl_certs.py has a few shortcomings. In particular:

it generates certificates, but does not update reference data in tests that use them, instead asking the user to copy paste the data by hand (expiration dates and serial numbers in particular)
it is supposed to be run by hand and isn't executed in builds, which means its output has to be checked into git, cluttering the source tree, and complicating reviews of pull requests that change that output.
expiration dates are hardcoded into the tool and can't be passed in as parameters

This pull request aims to address first and last issue, so then #107594 can move forward on top of them.

Issue: make_ssl_certs: run at build time, update reference test data automatically #119400

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

kanavin requested review from 1st1, asvetlov, gvanrossum, kumaraditya303 and willingc as code owners

May 22, 2024 13:15

bedevere-app Bot added the awaiting review label

May 22, 2024

bedevere-app Bot mentioned this pull request

May 22, 2024

make_ssl_certs: run at build time, update reference test data automatically #119400

Open

kanavin commented May 22, 2024

Copy link

Copy Markdown

Contributor Author

OSError: [Errno 30] Read-only file system: '/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/certdata'

This means CI is set up so that modifying the source tree is not possible. Suggestions? I still think it's worth making make_ssl_certs execution a part of the build, but the complication is that its output needs to be written into build dir, and both installation and tests needs to find it there.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kanavin mentioned this pull request

May 22, 2024

gh-107562: Lib/test: update test certificates to expire far in the future #107594

Merged

kanavin commented May 23, 2024

Copy link

Copy Markdown

Contributor Author

OSError: [Errno 30] Read-only file system: '/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/certdata'
This means CI is set up so that modifying the source tree is not possible. Suggestions? I still think it's worth making make_ssl_certs execution a part of the build, but the complication is that its output needs to be written into build dir, and both installation and tests needs to find it there.

I've concluded that this is not feasible for now:

needs invasive changes to Makefile
requires openssl executable at build time
breaks build reproducibility as every build is going to have different certificates installed, even if they're only used for testing.

I'll drop that from this PR, and make it only about not hardcoding reference certificate data and expiration parameters.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kanavin changed the title ~~gh-119400: make_ssl_certs: run at build time, update reference test data automatically #119400~~ gh-119400: make_ssl_certs: update reference test data automatically, pass in expiration dates as parameters #119400

May 23, 2024

kanavin force-pushed the fix-make-ssl-certs branch from 87c0672 to be2c50f Compare

May 23, 2024 09:59

kumaraditya303 removed their request for review

June 23, 2024 07:54

kanavin commented Sep 6, 2024

Copy link

Copy Markdown

Contributor Author

This seems to be not getting any attention, is there something I can do to push it forward?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum requested review from encukou and sethmlarson and removed request for 1st1, asvetlov and gvanrossum

September 6, 2024 15:59

encukou reviewed

Sep 13, 2024

View reviewed changes

encukou left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay; my review queue grew too long and I had to trim it.
This looks great, but let's change a few details:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

Lib/test/test_asyncio/utils.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Misc/NEWS.d/next/Build/2024-05-22-13-18-02.gh-issue-119400.WEt83v.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Lib/test/certdata/allsans.pem Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Lib/test/certdata/make_ssl_certs.py Resolved

Uh oh!

There was an error while loading. Please reload this page.


          📜🤖 Added by blurb_it.

0a7a141

kanavin force-pushed the fix-make-ssl-certs branch from 6a43765 to af59334 Compare

September 25, 2024 09:31

kanavin and others added 2 commits

September 25, 2024 11:37


          Lib/test/certdata: do not hardcode reference cert data into tests

0e82723

The script was simply printing the reference data and asking
users to update it by hand into the test suites. This can
be easily improved by writing the data into files and
having the test cases load the files.

Co-authored-by: Petr Viktorin <[email protected]>
Signed-off-by: Alexander Kanavin <[email protected]>


          make_ssl_certs: make it possible to pass in expiration dates from com…

4bff2d1

…mand line

Note that the defaults are same as they were, so if nothing is
specified, the script works exactly as before.

Signed-off-by: Alexander Kanavin <[email protected]>

kanavin force-pushed the fix-make-ssl-certs branch from af59334 to 4bff2d1 Compare

September 25, 2024 09:37

kanavin requested a review from encukou

September 25, 2024 10:02

kanavin commented Sep 25, 2024

Copy link

Copy Markdown

Contributor Author

@encukou Thanks, everything should be addressed now.

encukou reacted with thumbs up emoji

All reactions

👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou reviewed

Sep 25, 2024

View reviewed changes

Comment thread

Lib/test/certdata/make_ssl_certs.py Resolved

Uh oh!

There was an error while loading. Please reload this page.


          Print trailing newlines to the reference files

be13969

encukou enabled auto-merge (squash)

September 25, 2024 20:59

encukou merged commit 1ff1b89 into python:main

Sep 25, 2024

bedevere-app Bot removed the awaiting review label

Sep 25, 2024

emilyemorehouse added a commit to lysnikolaou/cpython that referenced this pull request

Sep 26, 2024


          Merge branch 'main' into fix-help-double-prompt

4cdb3ae

* main: (69 commits)
  Add "annotate" SET_FUNCTION_ATTRIBUTE bit to dis. (python#124566)
  pythongh-124412: Add helpers for converting annotations to source format (python#124551)
  pythongh-119180: Disallow instantiation of ConstEvaluator objects (python#124561)
  For-else deserves its own section in the tutorial (python#123946)
  Add 3.13 as a version option to the crash issue template (python#124560)
  pythongh-123242: Note that type.__annotations__ may not exist (python#124557)
  pythongh-119180: Make FORWARDREF format look at __annotations__ first (python#124479)
  pythonGH-58058: Add quick reference for `ArgumentParser` to argparse docs (pythongh-124227)
  pythongh-41431: Add `datetime.time.strptime()` and `datetime.date.strptime()` (python#120752)
  pythongh-102450: Add ISO-8601 alternative for midnight to `fromisoformat()` calls. (python#105856)
  pythongh-124370: Add "howto" for free-threaded Python (python#124371)
  pythongh-121277: Allow `.. versionadded:: next` in docs (pythonGH-121278)
  pythongh-119400:  make_ssl_certs: update reference test data automatically, pass in expiration dates as parameters python#119400  (pythonGH-119401)
  pythongh-119180: Avoid going through AST and eval() when possible in annotationlib (python#124337)
  pythongh-124448: Update Windows builds to use Tcl/Tk 8.6.15 (pythonGH-124449)
  pythongh-123884 Tee of tee was not producing n independent iterators (pythongh-124490)
  pythongh-124378: Update test_ttk for Tcl/Tk 8.6.15 (pythonGH-124542)
  pythongh-124513: Check args in framelocalsproxy_new() (python#124515)
  pythongh-101100: Add a table of class attributes to the "Custom classes" section of the data model docs (python#124480)
  Doc: Use ``major.minor`` for documentation distribution archive filenames (python#124489)
  ...

kanavin added a commit to kanavin/cpython that referenced this pull request

Sep 26, 2024


          Revert "pythongh-119400: make_ssl_certs: update reference test data a…

496df6c

…utomatically, pass in expiration dates as parameters python#119400  (pythonGH-119401)"

This reverts commit 1ff1b89.

halstead pushed a commit to openembedded/openembedded-core that referenced this pull request

Feb 19, 2026


          time64.inc: clean up and add upstream tickets where issues remain

640e2f9

Most issues were resolved via upstream version updates that bring in
needed fixes:

glib-2.0 update to 2.78.0 that includes:
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/3547
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/3550

curl update to 8.3.0 that includes
curl/curl#11610

util-linux update to 2.39 that includes
util-linux/util-linux#2430
util-linux/util-linux@3ab9e69
util-linux/util-linux#2435

glib-networking update to 2.78.0 that includes
https://gitlab.gnome.org/GNOME/glib-networking/-/merge_requests/241

python3-cryptography update to 42.0.0 which resolves
pyca/cryptography#9370 via
pyca/cryptography#9964

perl update to 5.40.0 which includes
Perl/perl5#21379

python3 update to 3.13.0 which includes
python/cpython#118425
python3 update to 3.13.1 which includes
python/cpython#124972
python3 update to 3.14.0 which includes
python/cpython#119401
python/cpython#125045
python/cpython#107594
python/cpython#125104

tcl update to 9.0.0 which includes
tcltk/tcl@4ca6172
(tcl8 recipe has a simple backport of this)

dbus update to 1.16.0 which includes
https://gitlab.freedesktop.org/dbus/dbus/-/merge_requests/444
https://gitlab.freedesktop.org/dbus/dbus/-/merge_requests/289

openssh update to 10.0p1 which includes
openssh/openssh-portable#425
https://bugzilla.mindrot.org/show_bug.cgi?id=3684
https://marc.info/?l=openbsd-bugs&m=172561736524815&w=2
https://lists.mindrot.org/pipermail/openssh-unix-dev/2024-October/041621.html
(all reporting the same issue)

gcc update to 15.1 which includes
llvm/llvm-project#99699
via gcc-mirror/gcc@fa32100
and allows dropping special flags and exceptions for gcc-sanitizers.

Signed-off-by: Alexander Kanavin <[email protected]>
Signed-off-by: Mathieu Dubois-Briand <[email protected]>
Signed-off-by: Richard Purdie <[email protected]>

Reviewers

encukou

encukou left review comments

willingc

Awaiting requested review from willingc

willingc is a code owner

sethmlarson

Awaiting requested review from sethmlarson

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

kanavin commented May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kanavin commented May 22, 2024

Uh oh!

kanavin commented May 23, 2024

Uh oh!

kanavin commented Sep 6, 2024

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kanavin commented Sep 25, 2024

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kanavin commented May 22, 2024 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.