GeezerVison

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be poked with soft cushions!

Side note: perhaps we mark all PEM files as generated?

Side note: perhaps we mark all PEM files as generated?

Done: I've marked certdata/*.{pem,0} as generated (the .0 files should also really be .pem I think -- I can do some cleanup there in a follow-up PR, if there's interest).

In practice, this should have no effect on >99% of users: most root programs use 5280 as their baseline, and the Web PKI's CABF rules are also built on 5280.

Your PR required to regenerate all PEM test certificates. Does it mean that they belong to the 1% of affected users?

Your PR required to regenerate all PEM test certificates. Does it mean that they belong to the 1% of affected users?

I made the numbers up for emphasis, but I'll stand by this having no effect on the vast majority of users: the vast majority use case is the Web PKI, which is significantly stricter than even VERIFY_X509_STRICT. The fact that the PEMs had to be regenerated is more a testament to how lax OpenSSL is by default than any actual common practice 🙂

Or put another way: most users are not manually generating certs with openssl req or, if they are, they're massaging them into RFC 5280 compliance because other ecosystems (like Go, Java, and OSes/major software) are much more strict.

Additional datapoints:

• The MS root program is at least as strict as these changes: https://learn.microsoft.com/en-us/security/trusted-root/program-requirements
• The Java root program specifies RFC 5280, making it at least as strict: https://www.oracle.com/java/technologies/javase/carootcertsprogram.html
• Same with Adobe: https://helpx.adobe.com/acrobat/kb/approved-trust-list2.html

(I picked these semi-arbitrarily from https://pkic.org/ltl/)

Since OpenSSL will likely be the way folks are generating certificates

Is that still the case? Aren't most users relying on something like Let's Encrypt nowadays (at least for Web certificates, but also perhaps e-mail)?

is there any value in providing guidance on how one does generate a certificate with OpenSSL that works with VERIFY_X509_STRICT?

I don't think writing an OpenSSL command line tutorial is really a responsibility for CPython maintainers.

Since OpenSSL will likely be the way folks are generating certificates and like you said, OpenSSL allows you to do things that isn't acceptable for Web PKI, is there any value in providing guidance on how one does generate a certificate with OpenSSL that works with VERIFY_X509_STRICT?

To clarify: I think most users won't be using OpenSSL to generate certs -- I think the majority will be getting their certs from a CA service (like Let's Encrypt), with a very small minority doing bespoke things. Putting an OpenSSL example in the docs might cause additional confusion, and IMO won't be too helpful anyways (since the context here is verification, in which case the user might not have any control over the certs they're being presented with).

What about non-Web PKI, does this change have any affect for certificates in that domain? I am less versed for this, so am relying a bit on others' experience

Nope, this should be wholly compatible with those -- "strict" validation for OpenSSL means "approximate RFC 5280," and the Web PKI profiles are a superset of RFC 5280 🙂

In other words: all public web certificates should be compatible with these changes.

These are the checks that OpenSSL applies when VERIFY_X509_STRICT is set: https://www.openssl.org/docs/man3.0/man1/openssl-verification-options.html#x509_strict (search for -x509_strict in case the link doesn't take you there directly)

These checks all make sense for certs generated with a trusted CA, e.g. Let's Encrypt.

The other common use case is having self-signed certs (e.g. for intranet/VPC devices). Those will already fail to verify with the current standard settings, since self-signed certs are rejected by OpenSSL's verify. You have to use the ssl._create_unverified_context() or a custom one to the same effect for those.

Note: The POP/FTP/IMAP and SMTP modules use the unverified context, since intranet/VPC mail and FTP servers will often not use certs created by a trusted CA for various reasons or they are referenced using IP addresses, which don't work with certs at all.

🤖 New build scheduled with the buildbot fleet by @gpshead for commit cef6950 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@woodruffw Looks like this change breaks trustme generated certificates, see python-trio/trustme#642

@woodruffw Looks like this change breaks trustme generated certificates, see python-trio/trustme#642

Thanks for sharing! Looking now.

@woodruffw Looks like this change breaks trustme generated certificates, see python-trio/trustme#642

If an application is affected by these changes, they can "just" disable the two options, no?

Using ctx.verify_flags &= ~(ssl.VERIFY_X509_STRICT | ssl.VERIFY_X509_PARTIAL_CHAIN):

$ ./python
Python 3.13.0a5+ (heads/main:b44898299a2, Mar 28 2024, 09:01:21) [GCC 13.2.1 20240316 (Red Hat 13.2.1-7)] on linux
>>> import ssl

>>> ctx=ssl.create_default_context()
>>> ctx.verify_flags & ssl.VERIFY_X509_STRICT
<VerifyFlags.VERIFY_X509_STRICT: 32>
>>> ctx.verify_flags & ssl.VERIFY_X509_PARTIAL_CHAIN
<VerifyFlags.VERIFY_X509_PARTIAL_CHAIN: 524288>

>>> ctx.verify_flags &= ~(ssl.VERIFY_X509_STRICT | ssl.VERIFY_X509_PARTIAL_CHAIN)
>>> ctx.verify_flags & ssl.VERIFY_X509_STRICT
<VerifyFlags.VERIFY_DEFAULT: 0>
>>> ctx.verify_flags & ssl.VERIFY_X509_PARTIAL_CHAIN
<VerifyFlags.VERIFY_DEFAULT: 0>

If an application is affected by these changes, they can "just" disable the two options, no?

Yep, correct. I think the above is also generating certs, so they wanted to get it right more generally, so that each Python downstream consumer doesn't have to also download the strict option.