GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-94172: Remove keyfile, certfile and check_hostname parameters by vstinner · Pull Request #94173 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Jun 23, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Remove the keyfile, certfile and check_hostname parameters,
deprecated since Python 3.6, in modules: ftplib, http.client,
imaplib, poplib and smtplib. Use the context parameter (ssl_context
in imaplib) instead.

ftplib: Remove the FTP_TLS.ssl_version class attribute: use the
context parameter instead.

  • Issue: Remove deprecated keyfile, certfile and check_hostname parameters, in favor of context #94172

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added the awaiting core review label Jun 23, 2022
@vstinner
vstinner marked this pull request as ready for review June 23, 2022 13:40
@vstinner
vstinner requested a review from a team as a code owner June 23, 2022 13:40
@vstinner

vstinner commented Jun 23, 2022

Copy link
Copy Markdown
Member Author

cc @tiran @serhiy-storchaka

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 23, 2022

Copy link
Copy Markdown
Member Author

cc @hugovk

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka
serhiy-storchaka requested a review from tiran June 23, 2022 13:48
serhiy-storchaka
serhiy-storchaka reviewed Jun 23, 2022
View reviewed changes
Comment thread Lib/ftplib.py Outdated

@serhiy-storchaka serhiy-storchaka Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you remove positional parameters, you have to make the following parameters keyword-only.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 23, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, done. I didn't document this change. Is it worth to document it?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@serhiy-storchaka serhiy-storchaka Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were similar changes documented in the past?

Please add tests that only such positional arguments are accepted. In most cases None is a valid argument both for the first deleted parameter and for the first remaining parameter, so you can just add None as an excessive positional argument.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 23, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were similar changes documented in the past?

In commit 9baf242, I removed a buffering parameter in bz2.BZ2File. I see that I wrote ".. versionchanged:: 3.9 (...) The compresslevel parameter became keyword-only." in the doc.

Honestly, I don't think that it should be documented. I don't expect people to call such constructor with 6 parameters or more. I'm only aware of people who called CodeType() with tons of arguments: problem solved with CodeType.replace() method addition.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/ftplib.py Outdated

@serhiy-storchaka serhiy-storchaka Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it removed?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 23, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you provide a SSLContext, you get different protocol than FTP_TLS.ssl_version. This removal is not required, it's more to make the API consistent with the other modified modules: the SSLContext constructor is the only place setting default parameter values. Do you prefer to keep it?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@serhiy-storchaka serhiy-storchaka Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know whether it is documented, but it specifies the default protocol if you do not provide an SSLContext. The user can override it in FTP_TLS or subclasses.

Who added this attribute? Ask him.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tiran tiran Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All protocols except PROTOCOL_TLS, TLS_CLIENT, and TLS_SERVER are deprecated anyways because OpenSSL is going to remove them eventually.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 23, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tiran: Do you see a reason to keep ssl_version to specify a different "ssl version" when context parameter is omitted?

The practical problem is more that applications relying on this feature may not notice that overriden ssl_version are now ignored silently.

Note: I hate mentions of "ssl", since SSL no longer exists, it was replaced with TLS, but the Python module is still called "ssl"... as OpenSSL ;-)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tiran tiran Jul 5, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no reason to keep the variable here. ssl.PROTOCOL_TLS_CLIENT is the only correct and fully supported value. In 3.13 I will remove all the other protocols. Only ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER will stay.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner

vstinner commented Jun 23, 2022

Copy link
Copy Markdown
Member Author

Oh, HTTPSConnection of urllib.request has context and check_hostname parameters, and context can be None. http.client code to create the default context is non trivial, I don't want to copy it in urllib/request.py. Maybe http.client check_hostname parameter should be kept. What do you think @serhiy-storchaka?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 23, 2022

Copy link
Copy Markdown
Member Author

Right now, 4 tests are failing because of the urllib issue: test_site test_ssl test_urllib test_urllib2_localnet

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka
serhiy-storchaka reviewed Jun 23, 2022
View reviewed changes
Comment thread Lib/ftplib.py Outdated

@serhiy-storchaka serhiy-storchaka Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know whether it is documented, but it specifies the default protocol if you do not provide an SSLContext. The user can override it in FTP_TLS or subclasses.

Who added this attribute? Ask him.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/ftplib.py Outdated

@serhiy-storchaka serhiy-storchaka Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is the right way to do, should this change be backported as a bug fix? I think it deserves a separate issue.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 26, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. Sadly, I don't know well the ftplib module. I'm not comfortable to change it in a stable Python version. In case of doubt, I prefer to leave the code as it is in stable branches, unless someone reports a bug.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/poplib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented Jun 23, 2022

Copy link
Copy Markdown
Member

Oh, HTTPSConnection of urllib.request has context and check_hostname parameters, and context can be None. http.client code to create the default context is non trivial, I don't want to copy it in urllib/request.py. Maybe http.client check_hostname parameter should be kept.

Specifying check_hostname for urllib.request.HTTPSConnection currently produces a warning, right? Therefore it should be removed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tiran
tiran reviewed Jun 23, 2022
View reviewed changes
Comment thread Lib/ftplib.py Outdated

@tiran tiran Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All protocols except PROTOCOL_TLS, TLS_CLIENT, and TLS_SERVER are deprecated anyways because OpenSSL is going to remove them eventually.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/ftplib.py Outdated

@tiran tiran Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SSLv2 and SSLv3 are no longer supported. You can remove the AUTH SSL block.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 26, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I read Modules/_ssl.c, it's not obvious that SSL is no longer supported. There is still conditonal code:

#ifndef OPENSSL_NO_SSL2
    PyModule_AddIntConstant(m, "PROTOCOL_SSLv2",
                            PY_SSL_VERSION_SSL2);
#endif
#ifndef OPENSSL_NO_SSL3
    PyModule_AddIntConstant(m, "PROTOCOL_SSLv3",
                            PY_SSL_VERSION_SSL3);
#endif

These constants are still documented but marked as "deprecated" in the doc: https://docs.python.org/dev/library/ssl.html#ssl.PROTOCOL_SSLv2

I would prefer to fully remove support for SSL in the _ssl/ssl modules, before changing the ftplib module.

Another example: https://docs.python.org/dev/library/ssl.html#ssl.OP_NO_SSLv3 still exists (at least in the doc).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tiran tiran Jul 6, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's documented in ssl.rst. I'm using the word "supported" in the sense of a social contract. The presence of the constants do not mean that a user can expect they are doing anything sensible.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/http/client.py Outdated
Comment on lines 1429 to 1417

@tiran tiran Jun 23, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove the check. It is performed by SSLContext internally.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner vstinner mentioned this pull request Jun 24, 2022
gh-94172: urllib.request avoids deprecated check_hostname #94193
Merged
@vstinner

vstinner commented Jun 24, 2022

Copy link
Copy Markdown
Member Author

I wrote PR #94193 to prepare urllib.request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 24, 2022

Copy link
Copy Markdown
Member Author

URLopener of urllib.request has key_file and cert_file parameters which are passed to http.client.HTTPSConnection. These parameters are not directly deprecated, but the whole URLopener class is deprecated since Python 3.3!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 24, 2022

Copy link
Copy Markdown
Member Author

URLopener of urllib.request has key_file and cert_file parameters which are passed to http.client.HTTPSConnection.

I created PR #94232 to fix this issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 26, 2022

Copy link
Copy Markdown
Member Author

I created PR #94232 to fix this issue.

Merged. I rebased this PR on top of it. Currently, the PR still removes FTP_TLS.ssl_version. This removal seems controversial, I may just revert it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tiran

tiran commented Jun 26, 2022

Copy link
Copy Markdown
Member

I created PR #94232 to fix this issue.

Merged. I rebased this PR on top of it. Currently, the PR still removes FTP_TLS.ssl_version. This removal seems controversial, I may just revert it.

I suggest to remove it. In a future OpenSSL version it won't be able to change the TLS version. Even now it cannot be used to switch to TLS 1.3-only.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 26, 2022

Copy link
Copy Markdown
Member Author

FTP_TLS.ssl_version was added with FTP_TLS by commit ccd5e02 in 2009: patch by @giampaolo, merged by @pitrou.

Changes:

  • commit 9fe67ce (2015) changed the default from PROTOCOL_TLSv1 to PROTOCOL_SSLv23.
  • commit a170fa1 (2017) changed the default from PROTOCOL_SSLv23 to PROTOCOL_TLS_CLIENT.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 26, 2022

Copy link
Copy Markdown
Member Author

About removing resp = self.voidcmd('AUTH SSL'): I wrote PR #94312 to clarify that Python no longer supports SSLv2.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Jun 28, 2022
Remove deprecated keyfile, certfile and check_hostname parameters, in favor of context #94172
Closed
ssl: Drop support for unsafe SSLv3 protocol, remove ssl.PROTOCOL_SSLv3 #94377
Closed
@vstinner

vstinner commented Jul 4, 2022

Copy link
Copy Markdown
Member Author

I tried to run a code search with:

./search_pypi_top.py PYPI-2022-06-24-TOP-5000/ '(keyfile|key_file|certfile|cert_file|check_hostname)=' -o cert_file.txt -q

Problem: these parameter names are common and they are many usages which remain perfectly fine and supported, like (example from Python ssl module documentation):

context.load_cert_chain(certfile="mycertfile", keyfile="mykeyfile")

The code search finds around 1976 lines in 188 projects.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jul 5, 2022

Copy link
Copy Markdown
Member Author

@tiran @serhiy-storchaka: Would you mind to review the updated PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka
serhiy-storchaka reviewed Jul 5, 2022
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You missed updating the documentation.

Please add also new tests in place of the deleted tests.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/imaplib.py
"""IMAP4 client class over SSL connection

Instantiate with: IMAP4_SSL([host[, port[, keyfile[, certfile[, ssl_context[, timeout=None]]]]]])
Instantiate with: IMAP4_SSL([host[, port[, ssl_context[, timeout=None]]]])

@serhiy-storchaka serhiy-storchaka Jul 5, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is completely redundant, because the output of pydoc already contains the constructor signature. The same in other classes in this module, and maybe in other modules.

I would prefer to remove this line and edit the following parameters descriptions, and do it before merging this PR to make backporting easier.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Nov 2, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is already big enough. If you want to enhance the documentation, please change it in a separated change. I don't see how updating this doc is a pre-requirement to remove the two parameters.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/poplib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_ftplib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_httplib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_httplib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

vstinner added 2 commits November 2, 2022 17:51
@vstinner
gh-94172: Remove keyfile, certfile and check_hostname parameters
e4089dc
Remove the keyfile, certfile and check_hostname parameters,
deprecated since Python 3.6, in modules: ftplib, http.client,
imaplib, poplib and smtplib. Use the context parameter (ssl_context
in imaplib) instead.

Parameters following the removed parameters become keyword-only
parameters.

ftplib: Remove the FTP_TLS.ssl_version class attribute: use the
context parameter instead.
@vstinner
Address Serhiy's review
1ba8627
* poplib: stls() context parameter can be positional again
* Keep test_ftplib tests but replace ValueError with TypeError
* Reformat test_httplib
* Keep but update test_tls13_pha() test
@vstinner

vstinner commented Nov 2, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I rebased my PR and addressed @serhiy-storchaka's review.

@serhiy-storchaka: Would you mind to review my updated PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

cc @hugovk

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner merged commit ef0e72b into python:main Nov 3, 2022
@bedevere-bot bedevere-bot removed the awaiting core review label Nov 3, 2022
@vstinner
vstinner deleted the remove_certfile branch November 3, 2022 17:32
@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

I merged my PR which was open for 6 months. I prefer to merge it as soon as possible at the beginning of the Python 3.12 development cycle, to get feedback as soon as possible. If someone sees rooms for enhance, I suggest to open a separated PR ;-)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

Thanks a lot @tiran and @serhiy-storchaka for the many reviews!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk hugovk mentioned this pull request Dec 22, 2022
gh-94172: Update docs for params removed in 3.12 #100431
Merged
@hugovk

hugovk commented Dec 22, 2022

Copy link
Copy Markdown
Member

Please see PR #100431 for some docs updates I missed here.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

donbarbos added a commit to donbarbos/typeshed that referenced this pull request Jan 11, 2026
@donbarbos
[stdlib] Deprecate keyfile, certfile and check_hostname parameters
46f2965
Source: python/cpython#94173
@donbarbos donbarbos mentioned this pull request Jan 11, 2026
[stdlib] Deprecate keyfile, certfile and check_hostname parameters python/typeshed#15259
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka left review comments
+1 more reviewer
@tiran tiran tiran left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@vstinner @serhiy-storchaka @tiran @hugovk @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.