GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-1635741: Port _locale extension module to multiphase initialization (PEP 489) by shihai1991 · Pull Request #18358 · python/cpython

Conversation

@shihai1991

@shihai1991 shihai1991 commented Feb 5, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

https://bugs.python.org/issue1635741

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@shihai1991
Port _locale extension module to multiphase initialization (PEP 489)
4ce0c7c
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Feb 5, 2020
@bedevere-bot bedevere-bot added the awaiting review label Feb 5, 2020
@blurb-it
📜🤖 Added by blurb_it.
c0e0174
@shihai1991
shihai1991 requested review from encukou and ncoghlan February 5, 2020 07:57
@codecov

codecov Bot commented Feb 5, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Codecov Report

Merging #18358 into master will decrease coverage by 0.04%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18358      +/-   ##
==========================================
- Coverage   82.11%   82.07%   -0.05%     
==========================================
  Files        1955     1955              
  Lines      588624   584187    -4437     
  Branches    44406    44464      +58     
==========================================
- Hits       483366   479456    -3910     
+ Misses      95615    95110     -505     
+ Partials     9643     9621      -22     
Impacted Files Coverage Δ
Lib/distutils/tests/test_bdist_rpm.py 30.00% <0.00%> (-65.00%) ⬇️
Lib/distutils/command/bdist_rpm.py 7.63% <0.00%> (-56.88%) ⬇️
Modules/_decimal/libmpdec/umodarith.h 80.76% <0.00%> (-19.24%) ⬇️
Lib/test/test_urllib2net.py 76.92% <0.00%> (-13.85%) ⬇️
Lib/test/test_smtpnet.py 78.57% <0.00%> (-7.15%) ⬇️
Lib/ftplib.py 63.85% <0.00%> (-6.06%) ⬇️
Lib/test/test_ftplib.py 87.11% <0.00%> (-4.72%) ⬇️
Tools/scripts/db2pickle.py 17.82% <0.00%> (-3.97%) ⬇️
Tools/scripts/pickle2db.py 16.98% <0.00%> (-3.78%) ⬇️
Lib/test/test_socket.py 71.94% <0.00%> (-3.77%) ⬇️
... and 399 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 850a4bd...f3a1d08. Read the comment docs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@shihai1991
shihai1991 requested a review from vstinner February 17, 2020 12:32
vstinner
vstinner reviewed Feb 17, 2020
View reviewed changes
Comment thread Modules/_localemodule.c Outdated
state = PyModule_GetState(self);
if (state == NULL) {
return NULL;
}

@vstinner vstinner Feb 17, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that it's worth it to check for error: this function cannot fail, self is always a module object.

I would prefer a function like:

static inline _locale_state* get_locale_state(PyObject *mod)
{
    void *state = PyModule_GetState(mod);
    assert(state != NULL);
    return (_locale_state*)state;
}

And then use:

PyErr_SetString(get_locale_state(self)->Error, "invalid locale category"); 

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@shihai1991 shihai1991 Feb 17, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, I update this PR.

Do we need add a global macros of get_state?
I found some other extension modules use local macro do this behavior too. such as: https://github.com/python/cpython/blob/master/Modules/_hashopenssl.c#L55

Thanks a million, victor. You helpe me merge much PRs today ;)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@encukou encukou Feb 18, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A static inline function is a bit better than a macro.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@shihai1991 shihai1991 Feb 18, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I remember victor have explained in other bpo/PR(oh, I forgot which one is). It's easy for debuging and something else.
I will try to add this inline function in free time.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
shihai1991 added 2 commits February 17, 2020 23:05
@shihai1991
add get_locale_state()
3aaf488
@shihai1991
Merge branch 'bpo_1635741_9' of https://github.com/shihai1991/cpython …
8e0e2bf
…into bpo_1635741_9
vstinner
vstinner reviewed Feb 17, 2020
View reviewed changes
Comment thread Modules/_localemodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Feb 17, 2020
View reviewed changes
Comment thread Modules/_localemodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@shihai1991
fix format bug
2a4ac1b
@shihai1991

shihai1991 commented Feb 18, 2020

Copy link
Copy Markdown
Member Author

Looks CI gate of macOS have broken, I saw some other PR have this problem.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou
encukou requested changes Feb 18, 2020
View reviewed changes
Comment thread Modules/_localemodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting review label Feb 18, 2020
@bedevere-bot

bedevere-bot commented Feb 18, 2020

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added the awaiting changes label Feb 18, 2020
@shihai1991
locale extension module add clean slots
70fddf4
@shihai1991

shihai1991 commented Feb 18, 2020

Copy link
Copy Markdown
Member Author

I remove assert operation in get_locale_state(), becasue this assert operation always failed when I add traverse slot.
I am not check the code detail. From this result, removing the assert() from get_locale_state() is ok, because we have chance get A NULL state.

some error info like:

$ ./python -E -S -m sysconfig --generate-posix-vars
python: ./Modules/_localemodule.c:52: get_locale_state: Assertion `state != ((void *)0)' failed.
[1]    17894 abort      ./python -E -S -m sysconfig --generate-posix-vars

and the trace back:

#0  0x00007ffff6ce2207 in raise () from /lib64/libc.so.6
#1  0x00007ffff6ce38f8 in abort () from /lib64/libc.so.6
#2  0x00007ffff6cdb026 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff6cdb0d2 in __assert_fail () from /lib64/libc.so.6
#4  0x00000000005986f3 in get_locale_state (m=<optimized out>) at ./Modules/_localemodule.c:52
#5  locale_traverse (m=<optimized out>, visit=0x462d26 <bad_traverse_test>, arg=0x0) at ./Modules/_localemodule.c:780
#6  0x000000000046384c in PyModule_FromDefAndSpec2TraceRefs (def=def@entry=0x96e680 <_localemodule>, spec=spec@entry=0x7ffff00567d0, module_api_version=module_api_version@entry=1013)
    at Objects/moduleobject.c:370
#7  0x0000000000512153 in _imp_create_builtin (module=<optimized out>, spec=0x7ffff00567d0) at Python/import.c:1300
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@shihai1991

shihai1991 commented Feb 18, 2020

Copy link
Copy Markdown
Member Author

I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Feb 18, 2020
@bedevere-bot

bedevere-bot commented Feb 18, 2020

Copy link
Copy Markdown

Thanks for making the requested changes!

@encukou: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot
bedevere-bot requested a review from encukou February 18, 2020 15:38
@encukou
Add back assertion in get_locale_state
f3a1d08
@encukou

encukou commented Feb 25, 2020

Copy link
Copy Markdown
Member

The assertion was good! If you remove it, you should add if (state != NULL) checks after any use of get_locale_state.
It's actually only the traverse/clear/free functions that can get a NULL module state. They can use PyModule_GetState with a NULL check directly.

I hope you don't mind me pushing to this branch directly – I already made the change and ran tests on it locally.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@shihai1991

shihai1991 commented Feb 25, 2020

Copy link
Copy Markdown
Member Author

The assertion was good! If you remove it, you should add if (state != NULL) checks after any use of get_locale_state.
It's actually only the traverse/clear/free functions that can get a NULL module state. They can use PyModule_GetState with a NULL check directly.

I hope you don't mind me pushing to this branch directly – I already made the change and ran tests on it locally.

petr, pls go ahead ;)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@shihai1991

shihai1991 commented Feb 25, 2020

Copy link
Copy Markdown
Member Author

The assertion was good! If you remove it, you should add if (state != NULL) checks after any use of get_locale_state.
It's actually only the traverse/clear/free functions that can get a NULL module state. They can use PyModule_GetState with a NULL check directly.

I hope you don't mind me pushing to this branch directly – I already made the change and ran tests on it locally.

Hi, petr. Do you have wins env to test this PR? #18608.
This' PR's failed info is weird. (Looks like module state have been confused~)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou
encukou approved these changes Feb 25, 2020
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I plan to merge when I have time to watch the buildbots afterwards.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

shihai1991 reacted with hooray emoji
All reactions
  • 🎉 1 reaction
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting change review labels Feb 25, 2020
vstinner
vstinner reviewed Feb 29, 2020
View reviewed changes
Comment thread Modules/_localemodule.c
locale_traverse(PyObject *m, visitproc visit, void *arg)
{
_locale_state *state = (_locale_state*)PyModule_GetState(m);
if (state) {

@vstinner vstinner Feb 29, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I asked on your two other PRs (binascii, audioop), I don't think that state can be NULL here. Same remark in locale_clear().

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@shihai1991 shihai1991 Mar 1, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, victor. There have some description info about traverse slot in https://docs.python.org/3/c-api/module.html?highlight=pymoduledef#c.PyModuleDef:

A traversal function to call during GC traversal of the module object, or NULL if not needed. This function may be called before module state is allocated (PyModule_GetState() may return NULL), and before the Py_mod_exec function is executed.

So this behavior is a planned behavior, isn't it?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Mar 2, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss that in https://bugs.python.org/issue39824

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@encukou encukou Mar 11, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The discussion won't be over before Nick Coghlan is satisfied (and that's a good thing!)

Meanwhile, this PR is correct according to current behavior and documentation. I don't think bpo-39824 should block it.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Mar 11, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright. I expected https://bugs.python.org/issue39824 to be resolved quick, but I'm ok to merge @shihai1991 PR's first, and revisit the code later if we decided that m_clear/m_free cannot be called with a NULL state.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
vstinner merged commit a158168 into python:master Mar 11, 2020
@bedevere-bot bedevere-bot removed the awaiting merge label Mar 11, 2020
@kylotan kylotan mannequin mentioned this pull request Sep 19, 2022
Py_Finalize() doesn't clear all Python objects at exit #44470
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner left review comments
@encukou encukou encukou approved these changes
@ncoghlan ncoghlan Awaiting requested review from ncoghlan

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@shihai1991 @bedevere-bot @encukou @vstinner @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.