GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-31031: Unify duplicate bits_in_digit and bit_length by niklasf · Pull Request #2866 · python/cpython

Conversation

@niklasf

@niklasf niklasf commented Jul 25, 2017 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue31031

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jul 25, 2017
@brettcannon brettcannon added the awaiting review label Feb 2, 2018
auvipy
auvipy suggested changes Jun 2, 2019
View reviewed changes

@auvipy auvipy left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check the build errors plz

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting core review and removed awaiting review labels Jun 2, 2019
@niklasf
niklasf force-pushed the bpo-31031-consolidate-bit-length branch from ccdbe59 to 900f7a8 Compare June 2, 2019 14:15
vstinner
vstinner requested changes Jun 4, 2019
View reviewed changes
Comment thread Include/pymath.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Include/pymath.h Outdated

@vstinner vstinner Jun 4, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use PyAPI_FUNC(unsigned int) rather than extern.

By the way, _Py symbols should be moved to a new Include/cython/pymath.h header IMHO, and excluded from the stable API.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mdickinson and ZackerySpytz reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

@niklasf niklasf Jun 5, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(1) Done.

(2) Sounds reasonable. Would you prefer to start this new file now (and move the other private functions later), or a seperate patch that moves all of the existing private functions including this new one?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/pymath.c Outdated

@vstinner vstinner Jun 4, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we unroll manually the loop using SIZEOF_LONG? For example, 4 bytes long need 0 loop. 8 bytes long only require a single if(). There is no platform supported by Python with long larger than 8 bytes (64 bits).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@mdickinson mdickinson Jun 4, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A 4-byte long would need up to 6 iterations (and an 8-byte long up to 11): we're only removing 6 bits per iteration.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@niklasf niklasf Jun 5, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can probably be optimized to work without branching (using the __clz intrinsic or equivalents). But I'd prefer to keep this patch as a pure refactoring, simply chosing the better of the existing implementations, for now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 5, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, I didn't notice that it's "only" 6 bits per iteration. I read 32 bits by mistake. You can ignore my comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 5, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using clz was discussed in: https://bugs.python.org/issue29782

But it was rejected: https://bugs.python.org/issue29782#msg290973

Maybe the best we can do is to add a short comment to explain why we use a "naive" implementation (not really naive, it uses a precomputed table) linking to bpo-29782.

Would you mind to add such short comment?

By the way, int.bit_length() has a complexity of compleO(1) in practice, no? (I consider that this function has a complexity of O(1), especially when you consider Python int objects made of many "Python digits".)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting core review labels Jun 4, 2019
@bedevere-bot

bedevere-bot commented Jun 4, 2019

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mdickinson
mdickinson reviewed Jun 4, 2019
View reviewed changes
Comment thread Modules/mathmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@niklasf
niklasf force-pushed the bpo-31031-consolidate-bit-length branch 2 times, most recently from f225c8e to dcaa962 Compare June 5, 2019 10:07
@niklasf

niklasf commented Jun 5, 2019

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing. I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Jun 5, 2019
@bedevere-bot

bedevere-bot commented Jun 5, 2019

Copy link
Copy Markdown

Thanks for making the requested changes!

@vstinner: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Jun 5, 2019
View reviewed changes
Comment thread Python/pymath.c Outdated

@vstinner vstinner Jun 5, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to convert this function to an static inline function only exposed in the internal C API?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@mdickinson mdickinson Jun 5, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure, the declaration in pymath.h should be protected by an #ifndef Py_LIMITED_API. @vstinner: I'm not sure I'm understanding you correctly, though. Making this static would make it unavailable to the other files that need it, wouldn't it?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@mdickinson mdickinson Jun 5, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see: you're suggesting moving the entire function definition to pymath.h, and making it static inline. Is that right?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jun 5, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Sorry, my comment was unclear. I converted some macros to static inline functions in header files.

I don't know if it would be worth it in term of performance.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@mdickinson mdickinson Jun 6, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be worth a try; I'd be surprised if it makes a measurable difference, but we won't know without testing.

But if not, we do need the Py_LIMITED_API guard: we don't want to expose _Py_bit_length to 3rd party libraries.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner vstinner added the skip news label Jun 7, 2019
vstinner
vstinner reviewed Jun 7, 2019
View reviewed changes
Comment thread Include/pymath.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@niklasf
bpo-31031: Unify duplicate bits_in_digit and bit_length
7b4198c
@niklasf
niklasf force-pushed the bpo-31031-consolidate-bit-length branch from dcaa962 to 7b4198c Compare June 12, 2019 14:42
@csabella
csabella requested review from mdickinson and vstinner January 16, 2020 11:27
vstinner
vstinner approved these changes Jan 16, 2020
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting change review labels Jan 16, 2020
@vstinner
vstinner merged commit c5b7900 into python:master Jan 16, 2020
@bedevere-bot bedevere-bot removed the awaiting merge label Jan 16, 2020
@vstinner

vstinner commented Jan 16, 2020

Copy link
Copy Markdown
Member

Thanks @niklasf, sorry for slow reviews and for being so pedantic :-) But it was worth it, I like the final change ;-)

I looked again at rejected https://bugs.python.org/issue29782 and #594 I was going to ask further change on this PR to prepare the code in case if someone wants to experiment again more efficient functions... But nah, the code is ok and can be updated later.

auvipy reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

shihai1991 pushed a commit to shihai1991/cpython that referenced this pull request Jan 31, 2020
@niklasf @shihai1991
bpo-31031: Unify duplicate bits_in_digit and bit_length (pythonGH-2866)
ae7bf81
Add _Py_bit_length() to unify duplicate bits_in_digit() and bit_length().
@niklasf niklasf mentioned this pull request Jun 8, 2020
bpo-29782: Consolidate _Py_Bit_Length() #20739
Merged
niklasf added a commit to niklasf/cpython that referenced this pull request Jun 8, 2020
@niklasf
bpo-29782: Consolidate _Py_Bit_Length()
281009c
In pythonGH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. pythonGH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

* Move the function to the new header.
* Changed return type to match __builtin_clzl and reviewed usage.
* Use intrinsics where available.
* Pick a (mostly theoretical) fallback implementation suitable for
  inlining.
vstinner pushed a commit that referenced this pull request Jun 15, 2020
@niklasf
bpo-29782: Consolidate _Py_Bit_Length() (GH-20739)
794e7d1
In GH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. GH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

* Move the function to the new header.
* Changed return type to match __builtin_clzl() and reviewed usage.
* Use intrinsics where available.
* Pick a fallback implementation suitable for inlining.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes
@mdickinson mdickinson Awaiting requested review from mdickinson
+1 more reviewer
@auvipy auvipy auvipy requested changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@niklasf @bedevere-bot @vstinner @mdickinson @auvipy @brettcannon @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.