GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-97786: Fix compiler warnings in pytime.c by mdickinson · Pull Request #101826 · python/cpython

Conversation

@mdickinson

@mdickinson mdickinson commented Feb 11, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This PR fixes some compiler warnings in pytime.c, and at the same time fixes our out-of-range double-to-integer checks to properly avoid undefined behaviour.

It's hard to write strictly portable code for this kind of thing, but the new check should work under a set of (very) mild assumptions, that are highly unlikely to be violated on any actual platform that we care about:

  • That time_t is a two's complement signed integer type with no trap representation. (The weakest part of this is the assumption that time_t is a signed integer type at all, but we're already making that assumption.)
  • That _PyTime_t is also a two's complement signed integer type with no trap representation. (This is already true with the current typedef int64_t _PyTime_t; declaration.)
  • That PY_TIME_T_MIN and _PyTime_MIN are within the range of a C double. These days we're assuming IEEE 754 binary64 doubles, so we're safe so long as the integer types _PyTime_t and time_t don't have a width of more than 1024.

Here's the underlying logic for the changes, for the record:

  • We want to check that a C double value x is within the range of a (potentially unknown) signed integer type I with min and max values IMIN and IMAX. More specifically, we want to be sure that a conversion from x to type I will not invoke undefined behaviour; this means that the integer part of x (discarding the fractional part, if any) must be in [IMIN, IMAX].
  • Assuming that I uses two's complement with no trap representation, IMIN must be the negation of a power of two, and IMAX == -1 - IMIN.
  • So IMIN is exactly representable as a double (assuming that it's not larger than 2**1023), and (double)IMIN does not change the value.
  • So the bottom-end check, (double)IMIN <= x, does exactly what we want it to.
  • For the top-end check, x <= IMAX is problematic since the implicit conversion of IMAX to type double may change the value. But assuming that x is an integer (which it is for all cases in this PR), x <= IMAX is mathematically equivalent to x < (IMAX + 1), which with our two's complement assumption is equivalent to x < -IMIN, and we can express this as x < -(double)IMIN.
  • Issue: Compiler warning for _Py_InIntegralTypeRange #97786

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@mdickinson
Fix compiler warnings in pytime.c
622ec0a
@mdickinson mdickinson added needs backport to 3.10 needs backport to 3.11 only security fixes labels Feb 11, 2023
@mdickinson
mdickinson requested review from brettcannon and gpshead February 11, 2023 13:21
@mdickinson
mdickinson requested review from abalkin and pganssle as code owners February 11, 2023 13:21
@bedevere-bot bedevere-bot added the awaiting core review label Feb 11, 2023
@bedevere-bot bedevere-bot mentioned this pull request Feb 11, 2023
Compiler warning for _Py_InIntegralTypeRange #97786
Closed
@mdickinson
Add news
32aee25
@mdickinson mdickinson added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Feb 11, 2023
@bedevere-bot

bedevere-bot commented Feb 11, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @mdickinson for commit 32aee25 🤖

If you want to schedule another build, you need to add the :hammer: test-with-buildbots label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Feb 11, 2023
mdickinson
mdickinson commented Feb 11, 2023
View reviewed changes
Comment thread Python/pytime.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
mdickinson
mdickinson commented Feb 11, 2023
View reviewed changes
Comment thread Python/pytime.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
mdickinson
mdickinson commented Feb 11, 2023
View reviewed changes
Comment thread Python/pytime.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@mdickinson
Add note on nans; remove unnecessary asserts
c381e4c
@mdickinson

mdickinson commented Feb 12, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I've removed the fmod asserts from the code. They were somewhat obscure, and it's not necessary for correctness that the target value is an integer.

The only possible issue with a non-integer intpart is one of false positives: if intpart is a large and negative double in the open interval (PY_TIME_T_MIN - 1, PY_TIME_T_MIN) then:

  • it's safe to cast intpart to time_t, since it's in range after discarding the fractional part, but
  • the condition if (!((double)PY_TIME_T_MIN <= intpart && intpart < -(double)PY_TIME_T_MIN)) will be true, so an overflow would be reported

So the failure mode is benign. But in any case, intpart must be an integer (or possibly an infinity or nan), so the above can't happen.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@mdickinson

mdickinson commented Feb 19, 2023

Copy link
Copy Markdown
Member Author

I think this is ready to merge. @gpshead Do you have bandwidth for a quick re-review?
gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
gpshead
gpshead approved these changes Feb 20, 2023
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Feb 20, 2023
@gpshead
gpshead merged commit b1b375e into python:main Feb 20, 2023
@bedevere-bot bedevere-bot removed the awaiting merge label Feb 20, 2023
@miss-islington

miss-islington commented Feb 20, 2023

Copy link
Copy Markdown
Contributor

Thanks @mdickinson for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington

miss-islington commented Feb 20, 2023

Copy link
Copy Markdown
Contributor

Sorry, @mdickinson and @gpshead, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker b1b375e2670a58fc37cb4c2629ed73b045159918 3.10
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 20, 2023
@mdickinson @miss-islington
pythongh-97786: Fix compiler warnings in pytime.c (pythonGH-101826)
7e25925 
Fixes compiler warnings in pytime.c.
(cherry picked from commit b1b375e)

Co-authored-by: Mark Dickinson <[email protected]>
@miss-islington miss-islington assigned gpshead Feb 20, 2023
@bedevere-bot

bedevere-bot commented Feb 20, 2023

Copy link
Copy Markdown

GH-102062 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Feb 20, 2023
@gpshead gpshead removed the needs backport to 3.10 label Feb 20, 2023
jaraco pushed a commit to jaraco/cpython that referenced this pull request Feb 20, 2023
@mdickinson @jaraco
pythongh-97786: Fix compiler warnings in pytime.c (python#101826)
c6f8395 
Fixes compiler warnings in pytime.c.
@jaraco jaraco mentioned this pull request Feb 20, 2023
gh-88233: zipfile: refactor _strip_extra #102083
Closed
carljm added a commit to carljm/cpython that referenced this pull request Feb 20, 2023
@carljm
Merge branch 'main' into inlinecomp2
9f0fc5b 
* main: (60 commits)
  pythongh-102056: Fix a few bugs in error handling of exception printing code (python#102078)
  pythongh-102011: use sys.exception() instead of sys.exc_info() in docs where possible (python#102012)
  pythongh-101566: Sync with zipp 3.14. (pythonGH-102018)
  pythonGH-99818: improve the documentation for zipfile.Path and Traversable (pythonGH-101589)
  pythongh-88233: zipfile: handle extras after a zip64 extra (pythonGH-96161)
  pythongh-101981: Apply HOMEBREW related environment variables (pythongh-102074)
  pythongh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (pythonGH-101912)
  pythongh-101819: Adapt _io types to heap types, batch 1 (pythonGH-101949)
  pythongh-101981: Build macOS as recommended by the devguide (pythonGH-102070)
  pythongh-97786: Fix compiler warnings in pytime.c (python#101826)
  pythongh-101578: Amend PyErr_{Set,Get}RaisedException docs (python#101962)
  Misc improvements to the float tutorial (pythonGH-102052)
  pythongh-85417: Clarify behaviour on branch cuts in cmath module (python#102046)
  pythongh-100425: Update tutorial docs related to sum() accuracy (FH-101854)
  Add missing 'is' to `cmath.log()` docstring (python#102049)
  pythongh-100210: Correct the comment link for unescaping HTML (python#100212)
  pythongh-97930: Also include subdirectory in makefile. (python#102030)
  pythongh-99735: Use required=True in argparse subparsers example (python#100927)
  Fix incorrectly documented attribute in csv docs (python#101250)
  pythonGH-84783: Make the slice object hashable (pythonGH-101264)
  ...
mdickinson added a commit that referenced this pull request Feb 22, 2023
@miss-islington @mdickinson @hauntsaninja
[3.11] gh-97786: Fix compiler warnings in pytime.c (GH-101826) (#102062)
ddb65c4 
gh-97786: Fix compiler warnings in pytime.c (GH-101826)

Fixes compiler warnings in pytime.c.
(cherry picked from commit b1b375e)

Co-authored-by: Mark Dickinson <[email protected]>
Co-authored-by: Shantanu <[email protected]>
@bedevere-bot

bedevere-bot commented Feb 22, 2023

Copy link
Copy Markdown

GH-102150 is a backport of this pull request to the 3.10 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
carljm added a commit to carljm/cpython that referenced this pull request Feb 22, 2023
@carljm
Merge branch 'main' into inlinecomp
544b30c 
* main: (225 commits)
  pythongh-102056: Fix a few bugs in error handling of exception printing code (python#102078)
  pythongh-102011: use sys.exception() instead of sys.exc_info() in docs where possible (python#102012)
  pythongh-101566: Sync with zipp 3.14. (pythonGH-102018)
  pythonGH-99818: improve the documentation for zipfile.Path and Traversable (pythonGH-101589)
  pythongh-88233: zipfile: handle extras after a zip64 extra (pythonGH-96161)
  pythongh-101981: Apply HOMEBREW related environment variables (pythongh-102074)
  pythongh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (pythonGH-101912)
  pythongh-101819: Adapt _io types to heap types, batch 1 (pythonGH-101949)
  pythongh-101981: Build macOS as recommended by the devguide (pythonGH-102070)
  pythongh-97786: Fix compiler warnings in pytime.c (python#101826)
  pythongh-101578: Amend PyErr_{Set,Get}RaisedException docs (python#101962)
  Misc improvements to the float tutorial (pythonGH-102052)
  pythongh-85417: Clarify behaviour on branch cuts in cmath module (python#102046)
  pythongh-100425: Update tutorial docs related to sum() accuracy (FH-101854)
  Add missing 'is' to `cmath.log()` docstring (python#102049)
  pythongh-100210: Correct the comment link for unescaping HTML (python#100212)
  pythongh-97930: Also include subdirectory in makefile. (python#102030)
  pythongh-99735: Use required=True in argparse subparsers example (python#100927)
  Fix incorrectly documented attribute in csv docs (python#101250)
  pythonGH-84783: Make the slice object hashable (pythonGH-101264)
  ...
mdickinson added a commit that referenced this pull request Feb 26, 2023
@mdickinson @gpshead
[3.10] gh-97786: Fix compiler warnings in pytime.c (GH-101826) (#102150)
5b610b5 
* [3.10] gh-97786: Fix compiler warnings in pytime.c (GH-101826)

Fixes compiler warnings in pytime.c..
(cherry picked from commit b1b375e)

Co-authored-by: Mark Dickinson <[email protected]>

* Add comment about the casts

---------

Co-authored-by: Gregory P. Smith <[email protected]>
This was referenced Mar 4, 2023
_PyTime_FromDouble() fails to detect an integer overflow when converting a C double to a C int64_t #83458
Closed
Overflow when casting from double to time_t, and_PyTime_t. #78604
Closed
Remove unused internal macros #102415
Merged
mdickinson added a commit that referenced this pull request Mar 4, 2023
@mdickinson
Remove unused internal macros (#102415)
b022250 
Since #101826 was merged, the internal macro `_Py_InIntegralTypeRange` is unused, as are its supporting macros `_Py_IntegralTypeMax` and `_Py_IntegralTypeMin`. This PR removes them.

Note that `_Py_InIntegralTypeRange` doesn't actually work as advertised - it's not a safe way to avoid undefined behaviour in an integer to double conversion.
hugovk pushed a commit to hugovk/cpython that referenced this pull request Mar 6, 2023
@mdickinson @hugovk
Remove unused internal macros (python#102415)
01ef4a6 
Since python#101826 was merged, the internal macro `_Py_InIntegralTypeRange` is unused, as are its supporting macros `_Py_IntegralTypeMax` and `_Py_IntegralTypeMin`. This PR removes them.

Note that `_Py_InIntegralTypeRange` doesn't actually work as advertised - it's not a safe way to avoid undefined behaviour in an integer to double conversion.
@vstinner

vstinner commented Apr 5, 2023

Copy link
Copy Markdown
Member

Thanks for fixing this old annoying compiler bug, @mdickinson!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Apr 5, 2023

Copy link
Copy Markdown
Member

I added math.nextafter() and math.ulp() to Python 3.9 to help me to understand this issue :-) But then I failed to come up with a fix for this warning.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@mdickinson

mdickinson commented Apr 6, 2023

Copy link
Copy Markdown
Member Author

math.nextafter and math.ulp are very good to have, regardless! So thank you for that.
vstinner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@mdickinson
mdickinson deleted the fix-pytime-c-compiler-warnings branch April 6, 2023 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead approved these changes
@brettcannon brettcannon Awaiting requested review from brettcannon
@pganssle pganssle Awaiting requested review from pganssle pganssle is a code owner
@abalkin abalkin Awaiting requested review from abalkin

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@gpshead gpshead

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@mdickinson @bedevere-bot @miss-islington @vstinner @gpshead
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.