GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-84649: Make TimedRotatingFileHandler use CTIME instead of MTIME by martonivan · Pull Request #24660 · python/cpython

Conversation

@martonivan

@martonivan martonivan commented Feb 26, 2021 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The TimedRotatingFileHandler previously used MTIME attribute of the log
file to detect whether it has to be rotate yet or not. In cases when the
file is changed within the rotation period the MTIME is also updated
to the current time and the rotation never happens. It's more
appropriate to check the file creation time (CTIME) instead.

https://bugs.python.org/issue40469

  • Issue: TimedRotatingFileHandler rotating on use not time #84649

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@martonivan
martonivan requested a review from vsajip as a code owner February 26, 2021 11:37
@the-knights-who-say-ni the-knights-who-say-ni added the CLA not signed label Feb 26, 2021
@the-knights-who-say-ni

the-knights-who-say-ni commented Feb 26, 2021

Copy link
Copy Markdown

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@martonivan

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added the awaiting review label Feb 26, 2021
@the-knights-who-say-ni the-knights-who-say-ni added CLA signed and removed CLA not signed labels Feb 26, 2021
jacobtylerwalls
jacobtylerwalls reviewed Mar 1, 2021
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2021-02-26-13-17-57.bpo-40469.yJHeQg.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@martonivan
martonivan force-pushed the fix-issue-40469 branch from 5335c93 to 197582f Compare March 4, 2021 10:22
@martonivan

martonivan commented Mar 4, 2021

Copy link
Copy Markdown
Contributor Author

I've rebased the branch, squashed the recent fix (about the NEWS typo) into the last commit, as it was partially recommended in the bug report about the flawed testing infrastructure: https://bugs.python.org/issue43382

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions

github-actions Bot commented Apr 4, 2021

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 4, 2021
@vsajip

vsajip commented Sep 2, 2021

Copy link
Copy Markdown
Member

See my suggestion on the issue if you want to progress things further.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@martonivan

martonivan commented Sep 4, 2021

Copy link
Copy Markdown
Contributor Author

See my suggestion on the issue if you want to progress things further.

Thanks Vinay, I'll implement the suggested and acceptable solution as soon as I get there and will update this merge request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

MaxwellDupre

This comment was marked as outdated.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting core review and removed awaiting review labels Apr 5, 2022
@markhallett markhallett mannequin mentioned this pull request Apr 10, 2022
TimedRotatingFileHandler rotating on use not time #84649
Closed
@ezio-melotti ezio-melotti removed the CLA signed label Jul 13, 2022
@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label Aug 4, 2022
serhiy-storchaka
serhiy-storchaka approved these changes Mar 1, 2024
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I was just about to create the same issue.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Mar 1, 2024
@serhiy-storchaka

serhiy-storchaka commented Mar 1, 2024

Copy link
Copy Markdown
Member

@martonivan, the mechanism for the CLA signing was changed, please sign the CLA again, it is necessary for merging. Thank you for your contribution.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@martonivan
martonivan force-pushed the fix-issue-40469 branch from 8c87f3f to 207a12b Compare March 1, 2024 19:57
@martonivan
martonivan requested review from a team, barneygale, ericsnowcurrently, erlend-aasland, jaraco and sethmlarson as code owners March 1, 2024 19:57
@martonivan
martonivan requested a review from hugovk as a code owner March 1, 2024 19:58
@martonivan

martonivan commented Mar 1, 2024

Copy link
Copy Markdown
Contributor Author

Unfortunately I cannot access to the email address that stands in the commit anymore. Tried to reword those commits and rebase the whole tree, but I may caused more issue than I solved. Could you please help me what would be the proper way to implement this change in the commits?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra

JelleZijlstra commented Mar 1, 2024

Copy link
Copy Markdown
Member

Either open a new PR, or reapply your changes to main and force-push to this PR branch to get rid of the problematic changes.

martonivan reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra
JelleZijlstra removed request for a team, JelleZijlstra, abalkin, barneygale, berkerpeksag, cjw296, encukou, ericsnowcurrently, ericvsmith, gpshead, jaraco, pfmoore, vsajip and warsaw March 1, 2024 20:02
@serhiy-storchaka

serhiy-storchaka commented Mar 1, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Well, it is complicated.

  1. First, there are failing tests. They change the modification time to trigger rollover (it is not easy to fake the creation time). It can be fixed if make the code taking the minimum of ctime and mtime.

  2. New tests for ctime are needed to test that old files are rolled over even if mtime is fresh. I see two ways:

    • Create a handler with very short interval (3 seconds), emit a log record, close the handler, wait 1.5 seconds, re-create the handler, emit the record, close the handler, check that the file was not rolled over, wait 1.5 seconds, re-create the handler, emit the record, close the handler, check that the file was rolled over. Exact times may need corrections. There may be surprises on Windows with FAT32.
    • Create a modification of one of currently failing tests. Instead changing the modification time to the past time, change it to the future time. You need also to patch time.time() to make it returning the time from future.

    It would be better to implement both ways.

  3. ctime is actually not the "creation time", but the "node change time", although in many cases there is no difference between them. But it is worth to use also the real creation time which is available on some platforms. It is exposed as the st_birthtime attribute (note that it is float). So perhaps the code should take the minimum os st_birthtime, st_ctime and st_mtime.

martonivan reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@martonivan

martonivan commented Mar 6, 2024

Copy link
Copy Markdown
Contributor Author

I've implemented many of the suggested changes, but realized a huge issue: on Linux, since ctime also changes when the data block changes (practically when mtime changes), the test that checks if only the birthdate is old enough for rotation will always fail. It would be nice the have statx implemented and birth date/creation date known there too, but this issue is already staling for a while: #19125
I could restrict this test only for Windows and MacOS, I guess. I didn't find the proper helper scripts yet in test/support, but I assume this could be a workaround until the PR mentioned above is not merged. What would you suggest? What's the best practice to resolve similar issues?

serhiy-storchaka reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented Mar 7, 2024

Copy link
Copy Markdown
Member

It is okay if this part of the feature is not supported on all platforms. We should do the best of what is currently possible on every platform. The availability of these attributes also depends on the file system.

martonivan reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

martonivan added 2 commits March 7, 2024 23:23
@martonivan
Add missing NEWS item about the change
39d9f1f
@martonivan
pythongh-84649: Make TimedRotatingFileHandler use CTIME instead of MTIME
6731309
The TimedRotatingFileHandler previously only used st_mtime attribute of the
log file to detect whether it has to be rotate yet or not. In cases when the
file is changed within the rotatation period the st_mtime is also updated
to the current time and the rotation never happens.

It's more appropriate to check the file creation time (st_ctime) instead.
Whenever available, the more appropriate st_birthtime will be in use. (This
feature is available on FreeBSD, MacOS and Windows at the moment.) If
the st_mtime would be newer than st_ctime (e.g.: because the inode
related to the file has been changed without any file content
modification), then the earliest attribute will be used.
@martonivan

martonivan commented Mar 7, 2024

Copy link
Copy Markdown
Contributor Author

All the tests are green on the supported operating systems.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Mar 8, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

@martonivan: when sync'ing with main, could you instead please do git merge --no-ff main, and not git rebase? Force-pushes do not play very well with GitHub's UI, and it makes reviewing harder. See also the devguide. Note that all PRs are squashed into a single commit when we merge, so it does not matter if the PR commit history is messy.

serhiy-storchaka and martonivan reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@martonivan

martonivan commented Mar 8, 2024

Copy link
Copy Markdown
Contributor Author

@martonivan: when sync'ing with main, could you instead please do git merge --no-ff main, and not git rebase? Force-pushed do not play very well with GitHub's UI, and it makes reviewing harder. See also the devguide. Note that all PRs are squashed into a single commit when we merge, so it does not matter if the PR commit history is messy.

Noted for further contribution, I'm not sure if any further commits will be necessary for this one.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions

github-actions Bot commented Apr 9, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka added 2 commits June 4, 2026 14:05
@serhiy-storchaka
Merge branch 'main' into fix-issue-40469
3464624
@serhiy-storchaka
Re-wrap long lines.
6e588d4
@vstinner

vstinner commented Jun 5, 2026

Copy link
Copy Markdown
Member

This change added test_rollover_based_on_st_birthtime_only() to test_logging, but the test fails randomly on Windows, macOS and FreeBSD: I created issue gh-150960 to track this regression.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes
+2 more reviewers
@jacobtylerwalls jacobtylerwalls jacobtylerwalls left review comments
@MaxwellDupre MaxwellDupre MaxwellDupre left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

11 participants

@martonivan @the-knights-who-say-ni @vsajip @serhiy-storchaka @JelleZijlstra @erlend-aasland @vstinner @MaxwellDupre @jacobtylerwalls @ezio-melotti @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.