Description

pganssle

opened on Sep 25, 2024

Issue body actions

Bug report

Bug description:

Apparently the strftime parser treats \x00 as "end of string" in the format code, and the remainder of the string is ignored:

>>> from datetime import datetime
>>> datetime(2024, 9, 25).strftime("\x00%Y-%m-%d")
""

I would have expected:

>>> from datetime import datetime
>>> datetime(2024, 9, 25).strftime("\x00%Y-%m-%d")
"\x002024-09-25"

Discovered this when adding some hypothesis tests for strptime/strftime. I suspect again that if you include a null character in your datetime format string you should expect something to act weird as hell about it, but we should probably fix this anyway if it's not too costly.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-124531: Fix strftime() with embedded null characters #125003
• gh-52551: Fix encoding issues in strftime() #125193

Activity

1. pganssle

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Sep 25, 2024
2. pganssle

   added this to Date and time issues 🕰️on Sep 25, 2024
3. JelleZijlstra commented on Sep 25, 2024

   

   JelleZijlstra

   on Sep 25, 2024

   Member

   More actions

   I guess this makes sense if we create a C string and pass this to the C strftime function. Perhaps we should raise an error in our code if we encounter a null character in the format string.
4. pganssle commented on Sep 25, 2024

   

   pganssle

   on Sep 25, 2024

   MemberAuthor

   More actions

   I guess this makes sense if we create a C string and pass this to the C strftime function. Perhaps we should raise an error in our code if we encounter a null character in the format string.

   I think we can just escape them and then un-escape them before returning, no?

   We handle this just fine in datetime.fromisoformat:

   >>> datetime.now().isoformat(sep="\x00")
   '2024-09-25\x0013:22:39.621972'
   >>> datetime.fromisoformat(datetime.now().isoformat(sep="\x00"))
   datetime.datetime(2024, 9, 25, 13, 22, 52, 309562)
5. Mariatta commented on Sep 27, 2024

   

   Mariatta

   on Sep 27, 2024

   Member

   More actions

   I also think it shouldn't raise error in this case. My expectation is the strftime shouldn't care about anything that isn't datetime format specifier, so it should ignore the \x00 instead of treating it as end of string.

   Reacted by Jan Kaliszewski
6. serhiy-storchaka

   self-assigned this

   on Oct 5, 2024
7. serhiy-storchaka

   added

   3.12only security fixesonly security fixes

   3.13only security fixesonly security fixes

   3.14bugs and security fixesbugs and security fixes

   on Oct 5, 2024
8. serhiy-storchaka

   added a commit that references this issue on Oct 5, 2024

   pythongh-124531: Fix strftime() with embedded null characters

   166b6fe
9. serhiy-storchaka commented on Oct 5, 2024

   

   serhiy-storchaka

   on Oct 5, 2024

   Member

   More actions

   There are many other bugs in strftime(), and I think that fixing this issue gives a key to fix #78662 and #52551.
10. serhiy-storchaka commented on Oct 7, 2024

    

    serhiy-storchaka

    on Oct 7, 2024

    Member

    More actions

    This approach did not work on platforms without wstrftime() because PyUnicode_EncodeLocale() does not support embedded null characters. So I was forced to write more complex and generic patch that fixes also other issues. All that remains is to write the tests, I will do it tomorrow.
11. serhiy-storchaka

    added 2 commits that reference this issue on Oct 9, 2024

    pythongh-124531: Fix encoding issues in strftime()

    f340383

    pythongh-52551: Fix encoding issues in strftime()

    c9f1156
12. serhiy-storchaka

    linked a pull request that will close this issuegh-52551: Fix encoding issues in strftime() #125193on Oct 9, 2024
13. serhiy-storchaka

    added a commit that references this issue on Oct 17, 2024

    gh-52551: Fix encoding issues in strftime() (GH-125193)

    ad3eac1
14. serhiy-storchaka

    closed this as completedin #125193on Oct 17, 2024
15. github-project-automation

    moved this to Done in Date and time issues 🕰️on Oct 17, 2024
16. serhiy-storchaka

    added 3 commits that reference this issue on Oct 17, 2024

    [3.13] pythongh-52551: Fix encoding issues in strftime() (pythonGH-12…

    13ef787

    pythongh-52551: Fix encoding issues in strftime() (pythonGH-125193)

    4019173

    [3.13] gh-52551: Fix encoding issues in strftime() (GH-125193) (GH-12…

    08ccbb9
17. miss-islington

    added a commit that references this issue on Oct 17, 2024

    [3.13] pythongh-52551: Fix encoding issues in strftime() (pythonGH-12…

    a34caba
18. serhiy-storchaka

    added a commit that references this issue on Oct 17, 2024

    [3.12] gh-52551: Fix encoding issues in strftime() (GH-125193) (GH-12…

    232b303
19. ebonnal

    added a commit that references this issue on Jan 12, 2025

    pythongh-52551: Fix encoding issues in strftime() (pythonGH-125193)

    3a7c158

Issue actions