GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-116608: Ignore UTF-16 BOM in importlib.resources._functional tests by encukou · Pull Request #117569 · python/cpython

Conversation

@encukou

@encukou encukou commented Apr 5, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

To test the errors argument, we read a UTF-16 file as UTF-8 with "backslashreplace" error handling. However, the utf-16 codec adds an endian-specific byte-order mark, so on big-endian machines the expectation doesn't match the test file (which was saved on a little-endian machine).

Use endswith to ignore the BOM.

  • Issue: Un-deprecate functional API for importlib resources & add subdirectory support #116608

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@encukou
pythongh-116609: Ignore UTF-16 BOM in importlib.resources._functional…
26ae210 
… tests

To test the `errors` argument, we read a UTF-16 file as UTF-8
with "backslashreplace" error handling. However, the utf-16
codec adds an endian-specific byte-order mark, so on big-endian
machines the expectation doesn't match the test file (which was
saved on a little-endian machine).

Use endswith to ignore the BOM.
@encukou
encukou requested review from FFY00, jaraco and warsaw as code owners April 5, 2024 13:46
@bedevere-app bedevere-app Bot added tests Tests in the Lib/test dir awaiting core review labels Apr 5, 2024
@encukou encukou changed the title gh-116609: Ignore UTF-16 BOM in importlib.resources._functional tests gh-116608: Ignore UTF-16 BOM in importlib.resources._functional tests Apr 5, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Apr 5, 2024
Un-deprecate functional API for importlib resources & add subdirectory support #116608
Open
@encukou encukou added the skip news label Apr 5, 2024
@encukou

encukou commented Apr 5, 2024

Copy link
Copy Markdown
Member Author

!buildbot s390x
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Apr 5, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit 26ae210 🤖

The command will test the builders whose names match following regular expression: s390x

The builders matched are:

  • s390x Fedora Rawhide Clang Installed PR
  • s390x Fedora Rawhide Clang PR
  • s390x Fedora LTO PR
  • s390x Fedora Refleaks PR
  • s390x RHEL7 LTO + PGO PR
  • s390x Fedora LTO + PGO PR
  • s390x Fedora Clang PR
  • s390x Fedora PR
  • s390x Fedora Rawhide LTO PR
  • s390x Fedora Rawhide PR
  • s390x RHEL8 LTO PR
  • s390x Fedora Rawhide Refleaks PR
  • s390x Fedora Clang Installed PR
  • s390x RHEL8 PR
  • s390x Fedora Rawhide LTO + PGO PR
  • s390x RHEL8 Refleaks PR
  • s390x RHEL8 LTO + PGO PR
  • s390x RHEL7 PR
  • s390x RHEL7 LTO PR
  • s390x RHEL7 Refleaks PR
  • s390x SLES PR
  • s390x Debian PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@encukou
encukou merged commit 4d4a6f1 into python:main Apr 5, 2024
@bedevere-app bedevere-app Bot removed the awaiting core review label Apr 5, 2024
@encukou
encukou deleted the importlib-tests-be branch April 5, 2024 15:00
@zooba

zooba commented Apr 8, 2024

Copy link
Copy Markdown
Member

@encukou Out of interest, was the endswith necessary? I thought using utf-16-le would strip the BOM automatically, and the issue you were hitting is that utf-16-be (implied by utf-16 on BE machines) was rejecting it. Explicitly specifying -le should have worked, I'd thought.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@encukou @diegorusso
pythongh-116608: Ignore UTF-16 BOM in importlib.resources._functional…
19ea065 
… tests (pythonGH-117569)

pythongh-116609: Ignore UTF-16 BOM in importlib.resources._functional tests

To test the `errors` argument, we read a UTF-16 file as UTF-8
with "backslashreplace" error handling. However, the utf-16
codec adds an endian-specific byte-order mark, so on big-endian
machines the expectation doesn't match the test file (which was
saved on a little-endian machine).

Use endswith to ignore the BOM.
@jaraco

jaraco commented Aug 14, 2024

Copy link
Copy Markdown
Member

This change needs to be applied to importlib_resources. It looks like a related issue was reported in python/importlib_resources#312.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@jaraco jaraco mentioned this pull request Aug 14, 2024
importlib_resources/tests/test_functional.py test failures on big endian architectures python/importlib_resources#312
Closed
jaraco added a commit to python/importlib_resources that referenced this pull request Aug 14, 2024
@jaraco
Merge fix for UTF-16 bom handling in tests.
ca5f0e1 
Closes #312. Closes #313. Ref python/cpython#117569
@jaraco jaraco mentioned this pull request Sep 12, 2024
Buildbot failure in test_importlib UTF-16 tests #123994
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jaraco jaraco Awaiting requested review from jaraco jaraco is a code owner
@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner
@FFY00 FFY00 Awaiting requested review from FFY00 FFY00 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news tests Tests in the Lib/test dir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@encukou @bedevere-bot @zooba @jaraco
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.