GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-144415: Android testbed fixes by mhsmith · Pull Request #142912 · python/cpython

Conversation

@mhsmith

@mhsmith mhsmith commented Dec 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Fixes for several issues found while building NumPy and related packages:

  • Gradle automatically decompresses assets whose filenames end with ".gz", which broke some packages' tests.

    • Worked around by escaping these filenames.

  • The stdout/stderr logging couldn't distinguish between log messages which were triggered by a newline, and those triggered by a manual call to flush. This particularly affected pytest's "one dot per test" progress indicator, which was printing every dot on a separate line.

    • I didn't end up needing the "trailing space" idea I mentioned the other day, because it turns out the newline information was already in the log, it just wasn't visible via the logcat tool in --format tag mode. Fixed by switching to --binary mode.
    • The logcat tool on API level 33 buffers its output incorrectly in --binary mode, causing lines to be delayed or lost. Fixed by reducing the tesbed to API level 32.
  • Closes Android testbed logs have excessive newlines #144415

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Herklos reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@mhsmith
Avoid automatic decompression of .gz files
aa79914
@mhsmith mhsmith added skip issue skip news needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Dec 18, 2025
This was referenced Dec 18, 2025
BLD: Add Android support numpy/numpy#30412
Merged
BLD: Add Android support pandas-dev/pandas#63405
Closed
@StanFromIreland StanFromIreland added the OS-android label Dec 18, 2025
This was referenced Jan 11, 2026
BLD: Add Android support scipy/scipy#24341
Draft
BLD Add Android support scikit-learn/scikit-learn#33055
Draft
@mhsmith mhsmith mentioned this pull request Jan 21, 2026
Android improvements needed for building NumPy and related packages pypa/cibuildwheel#2695
Merged
mhsmith added 4 commits January 28, 2026 21:50
@mhsmith
Set aaptOptions.ignoreAssetsPattern to something nonexistent but non-…
d37787c 
…empty
@mhsmith
Merge branch 'main' into android-2025-12
4c866e3
@mhsmith
Merge remote-tracking branch 'origin/main' into android-2025-12
eca6674
@mhsmith
Improve transparency of stdout and stderr logging
84e71c3
@mhsmith

mhsmith commented Feb 2, 2026

Copy link
Copy Markdown
Member Author

!buildbot android
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

This comment was marked as outdated.

Sign in to view
@mhsmith

mhsmith commented Feb 2, 2026

Copy link
Copy Markdown
Member Author

!buildbot aarch64 android
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

This comment was marked as outdated.

Sign in to view
@mhsmith
Correct stderr log level
29ec458
@mhsmith

mhsmith commented Feb 2, 2026

Copy link
Copy Markdown
Member Author

!buildbot android
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Feb 2, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @mhsmith for commit 29ec458 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F142912%2Fmerge

The command will test the builders whose names match following regular expression: android

The builders matched are:

  • aarch64 Android PR
  • AMD64 Android PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@mhsmith
mhsmith marked this pull request as ready for review February 2, 2026 21:59
@mhsmith
mhsmith requested a review from freakboy3742 as a code owner February 2, 2026 21:59
@bedevere-app bedevere-app Bot added the awaiting review label Feb 2, 2026
@mhsmith

mhsmith commented Feb 2, 2026

Copy link
Copy Markdown
Member Author

@freakboy3742: I didn't manage to complete the emulator RAM size fix today, so let's merge this PR without it. So far RAM size has only been a blocker for Pandas, and that's unlikely to be ready to release within the next two months anyway, given how many other things have to be released first.

GitHub Actions is broken at the moment; please rerun CI once that's resolved.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
freakboy3742
freakboy3742 reviewed Feb 3, 2026
View reviewed changes

@freakboy3742 freakboy3742 left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The buildbot and CI runs suggest this change works; but I want to make sure I follow how these changes avoid the "pytest progress" issue.

I can see this moves to a "raw binary" mode of reading the ADB log, which means we get the raw messages being posted rather than the logged output.

That then means that we're reading "raw" log output at the level of individual messages, rather than log output that is buffered by newline/flush elements; so when we're streaming log output, we have visibility on the newlines that have been printed to stdout/err, rather than that content being part of the buffering process.

The "leading space" trick is needed to preserve newlines that would otherwise been trimmed from the log by an internal logcat mechanism. AFAICT, this does mean that successive writes of hello and \nworld would result in a space at the end of the "hello" as printed to the screen - but that's visually inert unless you're really unlucky and trigger size of screen line wrapping, so it's worth living with.

It also means that anyone monitoring the "text mode" ADB stream will still get "one dot per line" output for pytest suites etc (e.g., Briefcase's Android runner logs); but the same "binary mode" logger could be used there as well.

Have I understood what is going on here?

Assuming I have - on a meta level, I think this is a big enough change that it needs a ticket and NEWS entry - if only to provide tracking for the backports.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@mhsmith mhsmith changed the title Android testbed fixes gh-144415: Android testbed fixes Feb 3, 2026
@mhsmith
Add news entry
09c8f2e
@mhsmith mhsmith removed the skip issue label Feb 3, 2026
@mhsmith mhsmith removed the skip news label Feb 3, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Feb 3, 2026
Android testbed logs have excessive newlines #144415
Closed
@mhsmith

mhsmith commented Feb 3, 2026

Copy link
Copy Markdown
Member Author

That's mostly correct, except for a couple of details:

That then means that we're reading "raw" log output at the level of individual messages, rather than log output that is buffered by newline/flush elements

We were always getting individual messages: the Logcat system doesn't do any buffering itself. The issue was that when you use adb logcat in text mode, it prints a newline after every log message regardless of whether the original message had a newline or not. This makes it difficult to reconstruct non-line-based output like pytest's progress bar.

AFAICT, this does mean that successive writes of hello and \nworld would result in a space at the end of the "hello" as printed to the screen - but that's visually inert unless you're really unlucky and trigger size of screen line wrapping, so it's worth living with.

If you called flush after each of these writes, then the behavior would be as you say. However, if you didn't manually flush, then Python itself would buffer until it hit the newline, so the newline would be logged in the same message as the "hello", it would no longer be at the start of a message, and no extra space would be added. This is probably the more common case.

I think this is a big enough change that it needs a ticket and NEWS entry

Done.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
freakboy3742
freakboy3742 approved these changes Feb 3, 2026
View reviewed changes

@freakboy3742 freakboy3742 left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for those explanations 👍

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Feb 3, 2026
@freakboy3742
freakboy3742 merged commit cb1dc91 into python:main Feb 3, 2026
55 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington-app

miss-islington-app Bot commented Feb 3, 2026

Copy link
Copy Markdown

Thanks @mhsmith for the PR, and @freakboy3742 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting merge label Feb 3, 2026
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 3, 2026
@mhsmith @miss-islington
pythongh-144415: Android testbed fixes (pythonGH-142912)
710c183 
Modifies handling of `.gz` files in Android app payloads, and ensures that
when the Android testbed streams logs, stream flushes aren't treated as
newlines. This improves the output of test suites that use "one dot per test"
progress indicators.
(cherry picked from commit cb1dc91)

Co-authored-by: Malcolm Smith <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 3, 2026
@mhsmith @miss-islington
pythongh-144415: Android testbed fixes (pythonGH-142912)
df56fba 
Modifies handling of `.gz` files in Android app payloads, and ensures that
when the Android testbed streams logs, stream flushes aren't treated as
newlines. This improves the output of test suites that use "one dot per test"
progress indicators.
(cherry picked from commit cb1dc91)

Co-authored-by: Malcolm Smith <[email protected]>
@bedevere-app

bedevere-app Bot commented Feb 3, 2026

Copy link
Copy Markdown

GH-144416 is a backport of this pull request to the 3.14 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Feb 3, 2026
@bedevere-app

bedevere-app Bot commented Feb 3, 2026

Copy link
Copy Markdown

GH-144417 is a backport of this pull request to the 3.13 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Feb 3, 2026
freakboy3742 pushed a commit that referenced this pull request Feb 3, 2026
@miss-islington @mhsmith
[3.14] gh-144415: Android testbed fixes (GH-142912) (#144416)
b393713 
Modifies handling of `.gz` files in Android app payloads, and ensures that
when the Android testbed streams logs, stream flushes aren't treated as
newlines. This improves the output of test suites that use "one dot per test"
progress indicators.
(cherry picked from commit cb1dc91)

Co-authored-by: Malcolm Smith <[email protected]>
freakboy3742 pushed a commit that referenced this pull request Feb 3, 2026
@miss-islington @mhsmith
[3.13] gh-144415: Android testbed fixes (GH-142912) (#144417)
ebc047f 
Modifies handling of `.gz` files in Android app payloads, and ensures that
when the Android testbed streams logs, stream flushes aren't treated as
newlines. This improves the output of test suites that use "one dot per test"
progress indicators.
(cherry picked from commit cb1dc91)

Co-authored-by: Malcolm Smith <[email protected]>
Aniketsy pushed a commit to Aniketsy/cpython that referenced this pull request Feb 3, 2026
@mhsmith @Aniketsy
pythongh-144415: Android testbed fixes (python#142912)
a01abdb 
Modifies handling of `.gz` files in Android app payloads, and ensures that
when the Android testbed streams logs, stream flushes aren't treated as
newlines. This improves the output of test suites that use "one dot per test"
progress indicators.
thunder-coding pushed a commit to thunder-coding/cpython that referenced this pull request Feb 15, 2026
@mhsmith @thunder-coding
pythongh-144415: Android testbed fixes (python#142912)
810d1f0 
Modifies handling of `.gz` files in Android app payloads, and ensures that
when the Android testbed streams logs, stream flushes aren't treated as
newlines. This improves the output of test suites that use "one dot per test"
progress indicators.
@mhsmith mhsmith mentioned this pull request Apr 7, 2026
Android buildbot failures after updating to API level 35 #142387
Closed
@mhsmith mhsmith mentioned this pull request Jun 9, 2026
Strange new line behavior and missing color in the iOS testbed logs #150932
Open
This was referenced Sep 28, 2026
test_stream failures on older versions of Android chaquo/chaquopy#1477
Closed
Split Android testbed into a separate package #158018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@freakboy3742 freakboy3742 freakboy3742 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

OS-android

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Android testbed logs have excessive newlines

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@mhsmith @bedevere-bot @freakboy3742 @StanFromIreland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.