GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-36511: Fix Windows arm32 buildbot pythoninfo, scp, and ssh by paulmon · Pull Request #13454 · python/cpython

Conversation

@paulmon

@paulmon paulmon commented May 20, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

When running the buildbot for Windows ARM32 test.pythoninfo must be run remotely.
Also seperating deploy step from both pythoninfo and test.bat

Requires buildmaster config change to work:
python/buildmaster-config#91

@zooba @zware

https://bugs.python.org/issue36511

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Paul Monson
bpo-36511: Fix Windows arm32 buildbot pythoninfo, scp, and ssh
1d33fe6
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label May 20, 2019
@bedevere-bot bedevere-bot added the awaiting review label May 20, 2019
@paulmon paulmon mentioned this pull request May 20, 2019
bpo-36511: Fix Windows arm32 buildbot, pythoninfo, scp, and ssh python/buildmaster-config#91
Merged
Paul Monson added 2 commits May 21, 2019 12:07
Paul Monson
add +d to match test.bat
3aadf52
Paul Monson
100% test pass when manually running test.bat
7f17d29
@paulmon

paulmon commented May 22, 2019

Copy link
Copy Markdown
Contributor Author

The debug part of the test pass should pass locally with these fixes, except for test_datetime.
See this PR for test_datetime: #13073
I am re-running test.bat now with these fixes, which takes more than an hour.
The last pass took 4 hours and 23 minutes on my secondary ARM test device, but on the buildbot ARM test device it's been taking about an hour and a half.
I still need to try the WindowsArm32ReleaseBuild steps, but expect the same results.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Paul Monson
fix file destination
9aaa2c4
zooba
zooba reviewed May 22, 2019
View reviewed changes

@zooba zooba left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer a tighter skip on the tests, and deferring to @zware on the buildbot scripts (as with the build config change).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_ssl.py Outdated
ctx.load_verify_locations(cadata=b"broken")


@unittest.skipIf(Py_DEBUG, "Crashes on debug python builds")

@zooba zooba May 22, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is certainly not a general issue - can we scope it down to the platform(s) where it crashes?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@paulmon paulmon May 22, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will scope it down.

I think it's limited to UCRT. The problem is that SSL has FILE* from the ucrtbase.dll table and python has a FILE* from the ucrtbased.dll table.

I think we could limit it to win32+arm or platform.win32_is_iot( ). Unless there's a way to check if python is using UCRT? I couldn't find one.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@zooba zooba May 23, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's because of mixing FILE* across debug/release builds? In that case we scope it down to Windows and enhance the skip message to say "Avoid mixing debug/release CRT on Windows"

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@paulmon paulmon May 23, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Paul Monson added 6 commits May 23, 2019 11:22
Paul Monson
only skip ssl tests on Windows IoT
3f38d18
Paul Monson
Avoid mixing debug/release CRT on Windows
4e147de
Paul Monson
test for win32
9819647
Paul Monson
propagate exit code
365701c
Paul Monson
fix 2 more exit 0 calls
69c40d7
Paul Monson
fixes for test_regrtest
086a1b9
@paulmon

paulmon commented May 24, 2019

Copy link
Copy Markdown
Contributor Author

Enabling is_python_build( ) to return true on the test target uncovered more failing tests.
There is still one failure in test_pyexpat that I will be investigating next.

======================================================================
FAIL: test_exception (test.test_pyexpat.HandlerExceptionTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\py\lib\test\test_pyexpat.py", line 453, in test_exception
    parser.Parse(b"<a><b><c/></b></a>", 1)
RuntimeError: a

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\py\lib\test\test_pyexpat.py", line 469, in test_exception
    self.assertIn('call_with_frame("StartElement"', entries[1][3])
AssertionError: 'call_with_frame("StartElement"' not found in ''

----------------------------------------------------------------------
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Paul Monson added 5 commits May 24, 2019 09:35
Paul Monson
merge upstream/master
042e0e9
Paul Monson
add tools/parser for test_tools
e230795
Paul Monson
copy entire Tools directory
1a1f12d
Paul Monson
skip check on Windows ARM32
29601d8
Paul Monson
restore 64-bit check in test_regrtest
786194d
@paulmon

paulmon commented May 24, 2019

Copy link
Copy Markdown
Contributor Author

I simplified the deployment script to copy entire directories instead of picking files, except for PCBuild and Modules which have many files that aren't needed for the test run. This should make the script more resilient to test changes.

I think the Windows ARM32 test pass is green now when I run the scripts manually, if #13073 is included.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

paulmon
paulmon commented May 24, 2019
View reviewed changes
Comment thread Lib/test/test_pyexpat.py
self.check_traceback_entry(entries[2],
"test_pyexpat.py", "StartElementHandler")
if sysconfig.is_python_build():
if sysconfig.is_python_build() and not (sys.platform == 'win32' and platform.machine() == 'ARM'):

@paulmon paulmon May 24, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather fix this than skip it, but I haven't figured out where to start looking yet.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@zooba zooba added skip news needs backport to 3.8 labels Jun 11, 2019
Paul Monson
merge upstream/master
cd2d5c5
@paulmon
paulmon requested a review from tiran as a code owner June 11, 2019 23:08
zooba
zooba reviewed Jun 12, 2019
View reviewed changes
Comment thread Lib/test/test_ssl.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_math.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@paulmon

paulmon commented Jun 13, 2019

Copy link
Copy Markdown
Contributor Author

@tiran can you please review?
Some of these changes are required to enable the Windows arm32 buildbot to start working.
Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba

zooba commented Jun 19, 2019

Copy link
Copy Markdown
Member

@paulmon I'm happy to merge without Christian's review once the updates I suggested in my last one are done.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

paulmon and others added 2 commits June 19, 2019 11:16
@paulmon @zooba
Update Lib/test/test_math.py
f5c917a
Co-Authored-By: Steve Dower <[email protected]>
@paulmon @zooba
Update Lib/test/test_ssl.py
e2fb14b
Co-Authored-By: Steve Dower <[email protected]>
@paulmon

paulmon commented Jun 19, 2019

Copy link
Copy Markdown
Contributor Author

@zooba I merged your suggested changes. I think it's ready

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba

zooba commented Jun 19, 2019

Copy link
Copy Markdown
Member

@paulmon The Py_DEBUG_WIN32 change needs the references to that variable updated as well. I didn't go through and suggest a change on each one.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@paulmon

paulmon commented Jun 19, 2019

Copy link
Copy Markdown
Contributor Author

@paulmon The Py_DEBUG_WIN32 change needs the references to that variable updated as well. I didn't go through and suggest a change on each one.

I'm not sure I'm following you. Are suggesting adding a check for arm/arm64 to Py_DEBUG_WIN32? That would be incorrect for Windows IoT Core x86/x64 which also use UCRT for everything.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Paul Monson
Merge remote-tracking branch 'upstream/master' into arm32_pythoninfo
ce3dfbc
@zooba

zooba commented Jun 19, 2019

Copy link
Copy Markdown
Member

@paulmon The Py_DEBUG_WIN32 change needs the references to that variable updated as well. I didn't go through and suggest a change on each one.

I'm not sure I'm following you. Are suggesting adding a check for arm/arm64 to Py_DEBUG_WIN32? That would be incorrect for Windows IoT Core x86/x64 which also use UCRT for everything.

No, I'm saying elsewhere you are referring to Py_DEBUG_Win32 which is now undefined. They need to be changed to Py_DEBUG_WIN32 as well

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Paul Monson added 2 commits June 19, 2019 11:45
Paul Monson
fix mismatched paren
5c355ab
Paul Monson
fix Py_DEBUG_WIN32 spelling
f1ce1bf
@paulmon

paulmon commented Jun 19, 2019

Copy link
Copy Markdown
Contributor Author

No, I'm saying elsewhere you are referring to Py_DEBUG_Win32 which is now undefined. They need to be changed to Py_DEBUG_WIN32 as well

Thanks. I noticed the whitespace change but somehow didn't see the uppercase change.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@paulmon

paulmon commented Jun 19, 2019

Copy link
Copy Markdown
Contributor Author

@zooba Thanks for the help
I fixed test_ssl and test_math so they pass on my dev machine.
All of the PR checks pass now.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba
zooba merged commit f355069 into python:master Jun 19, 2019
@miss-islington

miss-islington commented Jun 19, 2019

Copy link
Copy Markdown
Contributor

Thanks @paulmon for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting review label Jun 19, 2019
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 19, 2019
@paulmon @miss-islington
bpo-36511: Add buildbot scripts and fix tests for Windows ARM32 build…
3a9f4ee
…bot (pythonGH-13454)

(cherry picked from commit f355069)

Co-authored-by: Paul Monson <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 3.8 label Jun 19, 2019
@bedevere-bot

bedevere-bot commented Jun 19, 2019

Copy link
Copy Markdown

GH-14244 is a backport of this pull request to the 3.8 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba

zooba commented Jun 19, 2019

Copy link
Copy Markdown
Member

@paulmon Seems like it still fails: https://buildbot.python.org/all/#/builders/197/builds/338

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@paulmon

paulmon commented Jun 19, 2019

Copy link
Copy Markdown
Contributor Author

@paulmon Seems like it still fails: https://buildbot.python.org/all/#/builders/197/builds/338

The raspberry pi was in a bad state. I have restarted it

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@paulmon

paulmon commented Jun 19, 2019

Copy link
Copy Markdown
Contributor Author

Also, ssh.exe and scp.exe somehow don't exist when running under the buildbot, which is breaking the script.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba

zooba commented Jun 19, 2019

Copy link
Copy Markdown
Member

@paulmon Different PATH setting? Or perhaps it's running with different file redirection settings (as a 32-bit app, maybe)?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@paulmon

paulmon commented Jun 19, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

@paulmon Different PATH setting? Or perhaps it's running with different file redirection settings (as a 32-bit app, maybe)?

The PATH is echoed at the top of the log and includes the PATH to ssh.
I hadn't thought of WoW as a possible cause. I installed from the link on python.org so I got the 32-bit version of python. I'll update to the 64-bit python and see if that helps. There isn't a SysWOW64 version of ssh.exe, so that could be it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@paulmon paulmon mentioned this pull request Jun 19, 2019
bpo-36511 add arm32 buildbot diagnostic messages #14251
Merged
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Jun 21, 2019
@CuriousLearner
Merge branch 'master' into fix-issue24255
7bf6445
* master: (599 commits)
  Docs: Improved phrasing (pythonGH-14069)
  Remove redundant if check from optional argument function in argparse. (pythonGH-8766)
  bpo-37289: Add a test for if with ifexpr in the peephole optimiser to detect regressions (pythonGH-14127)
  Update What's New in Python 3.9 (pythonGH-14253)
  bpo-36511: Improve ARM32 buildbot scripts (pythonGH-14251)
  bpo-37151: remove _PyCFunction_FastCallDict (pythonGH-14269)
  Fix typo, 'widger' -> 'widget', in idlelib/tree.py (pythonGH-14263)
  Fix bpo number in News file. (pythonGH-14260)
  bpo-37342: Fix the incorrect nb_index's type in typeobj documentation (pythonGH-14241)
  Update What's New in Python 3.8 (pythonGH-14239)
  bpo-36710: Use tstate in pylifecycle.c (pythonGH-14249)
  Add missing single quote in io.TextIOWrapper.reconfigure documentation (pythonGH-14246)
  bpo-36511: Add buildbot scripts and fix tests for Windows ARM32 buildbot (pythonGH-13454)
  bpo-37333: Ensure IncludeTkinter has a value (pythonGH-14240)
  bpo-37331: Clarify format of socket handler messages in the documentation. (pythonGH-14234)
  bpo-37258: Not a bug, but added a unit test and updated documentation. (pythonGH-14229)
  bpo-36710: Remove PyImport_Cleanup() function (pythonGH-14221)
  Fix name of '\0'. (pythonGH-14222)
  bpo-36710: Add tstate parameter in import.c (pythonGH-14218)
  Document typing.ForwardRef (pythonGH-14216)
  ...
@paulmon
paulmon deleted the arm32_pythoninfo branch July 17, 2019 20:30
lisroach pushed a commit to lisroach/cpython that referenced this pull request Sep 10, 2019
@paulmon @lisroach
bpo-36511: Add buildbot scripts and fix tests for Windows ARM32 build…
cd062ef
…bot (pythonGH-13454)
DinoV pushed a commit to DinoV/cpython that referenced this pull request Jan 14, 2020
@paulmon @DinoV
bpo-36511: Add buildbot scripts and fix tests for Windows ARM32 build…
a739c78
…bot (pythonGH-13454)
@paulmon paulmon mannequin mentioned this pull request May 20, 2022
Add Windows ARM32 buildbot #80692
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zooba zooba zooba left review comments
@tiran tiran Awaiting requested review from tiran

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@paulmon @zooba @miss-islington @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.