GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — bpo-40280: Add Tools/wasm with helpers for cross building (GH-29984) by tiran · Pull Request #29984 · python/cpython

Conversation

tiran commented Dec 8, 2021 •

edited by bedevere-bot

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

https://bugs.python.org/issue40280

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

the-knights-who-say-ni added the CLA signed label

Dec 8, 2021

bedevere-bot added the awaiting core review label

Dec 8, 2021

tiran force-pushed the bpo-40280-wasm-tools branch from 3305617 to 877bb04 Compare

December 13, 2021 20:03

tiran marked this pull request as ready for review

December 13, 2021 20:10

tiran requested a review from brettcannon

December 13, 2021 20:10

tiran force-pushed the bpo-40280-wasm-tools branch from 877bb04 to f65a480 Compare

December 13, 2021 20:10

tiran commented Dec 13, 2021

Copy link

Copy Markdown

Member Author

$ CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \
  emconfigure ../../configure -C \
    --host=wasm32-unknown-emscripten \
    --build=$(../../config.guess) \
    --with-build-python=$(pwd)/../build/python
$ emmake make -j$(nproc) python.html
$ ls -s --si python.*
3.0M python.data
107k python.html
361k python.js
6.6M python.wasm

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

emmatyping reviewed

Dec 13, 2021

View reviewed changes

emmatyping left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will definitely make experimenting with Emscripten based CPython a lot easier!

I have a few suggestions/comments but overall this looks good.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/README.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

emmatyping commented Dec 13, 2021

Copy link

Copy Markdown

Member

The binary size looks really great! Though with the number of things disabled, it will probably grow over time...

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tiran force-pushed the bpo-40280-wasm-tools branch from a3797c0 to 6a103d9 Compare

December 14, 2021 08:56

tiran commented Dec 14, 2021

Copy link

Copy Markdown

Member Author

With --disabled-test-modules, without decimal unittest xml and without some C extension

# Modules/Setup.local
*disabled*
_decimal
pyexpat
_elementtree
_lsprof
audioop

I'm able to shrink the core to about 6 MB

# du --si -s -c python.*
2.7M    python.data
107k    python.html
275k    python.js
3.3M    python.wasm
6.4M    total

The python.wasm file can be compressed well. If we assume that all browsers support transparent compression, then we are down to about 4 MB with room for more removals.

# du --si -s -c python.*
2.7M    python.data
107k    python.html
275k    python.js
1.1M    python.wasm.gz
4.2M    total

The largest 25 files are:

# unzip -l usr/local/lib/python311.zip  | sort -n | tail -n 26
    57953  2021-12-13 19:58   threading.pyc
    59930  2021-12-13 19:59   optparse.pyc
    61650  2021-12-13 19:58   subprocess.pyc
    61916  2021-12-13 19:56   collections/__init__.pyc
    63723  2021-12-13 19:56   enum.pyc
    63980  2021-12-13 19:59   pickle.pyc
    66135  2021-12-13 19:59   pdb.pyc
    67936  2021-12-13 19:58   locale.pyc
    68530  2021-12-13 19:58   asyncio/base_events.pyc
    70300  2021-12-14 09:56   _sysconfigdata__emscripten_.pyc
    71134  2021-12-13 19:59   difflib.pyc
    77197  2021-12-13 19:58   datetime.pyc
    77293  2021-12-13 19:59   ipaddress.pyc
    78354  2021-12-13 19:56   ast.pyc
    79675  2021-12-13 19:59   pickletools.pyc
    82993  2021-12-13 19:58   logging/__init__.pyc
    85730  2021-12-13 19:56   zipfile.pyc
    86382  2021-12-13 19:59   mailbox.pyc
    86537  2021-12-13 19:56   argparse.pyc
    88029  2021-12-13 19:59   tarfile.pyc
    92877  2021-12-13 19:59   doctest.pyc
   108300  2021-12-13 19:56   inspect.pyc
   109640  2021-12-13 19:58   typing.pyc
   118759  2021-12-13 19:59   pydoc.pyc
   253750  2021-12-13 19:59   html/entities.pyc
  5711264                     347 files

@brettcannon now is a good time to discuss what modules belong in a minimum stdlib.

jezdez, emmatyping, brettcannon, and pannous reacted with hooray emoji

jezdez, isidentical, emmatyping, and pradyunsg reacted with rocket emoji

jezdez reacted with eyes emoji

All reactions

🎉 4 reactions
🚀 4 reactions
👀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

emmatyping commented Dec 14, 2021

Copy link

Copy Markdown

Member

Wow that's a great reduction in size! I played with thin lto and for some reason it greatly increases the binary size (to 21MB!). I guess inlining makes the build a lot bigger. One other idea I had which gives small gains in binary size is compiling with -Os, which seems to not make much of a difference vs -Oz.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tiran commented Dec 14, 2021

Copy link

Copy Markdown

Member Author

Actually it is less. I think I introduced a regression in GH-30001. makesetup is dropping too many modules from the final binary.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tiran force-pushed the bpo-40280-wasm-tools branch 2 times, most recently from 59d8c8e to 218aef5 Compare

December 14, 2021 14:47

tiran and others added 3 commits

December 17, 2021 17:05


          bpo-40280: Add Tools/wasm with helpers for cross building

1db84bd


          Apply suggestions from code review

30db875

Co-authored-by: Ethan Smith <[email protected]>


          Suggest emrun, explain writepy()

f933b91

tiran force-pushed the bpo-40280-wasm-tools branch from 218aef5 to f933b91 Compare

December 17, 2021 16:05

emmatyping approved these changes

Dec 17, 2021

View reviewed changes

brettcannon approved these changes

Dec 17, 2021

View reviewed changes

Comment thread

Tools/wasm/README.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/README.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/README.md

		## wasm32-emscripten build

		Cross compiling to wasm32-emscripten platform needs [Emscripten](https://emscripten.org/)
		tool chain and a build Python interpreter.

brettcannon Dec 17, 2021

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

	tool chain and a build Python interpreter.
	tool chain and a built Python interpreter.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

emmatyping Dec 18, 2021

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build here is I believe a reference to the autotools/configure --build argument. (Same for line 11 below).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

tiran Dec 18, 2021

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. In lack of a better term I introduced the term "build Python"/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

brettcannon Dec 20, 2021

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My brain just keeps reading "build" as a verb instead of an adjective. 😄

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

Tools/wasm/README.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/README.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

Tools/wasm/wasm_assets.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot added awaiting merge and removed awaiting core review labels

Dec 17, 2021

tiran and others added 2 commits

December 18, 2021 11:45


          Brett's code suggestions: upper cases and full stops

6c46ab4

Co-authored-by: Brett Cannon <[email protected]>


          More grammar fixes from Brett

d0782ca

Co-authored-by: Brett Cannon <[email protected]>

tiran changed the title ~~bpo-40280: Add Tools/wasm with helpers for cross building~~ bpo-40280: Add Tools/wasm with helpers for cross building (GH-29984)

Dec 18, 2021

tiran merged commit 0339434 into python:main

Dec 18, 2021

tiran deleted the bpo-40280-wasm-tools branch

December 18, 2021 14:54

bedevere-bot removed the awaiting merge label

Dec 18, 2021

tiran added a commit to tiran/python-wasm that referenced this pull request

Dec 18, 2021


          Use make python.html and config.site

2688a17

CPython upstream now has a config.site for WASM `and make python.html`.

See: python/cpython#29984
Signed-off-by: Christian Heimes <[email protected]>

tiran mentioned this pull request

Dec 18, 2021

Use make python.html and config.site emmatyping/python-wasm#38

Merged

SimonBiggs mannequin mentioned this pull request

Jun 7, 2022

Consider supporting emscripten/webassembly as a build target #84461

Closed

rth mentioned this pull request

Nov 12, 2022

Vendor standard libraries in a zip file pyodide/pyodide#3166

Closed

1 task done

Reviewers

brettcannon

brettcannon approved these changes

emmatyping

emmatyping approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

tiran commented Dec 8, 2021 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tiran commented Dec 13, 2021

Uh oh!

emmatyping left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

emmatyping commented Dec 13, 2021

Uh oh!

tiran commented Dec 14, 2021

Uh oh!

emmatyping commented Dec 14, 2021

Uh oh!

tiran commented Dec 14, 2021

Uh oh!

Uh oh!

Uh oh!

brettcannon Dec 17, 2021

Choose a reason for hiding this comment

Uh oh!

emmatyping Dec 18, 2021

Choose a reason for hiding this comment

Uh oh!

tiran Dec 18, 2021

Choose a reason for hiding this comment

Uh oh!

brettcannon Dec 20, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tiran commented Dec 8, 2021 •

edited by bedevere-bot

Loading

Uh oh!

There was an error while loading. Please reload this page.