GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-104692: Include commoninstall as a prerequisite for bininstall by jefferyto · Pull Request #104693 · python/cpython

Conversation

@jefferyto

@jefferyto jefferyto commented May 20, 2023 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This ensures that commoninstall is completed before bininstall is started when parallel builds are used (make -j install), and so the python3 symlink is only installed after all standard library modules are installed.

  • Issue: Race condition during installation in parallel builds #104692

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added the awaiting review label May 20, 2023
@bedevere-bot bedevere-bot mentioned this pull request May 20, 2023
Race condition during installation in parallel builds #104692
Closed
@bedevere-bot

bedevere-bot commented May 20, 2023

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ghost

ghost commented May 20, 2023 •
edited by ghost
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.
CLA signed
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@jefferyto
jefferyto force-pushed the gh-104692 branch from 3a09491 to 8859645 Compare May 20, 2023 15:53
@jefferyto
pythongh-104692: Include commoninstall as a prerequisite for bininstall
869cd18 
This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.
@jefferyto
jefferyto force-pushed the gh-104692 branch from 8859645 to 869cd18 Compare May 20, 2023 15:58
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 20, 2023
@jefferyto
python3: Fix race condition when doing parallel builds
039f515 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: openwrt#19241

Signed-off-by: Jeffery To <[email protected]>
@jefferyto jefferyto mentioned this pull request May 20, 2023
python3: Update to 3.11.3, refresh/redo patches openwrt/packages#21030
Merged
@arhadthedev arhadthedev added build The build process and cross-build needs backport to 3.11 only security fixes labels May 20, 2023
@corona10
corona10 requested a review from zware May 21, 2023 09:23
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 22, 2023
@jefferyto
python3: Fix race condition when doing parallel builds
3da2dd3 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: openwrt#19241

Signed-off-by: Jeffery To <[email protected]>
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 23, 2023
@jefferyto
python3: Fix race condition when doing parallel builds
c244106 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: openwrt#19241

Signed-off-by: Jeffery To <[email protected]>
neheb pushed a commit to openwrt/packages that referenced this pull request May 24, 2023
@jefferyto @neheb
python3: Fix race condition when doing parallel builds
67e47f1 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: #19241

Signed-off-by: Jeffery To <[email protected]>
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 24, 2023
@jefferyto
python3: Fix race condition when doing parallel builds
1342fd8 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: openwrt#19241

Signed-off-by: Jeffery To <[email protected]>
(cherry picked from commit 67e47f1)
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 24, 2023
@jefferyto
python3: Fix race condition when doing parallel builds
9c5eb22 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: openwrt#19241

Signed-off-by: Jeffery To <[email protected]>
(cherry picked from commit 67e47f1)
Signed-off-by: Jeffery To <[email protected]>
graysky2 pushed a commit to graysky2/packages that referenced this pull request May 25, 2023
@jefferyto @graysky2
python3: Fix race condition when doing parallel builds
4c776eb 
When doing parallel builds, host Python can install the python3 symlink
before the Python standard library is installed completely.

When this occurs, it is possible for other packages to detect the
python3 symlink and try to use host Python before it is fully installed.

This adds a patch to make commoninstall (where the standard library is
installed) a prerequisite of bininstall (where the python3 symlink is
installed), so that commoninstall is fully completed before bininstall
begins.

Patch has been submitted upstream:
python/cpython#104693

Fixes: openwrt#19241

Signed-off-by: Jeffery To <[email protected]>
@zware zware added the needs backport to 3.12 only security fixes label May 30, 2023
zware
zware reviewed May 30, 2023
View reviewed changes
Comment thread Makefile.pre.in Resolved

Uh oh!

There was an error while loading. Please reload this page.
@jefferyto @zware
Update Makefile.pre.in
b41adc2 
Add an explanation for why commoninstall is made a prerequisite of bininstall.

Co-authored-by: Zachary Ware <[email protected]>
@ambv
ambv merged commit 990cb36 into python:main Jun 7, 2023
@miss-islington

miss-islington commented Jun 7, 2023

Copy link
Copy Markdown
Contributor

Thanks @jefferyto for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the awaiting review label Jun 7, 2023
@miss-islington

miss-islington commented Jun 7, 2023

Copy link
Copy Markdown
Contributor

Sorry @jefferyto and @ambv, I had trouble checking out the 3.12 backport branch.
Please retry by removing and re-adding the "needs backport to 3.12" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 990cb3676c2edb7e5787372d6cbe360a73367f4c 3.12
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington miss-islington assigned ambv Jun 7, 2023
@miss-islington

miss-islington commented Jun 7, 2023

Copy link
Copy Markdown
Contributor

Sorry, @jefferyto and @ambv, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 990cb3676c2edb7e5787372d6cbe360a73367f4c 3.11
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
jefferyto added a commit to jefferyto/cpython that referenced this pull request Jun 7, 2023
@jefferyto
[3.12] pythongh-104692: Include commoninstall as a prerequisite for b…
c4c7ffe 
…ininstall (pythonGH-104693)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

Co-authored-by: Zachary Ware <[email protected]>
(cherry picked from commit 990cb36)

Co-authored-by: Jeffery To <[email protected]>
@jefferyto jefferyto mentioned this pull request Jun 7, 2023
[3.12] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) #105428
Merged
@bedevere-bot

bedevere-bot commented Jun 7, 2023

Copy link
Copy Markdown

GH-105428 is a backport of this pull request to the 3.12 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jun 7, 2023
@jefferyto jefferyto mentioned this pull request Jun 7, 2023
[3.11] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) #105429
Merged
@bedevere-bot

bedevere-bot commented Jun 7, 2023

Copy link
Copy Markdown

GH-105429 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Jun 7, 2023
@bedevere-bot

bedevere-bot commented Jun 7, 2023

Copy link
Copy Markdown

GH-105428 is a backport of this pull request to the 3.12 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
jefferyto added a commit to jefferyto/cpython that referenced this pull request Jun 7, 2023
@jefferyto
[3.11] pythongh-104692: Include commoninstall as a prerequisite for b…
94d5d97 
…ininstall (pythonGH-104693)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

Co-authored-by: Zachary Ware <[email protected]>.
(cherry picked from commit 990cb36)

Co-authored-by: Jeffery To <[email protected]>
@bedevere-bot

bedevere-bot commented Jun 7, 2023

Copy link
Copy Markdown

GH-105429 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@jefferyto
jefferyto deleted the gh-104692 branch June 7, 2023 09:03
@jefferyto

jefferyto commented Jul 4, 2023

Copy link
Copy Markdown
Contributor Author

Is there anything else I should do to have the backports merged?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
ambv pushed a commit that referenced this pull request Jul 5, 2023
@jefferyto
[3.12] gh-104692: Include commoninstall as a prerequisite for bininst…
2edec6a 
…all (GH-104693) (#105428)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

(cherry picked from commit 990cb36)
ambv pushed a commit that referenced this pull request Jul 5, 2023
@jefferyto @erlend-aasland
[3.11] gh-104692: Include commoninstall as a prerequisite for bininst…
6c6a2a9 
…all (GH-104693) (#105429)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

.
(cherry picked from commit 990cb36)

Co-authored-by: Erlend E. Aasland <[email protected]>
@ambv

ambv commented Jul 5, 2023

Copy link
Copy Markdown
Contributor

I merged them.
jefferyto and erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
lazka added a commit to lazka/MSYS2-packages that referenced this pull request Aug 28, 2023
@lazka
python: Update to 3.11.5
2060d32 
960-fix-parallel-make.patch: fixed upstream python/cpython#104693
@lazka lazka mentioned this pull request Aug 28, 2023
python: Update to 3.11.5 msys2/MSYS2-packages#3988
Merged
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request Sep 5, 2023
@jefferyto
python3: Update to 3.11.5, refresh patches
e7b8a61 
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).

This also:

* Remove 027-install-python3-symlink-after-stdlib.patch

  This was merged upstream in
  python/cpython#104693.

* Remove fix for unnecessary linking with libbsd
  (60bf01c)

  This was fixed upstream in
  python/cpython#105236.

Signed-off-by: Jeffery To <[email protected]>
@jefferyto jefferyto mentioned this pull request Sep 5, 2023
python3: Update to 3.11.5, refresh patches openwrt/packages#22049
Merged
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request Sep 5, 2023
@jefferyto
python3: Update to 3.11.5, refresh patches
8b34dfc 
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).

This also:

* Remove 027-install-python3-symlink-after-stdlib.patch

  This was merged upstream in
  python/cpython#104693.

* Remove fix for unnecessary linking with libbsd
  (60bf01c)

  This was fixed upstream in
  python/cpython#105236.

Signed-off-by: Jeffery To <[email protected]>
1715173329 pushed a commit to openwrt/packages that referenced this pull request Sep 8, 2023
@jefferyto @1715173329
python3: Update to 3.11.5, refresh patches
e42c845 
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).

This also:

* Remove 027-install-python3-symlink-after-stdlib.patch

  This was merged upstream in
  python/cpython#104693.

* Remove fix for unnecessary linking with libbsd
  (60bf01c)

  This was fixed upstream in
  python/cpython#105236.

Signed-off-by: Jeffery To <[email protected]>
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request Sep 12, 2023
@jefferyto
python3: Update to 3.11.5, refresh patches
102b3a7 
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).

This also:

* Remove 027-install-python3-symlink-after-stdlib.patch

  This was merged upstream in
  python/cpython#104693.

* Remove fix for unnecessary linking with libbsd
  (60bf01c)

  This was fixed upstream in
  python/cpython#105236.

Signed-off-by: Jeffery To <[email protected]>
(cherry picked from commit e42c845)
@jefferyto jefferyto mentioned this pull request Sep 12, 2023
[openwrt-23.05] python3: Update to 3.11.5, refresh patches openwrt/packages#22093
Merged
BKPepe pushed a commit to openwrt/packages that referenced this pull request Sep 14, 2023
@jefferyto @BKPepe
python3: Update to 3.11.5, refresh patches
e05cf55 
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).

This also:

* Remove 027-install-python3-symlink-after-stdlib.patch

  This was merged upstream in
  python/cpython#104693.

* Remove fix for unnecessary linking with libbsd
  (60bf01c)

  This was fixed upstream in
  python/cpython#105236.

Signed-off-by: Jeffery To <[email protected]>
(cherry picked from commit e42c845)
lu-zero pushed a commit to domo-iot/packages that referenced this pull request Oct 23, 2023
@jefferyto @lu-zero
python3: Update to 3.11.5, refresh patches
aea6177 
Includes fix for CVE-2023-40217 (Bypass TLS handshake on closed
sockets).

This also:

* Remove 027-install-python3-symlink-after-stdlib.patch

  This was merged upstream in
  python/cpython#104693.

* Remove fix for unnecessary linking with libbsd
  (60bf01c)

  This was fixed upstream in
  python/cpython#105236.

Signed-off-by: Jeffery To <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zware zware zware left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@ambv ambv

Labels

build The build process and cross-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@jefferyto @bedevere-bot @miss-islington @ambv @zware @arhadthedev
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.