Description

ericsnowcurrently

opened on Nov 2, 2021

Issue body actions

BPO	45695
Nosy	@gvanrossum, @gpshead, @tiran, @ericsnowcurrently, @zooba, @erlend-aasland
PRs	bpo-45695: Test out-of-tree builds on GHA (GH-29904) #29904

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = None
created_at = <Date 2021-11-02.23:29:45.063>
labels = ['build', '3.10', '3.11']
title = 'Out-of-tree builds are not tested.'
updated_at = <Date 2021-12-04.10:08:03.694>
user = 'https://github.com/ericsnowcurrently'

bugs.python.org fields:

activity = <Date 2021-12-04.10:08:03.694>
actor = 'christian.heimes'
assignee = 'gregory.p.smith'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2021-11-02.23:29:45.063>
creator = 'eric.snow'
dependencies = []
files = []
hgrepos = []
issue_num = 45695
keywords = ['patch']
message_count = 7.0
messages = ['405568', '405572', '405573', '407591', '407594', '407595', '407652']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'gregory.p.smith', 'christian.heimes', 'eric.snow', 'steve.dower', 'erlendaasland']
pr_nums = ['29904']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue45695'
versions = ['Python 3.10', 'Python 3.11']

Activity

1. ericsnowcurrently commented on Nov 2, 2021

   

   ericsnowcurrently

   on Nov 2, 2021

   MemberAuthor

   More actions

   Currently we don't test builds done outside the source tree, neither on GitHub nor the buildbots. [1] As a result, such builds get broken occasionally. I've certainly broken then a couple times, inadvertently. It would be helpful if we tested out-of-tree builds on GitHub, or at least on a stable buildbot.

   FTR, to do an out-of-tree build locally and test it:

   mkdir ../build
   cd ../build
   ../cpython/configure
   make
   ./python -m test

   [1] Actually I recently added test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script which happens to do an out-of-tree build incidentally. However, we should be testing out-of-tree builds explicitly.
2. ericsnowcurrently

   added

   3.10 (EOL)end of lifeend of life

   3.11only security fixesonly security fixes

   buildThe build process and cross-buildThe build process and cross-build

   on Nov 2, 2021
3. zooba commented on Nov 3, 2021

   

   zooba

   on Nov 3, 2021

   Member

   More actions

   FTR, I used to run all Windows builds out of tree on Azure Pipelines, but too many tests broke so I reverted it:

   cpython/.azure-pipelines/windows-steps.yml

   Lines 7 to 10 in 456e27a

   	# Relocate build outputs outside of source directory to make cleaning faster
   	Write-Host '##vso[task.setvariable variable=Py_IntDir]$(Build.BinariesDirectory)\obj'
   	# UNDONE: Do not build to a different directory because of broken tests
   	Write-Host '##vso[task.setvariable variable=Py_OutDir]$(Build.SourcesDirectory)\PCbuild'

   Setting $env:Py_OutDir before building is the easiest way to change it. You can also pass "/p:Py_OutDir=<path>" to build.bat, but you'll run into quoting issues almost immediately.
4. gpshead commented on Nov 3, 2021

   

   gpshead

   on Nov 3, 2021

   Member

   More actions

   I always do out of tree builds _(I find people who build anything in tree to be very weird - it makes a mess)_.

   I intend to look into making sure a Linux buildbot runs this way but don't let that stop anyone else from doing so.

   Reacted by Eric Snow
5. gpshead

   assigned

   gpshead

   on Nov 3, 2021
6. tiran commented on Dec 3, 2021

   

   tiran

   on Dec 3, 2021

   Member

   More actions

   Good news: I got a test case for OOT build with read-only sources

   Bad news: 15 test cases are failing and testing takes much longer because pyc files are missing.

   15 re-run tests:
   test__xxsubinterpreters test_ast test_bdb test_capi test_doctest
   test_import test_importlib test_interpreters test_pickle
   test_pydoc test_runpy test_support test_threading test_trace
   test_unicodedata

   We have to live with read-only OOT builds and writable tests for now.
7. gpshead commented on Dec 3, 2021

   

   gpshead

   on Dec 3, 2021

   Member

   More actions

   Nice trick with that RO bind mount. tests... yes we do have some writable source tree messes in there to be dealt with.
8. erlend-aasland commented on Dec 3, 2021

   

   erlend-aasland

   on Dec 3, 2021

   Contributor

   More actions

   FWIW, it is also impossible to build the docs OOT without cluttering the source tree.

       $ cd ../build
       $ make -C ../cpython.git/Doc venv
       $ make -C ../cpython.git/Doc html  # <= modifies Doc/build in the source tree
9. tiran commented on Dec 4, 2021

   

   tiran

   on Dec 4, 2021

   Member

   More actions

   New changeset cee07b1 by Christian Heimes in branch 'main':
   bpo-45695: Test out-of-tree builds on GHA (GH-29904)
   cee07b1
10. ezio-melotti

    transferred this issue fromon Apr 10, 2022
11. kumaraditya303 commented on Apr 28, 2022

    

    kumaraditya303

    on Apr 28, 2022

    Contributor

    More actions

    Test was added in #29904 so closing.
12. kumaraditya303

    closed this as completedon Apr 28, 2022
13. neonene commented on May 31, 2022

    

    neonene

    on May 31, 2022

    Contributor

    More actions

    gh-78436 mentions that test_embed is actually skipped on OOT.
14. gvanrossum commented on May 31, 2022

    

    gvanrossum

    on May 31, 2022

    Member

    More actions

    Reopening until there is agreement on whether this works as required.

    Reacted by Erlend E. Aasland
15. gvanrossum

    reopened this on May 31, 2022
16. kumaraditya303

    added

    3.12only security fixesonly security fixes

    on Jun 10, 2022
17. zooba

    closed this as completedin #93465on Jun 17, 2022
18. zooba

    added a commit that references this issue on Jun 17, 2022

    GH-89858: Fix test_embed for out-of-tree builds (GH-93465)

    96464e5
19. miss-islington

    added 2 commits that reference this issue on Jun 17, 2022

    pythonGH-89858: Fix test_embed for out-of-tree builds (pythonGH-93465)

    2def8da

    GH-89858: Fix test_embed for out-of-tree builds (GH-93465)

    79d2269
20. serhiy-storchaka

    added a commit that references this issue on Jun 26, 2022

    Revert 30935 defer bpo45162 to 312 (#94285)

    1ffde3e

Issue actions