GeezerVison is showing github.com — Improve devcontainer build speed · Issue #8 · python/cpython-devcontainers
Improve devcontainer build speed #8ActivityAA-Turner commented on Sep 26, 2024 The below is extracted from the logs, the main dnf step takes 2,038 seconds (34 minutes).
Notably, no output is produced for ~12 minutes (730 seconds). "Fedora 40 - aarch64 - Updates" then takes another 6 minutes (370 seconds). In the second builddep command, another six minutes (350 seconds) elapse from "enabling fedora-source repository" to the first "Package [...] is already installed.".
cc @hroncok for any advice (as author of python/cpython#103283)
dnf install ...:
[linux/arm64 2/4] RUN dnf -y --nodocs --setopt=install_weak_deps=False install /usr/bin/{blurb,clang,curl,git,ln,tar,xz} 'dnf-command(builddep)' && dnf -y --nodocs --setopt=install_weak_deps=False builddep python3 && dnf -y clean all
729.3 Fedora 40 openh264 (From Cisco) - aarch64 476 B/s | 2.1 kB 00:04
785.2 Fedora 40 - aarch64 - Updates 1.1 MB/s | 38 MB 00:33
1156.0 Last metadata expiration check: 0:00:23 ago on Wed Sep 25 18:39:54 2024.
1241.3 Package [...] is already installed. [x4]
1250.7 Dependencies resolved.
1251.6 ================================================================================
1251.6 Package Arch Version Repo Size
1251.6 ================================================================================
1251.6 Installing:
...
1251.6 Transaction Summary
1251.6 ================================================================================
1251.6 Install 47 Packages
1251.6
1251.7 Total download size: 140 M
1251.7 Installed size: 562 M
1251.7 Downloading Packages:
...
1267.9 --------------------------------------------------------------------------------
1267.9 Total 8.7 MB/s | 140 MB 00:16
1293.7 Running transaction check
1295.7 Transaction check succeeded.
1295.7 Running transaction test
1305.3 Transaction test succeeded.
1305.3 Running transaction
1313.6 Preparing : 1/1
1315.1 Installing : systemd-libs-255.12-1.fc40.aarch64 1/47
...
1344.5 Running scriptlet: dnf-plugins-core-4.9.0-1.fc40.noarch 47/47
...
1348.7 Complete!
dnf builddep python3:
1358.6 enabling fedora-source repository
1358.6 enabling fedora-cisco-openh264-source repository
1358.6 enabling updates-source repository
1370.6 Fedora 40 - Source 1.2 MB/s | 6.6 MB 00:05
1433.9 Fedora 40 openh264 (From Cisco) - aarch64 - Sou 30 B/s | 134 B 00:04
1452.2 Fedora 40 - Updates Source 359 kB/s | 1.7 MB 00:04
1544.5 Last metadata expiration check: 0:00:08 ago on Wed Sep 25 18:51:10 2024.
1708.0 Package [...] is already installed. [x12]
1722.1 Dependencies resolved.
1724.3 ================================================================================
1724.3 Package Arch Version Repo Size
1724.3 ================================================================================
1724.3 Installing:
...
1724.3 Transaction Summary
1724.3 ================================================================================
1724.3 Install 233 Packages
1724.3 Upgrade 11 Packages
1724.3
1724.7 Total download size: 123 M
1724.8 Downloading Packages:
...
1746.9 --------------------------------------------------------------------------------
1746.9 Total 5.6 MB/s | 123 MB 00:21
1861.1 Running transaction check
1867.8 Transaction check succeeded.
1867.8 Running transaction test
1893.0 Transaction test succeeded.
1893.0 Running transaction
1914.6 Preparing : 1/1
1917.6 Upgrading : zlib-ng-compat-2.1.7-2.fc40.aarch64 1/255
...
1989.6 Running scriptlet: expat-2.6.0-1.fc40.aarch64 255/255
...
2025.9 Complete!
dnf clean:
2036.9 37 files removed
DONE 2037.9s
hroncok commented on Sep 26, 2024
Notably, no output is produced for ~12 minutes (730 seconds). "Fedora 40 - aarch64 - Updates" then takes another 6 minutes (370 seconds). In the second builddep command, another six minutes (350 seconds) elapse from "enabling fedora-source repository" to the first "Package [...] is already installed.".
That's pretty bad, but I have no idea why would it take so long. Perhaps it's the emulation/virtualization that is there that is so slow? trallard commented on Oct 7, 2024 I was having a quick nosey here 👋🏽 but looking at the Actions runs, the build is taking < 3mins and the 35 mins + was an outlier. corona10 commented on Oct 8, 2024 @trallard
FYI, 3 mins running releases were about Autoconf :)
For the devcontainer it takes a long time, but thanks to your comment, I've written up a new PR for distinguishing which packages were released :)
Please take a look if you are interested :)
#16 Reacted by Tania Allard and Erlend E. Aasland trallard commented on Oct 8, 2024 Ah that makes sense. Will have a look over there and maybe do some digging on why this is taking so long then. corona10 commented on Oct 9, 2024 hroncok commented on Oct 10, 2024 We have a container job at github actions that runs dnf on 4 architectures. The arm64 one (third in the screenshot) is regularly extremely slow:
efimov-mikhail commented on Nov 24, 2024 What do you think about splitting actual container creation into two steps? One with big dnf steps, which rarely changes. And another container with all other steps. Unfortunately, we should use some sort of links to actual containers. But regular updates can be much faster with this scheme. Reacted by Erlend E. Aasland brettcannon commented on Nov 25, 2024 I'm personally fine with that if @corona10 and @erlend-aasland are. I have #12 open to split things up in regards to WASI, but I wasn't sure about following through with it due to having to change the CPython repo when new containers go out. We might need to settle on some policy in #11 to know how much work would be expected. erlend-aasland commented on Nov 25, 2024
I'm personally fine with that if @corona10 and @erlend-aasland are.
+1 corona10 commented on Nov 26, 2024 brettcannon commented on Nov 17, 2025 Based on the last release run for the dev container, this is now down to under 11 minutes, which seems reasonable to me, so I'm going to close this issue.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|
