GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — BUG: Avoid re-extracting the ITK build cache for module dependencies by axel-grc · Pull Request #307 · InsightSoftwareConsortium/ITKPythonPackage

Conversation

@axel-grc

@axel-grc axel-grc commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

When building a module that depends on another ITK module (e.g. RTK depends on ITKCudaCommon), the ITK build cache is downloaded and extracted once in the parent directory. build-module-deps.sh then symlinks the module directory to that cache (ITKPythonPackage -> ../ITKPythonPackage) and used to re-run the full download-and-extract flow inside the module directory.

Re-extracting in the module directory was problematic for two reasons:

  • Redundant: the same cache was just extracted in the parent directory with the same arguments, and the module directory writes through the symlink into that same shared tree.

  • Recent GNU tar (openat2 jailification, e.g. Ubuntu tar 1.35+dfsg-4ubuntu0.2+) fails with 'Invalid cross-device link' (EXDEV) when extracting members under ITKPythonPackage/ because the symlink escapes the extraction root.

The module dependency build also registers the module (e.g. CudaCommon) into the shared ITK build tree (ITK_DIR) it builds against, so the registration must land in the tree the parent build reuses afterwards. Building against a module-local copy instead of the shared tree breaks serial module dependencies.

Fix: when the parent already holds the extracted cache, build the module directly against the shared tree with
dockcross-manylinux-build-module-wheels.sh, without re-extracting it. When no cache is available (standalone use of build-module-deps.sh), keep the full download-and-extract-and-build path.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@axel-grc
BUG: Avoid re-extracting the ITK build cache for module dependencies
84c6c7f
When building a module that depends on another ITK module (e.g. RTK
depends on ITKCudaCommon), the ITK build cache is downloaded and
extracted once in the parent directory. build-module-deps.sh then
symlinks the module directory to that cache (ITKPythonPackage ->
../ITKPythonPackage) and used to re-run the full download-and-extract
flow inside the module directory.

Re-extracting in the module directory was problematic for two reasons:

- Redundant: the same cache was just extracted in the parent directory
  with the same arguments, and the module directory writes through the
  symlink into that same shared tree.

- Recent GNU tar (openat2 jailification, e.g. Ubuntu tar
  1.35+dfsg-4ubuntu0.2+) fails with 'Invalid cross-device link' (EXDEV)
  when extracting members under ITKPythonPackage/ because the symlink
  escapes the extraction root.

The module dependency build also registers the module (e.g. CudaCommon)
into the shared ITK build tree (ITK_DIR) it builds against, so the
registration must land in the tree the parent build reuses afterwards.
Building against a module-local copy instead of the shared tree breaks
serial module dependencies.

Fix: when the parent already holds the extracted cache, build the module
directly against the shared tree with
dockcross-manylinux-build-module-wheels.sh, without re-extracting it.
When no cache is available (standalone use of build-module-deps.sh),
keep the full download-and-extract-and-build path.
dzenanz
dzenanz reviewed Sep 1, 2026
View reviewed changes

@dzenanz dzenanz left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good on a glance. It would be good if someone else reviewed too. Does it make sense to apply this patch to main?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@axel-grc

axel-grc commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Looks good on a glance. It would be good if someone else reviewed too. Does it make sense to apply this patch to main?

We need this fix for RTK next release which will be based on release 5.4, but I think we can apply this to main as well.

dzenanz reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@dzenanz
dzenanz requested review from hjmjohnson and thewtex September 1, 2026 15:35
@SimonRit

SimonRit commented Sep 2, 2026

Copy link
Copy Markdown

Thanks @axel-grc. I have the feeling that the problem this PR addresses has been fixed, I guess by an update of the GitHub runner. See failing workflow with runner 2.336 and successful workflow with runner 2.336. Do you confirm? The problem is I don't see anything in the runner release log that explains it.
The PR is still beneficial IMO as it avoids unnecessarily untaring one file.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@axel-grc

axel-grc commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @axel-grc. I have the feeling that the problem this PR addresses has been fixed, I guess by an update of the GitHub runner. See failing workflow with runner 2.336 and successful workflow with runner 2.336. Do you confirm? The problem is I don't see anything in the runner release log that explains it. The PR is still beneficial IMO as it avoids unnecessarily untaring one file.

Even with runner 2.337, the tar error is still there, so the fix is required.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

thewtex
thewtex approved these changes Sep 3, 2026
View reviewed changes

@thewtex thewtex left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@axel-grc thank you!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@thewtex
thewtex merged commit a3b037d into InsightSoftwareConsortium:release-5.4 Sep 3, 2026
@thewtex

thewtex commented Sep 3, 2026

Copy link
Copy Markdown
Member

Also pushed to main

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SimonRit SimonRit mentioned this pull request Sep 4, 2026
WIP: Validate ITKPythonPackage EXDEV fix RTKConsortium/RTK#991
Closed
@hjmjohnson hjmjohnson mentioned this pull request Sep 16, 2026
ENH: Python build system for ITK 6 wheels #302
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dzenanz dzenanz dzenanz left review comments
@thewtex thewtex thewtex approved these changes
@hjmjohnson hjmjohnson Awaiting requested review from hjmjohnson

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@axel-grc @SimonRit @thewtex @dzenanz
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.