GeezerVison

libc++ 23 dropped many transitive includes, which breaks the build with
the LLVM toolchain.

Include <cerrno> explicitly.

Signed-off-by: Pepper Gray <[email protected]>

The tests used the first objcopy in $PATH, which does not exist
without GNU binutils.

Use the objcopy CMake found for the toolchain (e.g., llvm-objcopy).

Signed-off-by: Pepper Gray <[email protected]>

which() listed every directory in $PATH, which throws on directories
which are searchable but not readable, and it returned files which are
not executable.

- check each candidate with access() instead
- skip empty entries
- check names containing a slash directly, like exec*
- add tests

Signed-off-by: Pepper Gray <[email protected]>

The plugin search split getenv("PATH") without checking it for null.

Use splitEnv(), which treats an unset $PATH as empty.

Signed-off-by: Pepper Gray <[email protected]>

The plugin search matched each file name in a temporary string, which
was destroyed before the plugin's name was read from the match, so it
was read from freed memory.

Match a named string with std::smatch instead.

Signed-off-by: Pepper Gray <[email protected]>

executeDeferredOperations() recorded that patchelf failed to set an
rpath, but returned true, so linuxdeploy succeeded with an AppDir whose
files do not find their libraries.

- return whether all rpaths were set
- add tests

Signed-off-by: Pepper Gray <[email protected]>

A shared library which needs no other libraries looks like a static-pie
executable, but no fixture lets the tests tell them apart.

Add simple_library_no_dependencies, linked with -nostdlib. Disable
clang's warning about unused runtime options from $LDFLAGS for it, and
do not instrument it for coverage, as the coverage runtime needs the C
library.

Signed-off-by: Pepper Gray <[email protected]>

static-pie executables (e.g., Rust's musl targets) have no PT_INTERP,
but a PT_DYNAMIC segment like shared libraries, so they were traced
with ldd and had their rpath set.

Treat files with the DF_1_PIE flag and no DT_NEEDED entries as static,
and return no dependencies for static files instead of asserting. Add
static-pie and interpreter-less PIE fixtures, and keep the static
fixture non-PIE, as GCC on Alpine links -static as static-pie.

Signed-off-by: Pepper Gray <[email protected]>

The constructor threw without closing the pipes it had created (e.g.,
if the second pipe() or fork() failed), leaking up to four file
descriptors per call.

- give the file descriptors owners, which close them unless handed over
- add tests

Signed-off-by: Pepper Gray <[email protected]>

If exec* failed (e.g., for a missing ldd), the child threw, so the
caller's code continued to run in the child.

Prepare the exec* vectors before forking, and do not allocate or throw
in the child (execvpe, which POSIX does not list as async-signal-safe,
does not allocate on the heap in glibc, musl or FreeBSD), but exit with
127, like shells do for commands which cannot be run.

Signed-off-by: Pepper Gray <[email protected]>

A process terminated by a signal reported the signal's number as its
exit code, so callers could not tell SIGKILL apart from exit 9.

Add term_signal(), and exit_description() for messages, which
check_output() uses. exit_code() is unchanged.

Signed-off-by: Pepper Gray <[email protected]>

If linuxdeploy's standard streams are closed, the pipes get their
numbers, so connecting one end in the child replaced another.

Move the pipe ends above the standard streams, and test it.

Signed-off-by: Pepper Gray <[email protected]>

Programs started while the pipes were open (e.g., by another thread)
inherited them, so the reader saw no end of file until they exited.

Create the pipes with O_CLOEXEC.

Signed-off-by: Pepper Gray <[email protected]>

If exec* failed, the child exited with 127 without saying why, which
callers could not tell from a program exiting with 127. Instead, they
searched for programs up front, which does not follow the rules of
exec* (e.g., for an unset $PATH or empty entries).

Send the errno to the parent through a pipe which exec* closes, and add
exec_error(). exit_description() describes it, so check_output()
reports it, and the plugin process handler logs it. Test the search
rules of exec*, so callers can rely on them instead of searching up
front.

Signed-off-by: Pepper Gray <[email protected]>

check_output() threw a std::logic_error if the program failed or could
not be run, like the errors of starting any program (e.g., too many
open files), so callers could not tell them apart.

Throw the new called_process_error instead, which derives from
std::logic_error, so that existing handlers still catch it.

Signed-off-by: Pepper Gray <[email protected]>

run() read stdout until its end before reading stderr, so a program
which wrote more to stderr than its pipe holds (e.g., musl's ldd,
which reports every unresolved symbol of a plugin) blocked, and with
it linuxdeploy.

Read the data which is available from each pipe in turn, and restore
the read buffer's size before each read, as read() shrinks it. Drop
the sleep between the rounds, as poll() in each read already waits for
data.

Signed-off-by: Pepper Gray <[email protected]>

An uncaught exception (e.g., for a missing patchelf) aborted linuxdeploy
and dumped core.

Catch exceptions in main(), report them, and exit with an error.

Signed-off-by: Pepper Gray <[email protected]>

If patchelf failed to read an rpath, or could not be started,
getRPath() returned an empty one, which replaced the existing rpath
with $ORIGIN.

Throw a PatchelfError instead, which fails the deployment, and do not
strip a file whose rpath cannot be read. Return an empty rpath without
running patchelf for static and debug symbols files.

Signed-off-by: Pepper Gray <[email protected]>

A patchelf which could not be run was treated like one which failed, and
a strip which could not be run was reported as "Strip call failed:"
without a reason, once per file.

Report both with the reason: patchelf through a PatchelfError, which
fails the deployment, and strip once, then stop stripping the others.
For a patchelf or strip which ran, but failed, report how it ended.
Tests follow with $STRIP and $PATCHELF.

Signed-off-by: Pepper Gray <[email protected]>

The LLVM toolchain provides llvm-strip rather than strip, so stripping
failed.

Run $STRIP (a name or a path), otherwise the strip next to linuxdeploy
or the first in $PATH, without falling back. Implement the lookup as
util::findTool(), so patchelf can share it, document it in the README
and test it. FreeBSD has no /proc to read linuxdeploy's path from, so
the tests of the bundled strip are skipped there.

Closes: linuxdeploy#325
Signed-off-by: Pepper Gray <[email protected]>

$PATCHELF had to be an existing file, so a name failed, an empty
$PATCHELF counted as set, and the search in $PATH crashed if $PATH was
unset.

- use util::findTool() like for strip
- test the lookup and how patchelf errors are handled
- update README

Signed-off-by: Pepper Gray <[email protected]>

The plugin search threw on directories in $PATH which are searchable
but not readable, on plugins which cannot be accessed (e.g., a symlink
loop), and on plugins whose query fails (e.g., which cannot be run),
which ended linuxdeploy on every start.

Skip such directories and plugins, and report the plugins as "Could not
load plugin". Errors which affect any program (e.g., too many open
files) still end linuxdeploy. Test these through --list-plugins.

Signed-off-by: Pepper Gray <[email protected]>

build.sh and build-in-docker.sh failed without $ARCH.

Default it to uname -m, and accept uname's names (e.g., armv7l for
armhf) in both, as $ARCH also names the AppImage.

Signed-off-by: Pepper Gray <[email protected]>

Rootless Podman maps the host user's uid (which is used by the build)
to a subordinate uid, which cannot write to the mounted working tree.

Pass --userns=keep-id with rootless Podman, and ignore the HSTS database
wget then saves in $HOME, which is the working tree.

Signed-off-by: Pepper Gray <[email protected]>

build.sh bundled the first patchelf and strip in $PATH, not the ones
linuxdeploy runs (e.g., llvm-strip set in $STRIP).

Bundle those, linked under their default names, by which linuxdeploy
looks for the bundled tools.

Signed-off-by: Pepper Gray <[email protected]>

Linux CI only builds in Debian, with glibc and GCC.

Add images for Alpine (musl with GCC or LLVM) and Gentoo (glibc or musl
with GCC, musl with LLVM). build-in-docker.sh selects the distribution
by its first argument (or $TARGET, default debian) and the toolchain by
$TOOLCHAIN. Set CMAKE_POLICY_VERSION_MINIMUM for Alpine's CMake 4, and
build gtest from source for libc++. The musl builds also need unmerged
fixes of the submodules, which "ci: Apply the patches of unmerged PRs"
applies.

Signed-off-by: Pepper Gray <[email protected]>

CImg sizes its buffer limit by UINTPTR_MAX without including the header
defining it. When CImg.h comes first, the 64-bit limit of 16 GiB
overflows i386's 32-bit unsigned long to 0, so every CImg allocation
throws, including cimg::info()'s. No test ran simple_executable before,
so this went unnoticed.

src/core/appdir.cpp only gets UINTPTR_MAX through <filesystem> (e.g.,
with libstdc++), which need not include <cstdint>, so include it there,
too.

Signed-off-by: Pepper Gray <[email protected]>

Dependency tracing depends on the platform's ldd (glibc, musl or
FreeBSD), which no single test machine covers.

Add scenarios which test_dependency_tracing_real runs with the system's
ldd, recording its calls, and test_dependency_tracing_mock replays with
the recordings of every platform. Add the fixture chain
simple_executable -> dependency_parent -> dependency_leaf.

Signed-off-by: Pepper Gray <[email protected]>

There is no script to check dependency tracing on a platform, from the
tests to a deployed AppDir.

test-dependency-tracing.sh builds linuxdeploy, runs all tests, deploys
simple_executable to an AppDir, checks its libraries and runs it.
$BUILD_DIR reuses a build directory. build-in-docker.sh runs it if
$BUILD_TYPE is dependency-tracing.

Signed-off-by: Pepper Gray <[email protected]>

FreeBSD's ldd appends an address to missing libraries
("libfoo.so => not found (0)"), which the expression for resolved
libraries matched, so "not found" was deployed as a library path.

Check for missing libraries first, with an expression of their own, and
move the exclusion check into a lambda, so other reports can share it.

Signed-off-by: Pepper Gray <[email protected]>

musl's ldd exits with a non-zero status for missing libraries and for
unresolved symbols (e.g., of plugins whose symbols the host program
provides), while still listing the resolved libraries. This threw a
std::runtime_error, which ended linuxdeploy.

Accept excluded missing libraries and unresolved symbols. As every
deployed library is traced again, the warning only counts the symbols,
and the debug log lists them. Reject all other diagnostics, and an ldd
which could not be run or was terminated, through the new
DependencyTraceError, which fails the deployment.

Signed-off-by: Pepper Gray <[email protected]>

Without ldd (e.g., Alpine without musl-utils), the error read "Failed to
run ldd on <file>: No such file or directory", as if the traced file
were missing.

Report ldd or its interpreter as missing if exec* reports ENOENT, and
test that other errors are still reported with their reason.

Signed-off-by: Pepper Gray <[email protected]>

FreeBSD's ldd lists libraries not named lib* without their path or
whether they were found ("ldd (0x...)"), so they were neither deployed
nor reported as missing.

Set their format to that of the others through
LD_TRACE_LOADED_OBJECTS_FMT2.

Signed-off-by: Pepper Gray <[email protected]>

musl's libc is also its loader. Its ldd lists it without "=>" by the
name it was run as (e.g., /lib/ld-musl-*.so.1 or "ldd"), and the
libraries it provides itself (e.g., libc.so) resolve to that name, so
they were deployed.

Skip entries which resolve to an object listed without "=>", so musl is
recognized by any name, while libraries which only share such a name
are still deployed. Check in test-dependency-tracing.sh that no musl
runtime object is deployed.

Closes: linuxdeploy#340
Signed-off-by: Pepper Gray <[email protected]>

To skip glibc's loader and the vDSO, every ldd line containing
"ld-linux-" or "linux-vdso.so" was dropped, and with it libraries in
paths like build-linux-x86_64/.

Recognize the loaders by their file name instead (e.g., ld-linux-*.so.*,
ld64.so.*, ld.so.* or ld-musl-*.so.1), but only where ldd lists them, by
their path, so that libraries searched for by a loader-like name (e.g.,
ld.so.1) are still deployed. Recognize the vDSO by its name without a
path. Both rules also cover the lines glibc 2.36 listed with "=>" (see
linuxdeploy#210).

Signed-off-by: Pepper Gray <[email protected]>

glibc's ldd lists libraries needed by their path (e.g., an absolute
DT_NEEDED, or one relative to $ORIGIN, which it expands) without "=>",
so they were dropped, while musl's and FreeBSD's ldd list them with
"=>", so they were deployed without a word.

Deploy them on all platforms, with a warning unless they are excluded,
as the loader only uses the copy if the path is relative to $ORIGIN and
the copy is at that path relative to the file which needs it. Exclude
them by their file name, also when they are missing. Skip the traced
file's PT_INTERP, which is listed without "=>", too, in case the
loader's file name is not recognized (e.g., LSB's ld-lsb-x86-64.so.3).

Signed-off-by: Pepper Gray <[email protected]>

ldd lists the libraries which $LD_PRELOAD or glibc's /etc/ld.so.preload
preload like dependencies, so they were deployed (e.g., libarmmem,
which Raspberry Pi OS preloads system-wide).

Skip all libraries $LD_PRELOAD and /etc/ld.so.preload list, which ldd
lists by their entry there (e.g., with $LIB), also those given by name
(e.g., fakeroot's libfakeroot-sysv.so). Keep $LD_PRELOAD for ldd, as
tools which rely on it (e.g., fakechroot) would not apply to it
otherwise. The dependencies of preloaded libraries are still deployed,
as ldd lists them like the program's. Warn once per deployment about
each skipped library, as one the program needs, too, is only listed as
preloaded.

Signed-off-by: Pepper Gray <[email protected]>

The mock tests replay the ldd calls of every platform, which must be
recorded on each of them.

test-dependency-tracing.sh --record <platform> runs the real tests and
replaces the platform's recordings, or only adds those which ran if the
tests failed or were filtered. build-in-docker.sh records if
$BUILD_TYPE is record-ldd-calls. Add the recordings of Debian, Alpine
(musl and LLVM), Gentoo (LLVM) and FreeBSD.

Signed-off-by: Pepper Gray <[email protected]>

The dependency tracing tests do not run in CI.

Run test-dependency-tracing.sh on x86_64 in Debian (glibc with GCC),
Alpine (musl with GCC and LLVM) and Gentoo (musl with LLVM), and on
FreeBSD.

Signed-off-by: Pepper Gray <[email protected]>

The musl builds need fixes which are not merged yet:
- linuxdeploy/linuxdeploy-plugin-appimage#45
- linuxdeploy/linuxdeploy-plugin-appimage#46
- linuxdeploy/linuxdeploy-desktopfile#11
- linuxdeploy/cmake-scripts#1

Carry them in ci/upstream-patches, and apply them in build.sh and the
workflow. Revert this commit once all of them are merged.

Signed-off-by: Pepper Gray <[email protected]>