GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: Build on UBI 10 and fix the Hadoop native build against OpenSSL 3 by razvan · Pull Request #1534 · stackabletech/docker-images

Conversation

@razvan

@razvan razvan commented Jun 16, 2026 •
edited by lfrancke
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Description

Fixes: stackabletech/issues#845

Moves stackable-base and stackable-devel to ubi10/ubi-minimal, and makes the Hadoop native code build and work against RHEL 10's OpenSSL 3.5.

What changes

  • stackable-base, stackable-devel: ubi10/ubi-minimal@sha256:ab67cd02caf8706e34a695a1432ebe965934f6128ec352a6c3f4048b43a4fb5e, expected certs package ca-certificates-2026.2.92_v9.0.319-100.3.el10_2.

  • java-base installs gzip.
    tar -z needs it, and the hadoop final stage unpacks async-profiler with curl | tar -xz.
    On UBI 9 every image built on java-base had gzip, because krb5-workstation pulls in pam, which pulls in cracklib, which requires gzip; on UBI 10 krb5-workstation no longer depends on pam.
    It goes into java-base rather than the hadoop image so that every Java image keeps what it had on UBI 9, and images that never had it do not gain it.
    The installed version is gzip-1.13-4.el10_2, which contains the fixes for CVE-2026-41991 and CVE-2026-41992 (RHSA-2026:61625).

  • hadoop 3.4.3 and 3.5.0: three patches for HADOOP-19872:

    • HADOOP-19872: Exclude Engine API when building with OpenSSL 3.x apache/hadoop#8550, which excludes the ENGINE API from OpensslSecureRandom.c when OpenSSL is built without it;
    • a guard of the remaining ENGINE code paths in OpensslSecureRandom;
    • a guard of the ENGINE_by_id and ENGINE_free lookups in OpensslCipher.c, plus the Windows ENGINE_by_free typo.

    The touched files are byte-identical in 3.4.3, 3.5.0 and trunk.
    The two follow-up patches are not part of HADOOP-19872: Exclude Engine API when building with OpenSSL 3.x apache/hadoop#8550.

  • hadoop 3.3.6, which hive 3.1.3 and 4.0.0 build against:

    • the first two HADOOP-19872 patches.
      Its OpensslSecureRandom.c is byte-identical to 3.4.3's, and its OpensslCipher.c never looks up an ENGINE, because engine selection arrived in 3.4, so the third patch has nothing to guard;
    • HADOOP-19551 (HADOOP-19551. Fix compilation error of native libraries on newer GCC. apache/hadoop#7644) and HDFS-17226 (HDFS-17226. Building native libraries fails on Fedora 38. apache/hadoop#6191), which 3.4.3 and 3.5.0 already contain.
      GCC 14 makes implicit function declarations an error, which OpensslSecureRandom.c trips over in the pre-1.1 lock helpers, and its headers no longer pull in <cstdint> for libhdfspp.

Why the cipher needs the guard as well

apache/hadoop#8550 alone makes the build pass, but OpensslCipher.initIDs still resolves ENGINE_by_id and ENGINE_free at runtime.
RHEL 10's headers define OPENSSL_NO_ENGINE, yet its libcrypto still exports both symbols, so the lookup succeeds there by accident.
Against a libcrypto without them, initIDs throws UnsatisfiedLinkError and the AES/CTR codec falls back to JceAesCtrCryptoCodec with only a warning, so encrypted data transfer keeps working and nothing reports that native crypto is gone.
With the guard, libhadoop.so references no ENGINE symbol at all.

Testing

Built with boil build hadoop=3.4.3,3.5.0.

In both images:

  • hadoop checknative -a reports openssl: true /lib64/libcrypto.so (openssl-libs 3.5.8).
  • strings libhadoop.so shows no ENGINE_ symbol.
  • A native AES/CTR round trip through OpensslAesCtrCryptoCodec with OpensslSecureRandom: 1 MiB, ciphertext identical to JCE's AES/CTR/NoPadding, decrypts back to the input.

hadoop 3.3.6 builds and passes the same three checks: openssl: true /lib64/libcrypto.so, no ENGINE_ string, and the AES/CTR round trip matching JCE.
zstd and bzip2 report false in checknative, as they do in the 26.7.0 hadoop image.

hive 3.1.3 builds against the patched 3.3.6, and schematool -dbType derby -initSchema completes in the image.
Its checknative reports openssl: false because the hive image has no libcrypto.so symlink; the UBI 9 hive images behave the same, so this is not new.

On a k3d cluster with the 26.7.0 operators, the HDFS kerberos test (both realms, ZooKeeper 3.9.5, OPA 1.16.2, dfs.encrypt.data.transfer with AES/CTR) passes for 3.4.3 and 3.5.0.
Its datanodes log OpensslAesCtrCryptoCodec, which they do not when the codec falls back to JCE.

Known UBI 10 differences

  • gzip, see above.
  • tput is not installed, so the hadoop scripts print tput: command not found in places. Cosmetic.
  • Vector keeps the el9 inotify-tools RPM; it installs and inotifywait runs on UBI 10.

Open question

This changes the base of every image, not only Hadoop.
#1349 asks whether UBI 10 images should be built alongside the UBI 9 ones or replace them, and that is not settled.
Only hadoop 3.4.3 and 3.5.0 ran on a cluster; 3.3.6 and hive 3.1.3 were built and checked in the image only.
No image other than hadoop and hive was built or tested on UBI 10.

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

  • Changes are OpenShift compatible
  • All added packages (via microdnf or otherwise) have a comment on why they are added
  • Add an entry to the CHANGELOG.md file
  • Integration tests ran successfully
TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
razvan added 5 commits April 21, 2026 11:44
@razvan
chore: update ubi to version 10
9b1cc94
@razvan
Squash merge chore/boost-from-source into feat/ubi10-hadoop
f569bc3
@razvan
Merge branch 'main' into feat/ubi10-hadoop
cfde301
@razvan
update image hash and try out ssl 3 fix for 3.5.0
baf1bde
@razvan
add openssl fix for ubi 10 to 3.4.x versions
8da2464
@razvan razvan self-assigned this Jun 16, 2026
razvan added 4 commits June 16, 2026 12:59
@razvan
remove hadoop/boost
2f24703
@razvan
merge hadoop Dockerfiles from maim
8f64553
@razvan
Merge branch 'main' into feat/ubi10-hadoop
9eee43c
@razvan
install gzip
c66329b
@lfrancke lfrancke added this to Stackable Engineering Oct 29, 2026
@lfrancke lfrancke moved this to Development: In Review in Stackable Engineering Oct 29, 2026
@lfrancke lfrancke removed the status in Stackable Engineering Oct 29, 2026
@lfrancke lfrancke removed this from Stackable Engineering Oct 29, 2026
lfrancke added 7 commits October 29, 2026 13:00
@lfrancke
Merge branch 'main' into feat/ubi10-hadoop
670db25
@lfrancke
fix: Bump the UBI 10 base image and install gzip in java-base
abbb5a2
The hadoop final stage unpacks async-profiler with `curl | tar -xz`, which needs gzip.
On UBI 9 every image built on java-base had it, because krb5-workstation pulls in pam, which pulls in cracklib, which requires gzip.
On UBI 10 krb5-workstation no longer depends on pam, so nothing installs gzip.
It goes into java-base so that the Java images keep what they had on UBI 9, and images that never had it do not gain it.
@lfrancke
fix(hadoop): Guard every OpenSSL ENGINE call in the native code
56ef78e
apache/hadoop#8550 alone leaves OpensslCipher resolving ENGINE_by_id and ENGINE_free in initIDs.
RHEL 10's headers define OPENSSL_NO_ENGINE, but its libcrypto still exports both symbols, so the lookup succeeds there by accident.
Against a libcrypto without them, initIDs throws UnsatisfiedLinkError and the AES-CTR codec falls back to JCE with only a warning.
The two follow-up patches guard the remaining ENGINE paths in OpensslSecureRandom and OpensslCipher, and fix the misspelt ENGINE_by_free on Windows.
@lfrancke
fix(hadoop): Backport HADOOP-19551 to 3.3.6 for GCC 14
81f2a8c
GCC 14, which UBI 10 ships, turns implicit-function-declaration into an error by default.
OpensslSecureRandom.c defines locks_setup and locks_cleanup outside the version guard that declares the dlsym_CRYPTO_* lock functions they call, so hadoop-common's native build fails on UBI 10.
Only OpenSSL before 1.1 calls them.
3.4.3 and 3.5.0 already contain HADOOP-19551, which restores the old default with -Wno-error=implicit-function-declaration.
@lfrancke
fix(hadoop): Backport HDFS-17226 to 3.3.6 for GCC 14
f300df9
The libstdc++ headers of GCC 14, which UBI 10 ships, no longer pull in <cstdint> transitively, so the libhdfspp public headers that use uint16_t, uint32_t and uint64_t without including it fail to compile on UBI 10.
3.4.3 and 3.5.0 already contain HDFS-17226, which adds the include.
@lfrancke
fix(hadoop): Guard the OpenSSL ENGINE calls in 3.3.6 as well
c533ed1
Hive 3.1.3 and 4.0.0 build against Hadoop 3.3.6, so 3.3.6 has to build on UBI 10 too.
Its OpensslSecureRandom.c is identical to 3.4.3's and fails to compile against headers that define OPENSSL_NO_ENGINE in the same way.
OpensslCipher.c in 3.3.6 never looks up an ENGINE, because engine selection and SM4 only arrived in 3.4, so the third patch applied to 3.4.3 and 3.5.0 has nothing to guard here.
@lfrancke
docs: Add a changelog entry for UBI 10 and the Hadoop OpenSSL 3 fix
f6dac7f
@lfrancke lfrancke mentioned this pull request Oct 29, 2026
Migrate to UBI 10 stackabletech/issues#905
Open
@lfrancke lfrancke changed the title fix: hadoop build with openssl 3 on UBI 10 feat: Build on UBI 10 and fix the Hadoop native build against OpenSSL 3 Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@razvan razvan

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Hadoop on UBI10

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@razvan @lfrancke
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.