GeezerVison is showing github.com — Migrate from cisd:jhdf5 to janelia-jhdf5 · Issue #314 · scijava/pom-scijava
Migrate from cisd:jhdf5 to janelia-jhdf5 #314DescriptionFollow-up to #181. We are now on cisd:jhdf5 19.04.1, which is unmaintained. It bundles HDF5 1.10 native libraries for only a few platforms. janelia-jhdf5 (org.janelia:janelia-jhdf5) is a maintained fork with the same ch.systemsx.cisd.hdf5.* API (Java 8+, Apache-2.0). It runs on HDF5 2.x with natives from org.bytedeco:hdf5 for Linux, macOS and Windows on x86-64 and arm64. It vendors cisd:base under the same package names, so it must not be on the class path together with cisd:jhdf5 or cisd:base.
Status
janelia-jhdf5 is not released yet (26.10.0-SNAPSHOT, depending on org.bytedeco hdf5 and javacpp snapshots). Until it is, the PRs below are drafts that carry temporary plumbing (snapshot repository, javacpp.version / hdf5.version overrides, a CI step that builds and installs the janelia-jhdf5 snapshot, -Denforcer.skip=true), each marked "remove once janelia-jhdf5 is released".
Plan:
- Release janelia-jhdf5 as
26.10.0 (versions follow JHDF5's YY.MM.patch scheme; the groupId is org.janelia, not org.scijava). This is blocked on a stable javacpp-presets 1.5.15 release that includes HDF5 2.2.0 (Upgrade presets for HDF5 2.2.0 bytedeco/javacpp-presets#1808, merged) and the Windows fixes (#1814, merged). Both are in the 1.5.15-SNAPSHOT builds only; the latest release is 1.5.14 with HDF5 1.14.3. Multi-platform natives (Depend on hdf5-platform instead of only the macosx-arm64 natives JaneliaSciComp/janelia-jhdf5#11) are merged.
- Manage
org.janelia:janelia-jhdf5 in pom-scijava.
- Take the draft PRs out of draft, with the temporary plumbing removed.
- Drop the
cisd:jhdf5 exclusions that some PRs add while the migration is in progress (see below).
Changes in janelia-jhdf5 since this issue was opened
The draft PRs below were updated to the new groupId and version.
API differences from cisd:jhdf5 19.04.1
Compared by signature: all public classes of 19.04.1 exist in janelia-jhdf5. The only missing public members:
HDF5ObjectInformation.getAddress(): use getToken() / getFileNumber() (HDF5 2.x identifies objects by token).- The old JNI helper natives in
HDFHelper (getPointerSize, compoundCpyVLStr, freeCompoundVLStr, createVLStrFromCompound); replaced by pure Java.
Note that IHDF5Reader.getDoubleArrayAttribute(...) and the other IHDF5LegacyReader methods were already removed in 19.04.0. Code that only still compiled because cisd:jhdf5 14.12.x was on the class path through some dependency needs reader.float64().getArrayAttr(...) etc. (see saalfeldlab/cremi-deform#3).
Tier 1: direct dependency, source uses the API
Needs an actual migration PR. [x] = merged and released.
Tier 2: direct dependency, no (or test-only) source use
Verify whether the dependency is needed at all; if it is only a transitive workaround, it can simply be removed.
Tier 3: transitive through bioformats (or n5-hdf5 / bigdataviewer-core)
These only exclude ch.systems.cisd:jhdf5 today and get cisd:jhdf5 transitively. They need no change of their own once bioformats migrates, other than dropping the exclusion for the old groupId and checking that cisd:jhdf5 / cisd:base no longer appear in mvn dependency:tree.
- saalfeldlab/bigwarp, saalfeldlab/template-building
- fiji/SPIM_Registration
- JaneliaSciComp/multiview-reconstruction, JaneliaSciComp/multiview-simulation, JaneliaSciComp/BigStitcher-Spark
- PreibischLab/mvrecon, PreibischLab/Blob_Tracker4D (inactive since 2017/2020)
- juglab/labkit-ui
- UU-cellbiology/BigTrace, UU-cellbiology/bigvolumebrowser
- mianalysis/mia, BIOP/bigdataviewer-image-loaders, BIOP/quick-start-czi-reader
Temporary exclusions to remove
Until n5-hdf5 and bigdataviewer-core are released with janelia-jhdf5, projects that depend on them get cisd:jhdf5 19.04.1 transitively, so both jars would be on the class path. The PRs for n5-utils, STIM, transform-helpers and cremi-deform therefore add cisd:jhdf5 exclusions on n5-hdf5 / bigdataviewer-core / bigcat. Remove them once those are released.
Finding more consumers
gh search code jhdf5 --owner <org> --filename pom.xml per organization, then check each pom for a direct dependency vs. an exclusion and for HDF5Factory in the Java sources. The script used for this list:
find-jhdf5-consumers.sh
#!/usr/bin/env bash
# List GitHub repos whose pom.xml mentions jhdf5, classified as direct dependency / exclusion-only,
# plus whether Java sources use HDF5Factory. Code search is limited to 10 req/min, hence the sleeps.
# Usage: find-jhdf5-consumers.sh org1 org2 ...
set -u
for o in "$@"; do
gh search code jhdf5 --owner "$o" --filename pom.xml --limit 100 --json repository,path \
--jq '.[]|.repository.nameWithOwner+" "+.path'
sleep 7
done | sort -u | while read -r repo path; do
pom=$(gh api -H "Accept: application/vnd.github.raw" "repos/$repo/contents/$path" 2>/dev/null) || continue
kind=$(python3 -I -c '
import re,sys
t=sys.stdin.read(); k=set()
for m in re.finditer(r"<dependency>(.*?)</dependency>",t,re.S):
b=m.group(1)
if re.search(r"<artifactId>\s*jhdf5\s*</artifactId>",b):
# outside any <exclusion> block => direct
k.add("direct" if re.search(r"<artifactId>\s*jhdf5\s*</artifactId>",re.sub(r"<exclusions>.*?</exclusions>","",b,flags=re.S)) else "exclusion")
print(",".join(sorted(k)) or "mention-only")' <<<"$pom")
src=$(gh search code HDF5Factory --repo "$repo" --json path --jq 'length'); sleep 7
echo "$repo $path | pom: $kind | HDF5Factory files: $src"
done
CC @ctrueden Reacted by Curtis Rueden Reacted by Dominik Kutra MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|