Release 4.3.0 - ???

BREAKING CHANGES AND DEPRECATIONS

* tika-pipes-* and two tika-langdetect modules no longer pull in a log4j backend;

embedding apps (and their PipesServer forks) must add an slf4j binding, e.g.

log4j-slf4j2-impl (TIKA-5058).

* Markdown output writes math as $`...`$ (a code span, which GitHub and GitLab

render as math) instead of raw $...$, so text in a math span can no longer become

HTML or links. Readers that only know $...$, such as pandoc, show it as code. Only

spans with class "math inline" or "math display" count as math; a PDF structure

type or Word style named "Math" is escaped like other text (TIKA-5056).

* tika-ml-junkdetect has a new model and API. JunkDetector wraps a HashedJunkModel

(junkdetect.bin, format JUNKHNG1; the old model format is not read). knownScripts(),

scoreWithFeatureComponents() and the per-feature compute* methods are gone; use

getModel() for the trained scripts, the worst-window score and the pairwise head.

TextQualityScore.getZScore() is now a logit (positive = reads as clean text) with no

interval estimate, and is unknown for text with no letters or with letters only in

untrained scripts. JunkFilterEncodingDetector takes a HashedJunkModel instead of a

TextQualityDetector (TIKA-5027).

* OneNote sections, .one and FSSHTTPB, give each page's current text and metadata

only, not every stored revision. Set includePageHistory on one-note-parser, in the

config or per request, for earlier revisions and saved page versions, and

includeDeletedContent for FSSHTTPB recycle-bin pages. All .one attachments are still

extracted. A file over a parse limit throws TikaMemoryLimitException instead of

falling back to the legacy text dump. OneNoteTreeWalkerOptions, the public

OneNoteLegacyDumpStrings settings and most public classes of the OneNote structure

model are removed (TIKA-5057).

* FetcherManager.getFetcher and EmitterManager.getEmitter return a Lease

(AbstractComponentManager.getComponent is now acquire); close it when done with

the component and anything it returned. Pipes now closes a

Closeable fetcher or emitter when a runtime save or delete replaces it (after the

last task using it ends) and on graceful shutdown; nothing closed them before.

EmittingUnpackHandler takes the emitter, not the EmitterManager (TIKA-5001).

* tika-grpc refuses SaveFetcher with configStoreType memory (FAILED_PRECONDITION):

the forks never saw such a fetcher, so every fetch through it failed. Use file or

ignite (TIKA-5001).

* A mime-types XML file that defines the same type twice, directly or through an

alias, fails to load with "Duplicate definition of ..."; the second definition's

magic and globs were silently added to the first, so tightening a rule in one copy

had no effect. A custom-mimetypes.xml that repeats a type within itself now stops

Tika from loading its types; merge the repeated blocks into one. A custom file may

still add globs or magic to a type defined in tika-mimetypes.xml. The eleven

duplicates in tika-mimetypes.xml are merged; detection is unchanged (TIKA-5029).

* 3MF packages are detected as model/3mf, their IANA type, instead of

application/vnd.ms-package.3dmanufacturing-3dmodel+xml, which is the content type of

the model part inside the package. The old name is an alias, so it still resolves as

input, but detection output and metadata now carry model/3mf (TIKA-5049).

* tika-app -T (--text-main) and the GUI's "Main content" view use Readability instead of

boilerpipe, and trim HTML pages only: other formats come out as full text, as with -t.

BoilerpipeContentHandler (tika-handler-boilerpipe) is deprecated (TIKA-5054).

SERVER, PIPES AND OPERATIONS

* A parse-context block (config file, preset, tika-server /config, gRPC

parse_context_json, pipes FetchEmitTuple) for a parser, detector, renderer or

translator that never reads one is logged as a warning instead of being silently

ignored; Tika 5.0 will refuse it. Parsers that read a block carry the new

@RequestConfigurable annotation, which a custom parser that reads its own block

should add; until then it still reads it, and a warning names the class (TIKA-5059).

* tika-server sets any field of the config's "server" block from the command line

(--serverSetting <field>=<value>, repeatable) and the environment (TIKA_SERVER_SETTING_<FIELD>,

e.g. TIKA_SERVER_SETTING_MAX_REQUEST_SIZE_BYTES), over the file's own value, so raising the

1 GiB request body limit no longer needs a config file. The 413 names the limit and

where to raise it (TIKA-5025).

* When pipes kills a forked server, it now kills the server's descendant processes too,

such as ExternalParser commands and their children; they kept running. On a timeout,

an interrupt or JVM shutdown, ProcessUtils.execute also kills the descendants of the

process it started (TIKA-5040).

* Fetchers and plugins built on HttpClientFactory (HTTP, Atlassian JWT, ES, OpenSearch,

Solr) no longer send Authorization or Cookie headers, or answer a 401 with the

configured password, after a redirect to another host, port or scheme. That includes

http to https, so use the https URL in the fetch key. Applies only with

maxRedirects > 0.

* tika-server no longer requires a "server" block in its config file; an absent block

means the server defaults, as "server": {} did. A tika-app config, or the documented

Docker example { "text-recognizers": [] }, now works as is.

* The tika-server full and tika-grpc Docker images no longer install

ttf-mscorefonts-installer, which downloaded Microsoft's fonts from SourceForge

at build time; non-embedded Verdana, Georgia, Trebuchet, Impact, Comic Sans and

Arial Black now render with Liberation fonts. The images add the IPA and

WenQuanYi fonts, so pages in non-embedded Japanese and Chinese fonts render

instead of showing empty boxes or nothing (TIKA-5005).

* The S3 fetcher, emitter and pipes iterator accept credentialsProvider "web_identity"

for EKS IAM roles for service accounts (IRSA), and "default" for the AWS SDK default

chain, which covers ECS task roles and EKS Pod Identity; the plugin now bundles the

AWS STS module these need (TIKA-5006).

* tika-grpc applies parse-context.timeout-limits and exception-reporting in the server

process too, so its backstop for a hung worker no longer falls back to the built-in

timeouts. A FetchAndParse call interrupted before the parse finished (server shutdown)

closes with UNAVAILABLE; a unary call used to fail with INTERNAL "no value received",

a server-streaming call completed with no reply, and a bidirectional stream went on

to throw on the closed call. Thanks to Kristian Rickert and Davide Polato (TIKA-5010).

* The GCS fetcher, emitter and pipes iterator no longer require projectId. When it is

unset, the client resolves a project from the environment, as it does for its

credentials (TIKA-5009).

* The Kafka emitter reports success only once Kafka has acknowledged every record

(per acks); a rejected or timed-out record fails the emit instead of being lost

silently, and the producer is flushed and closed on shutdown. (TIKA-5008).

* The JDBC emitter is thread-safe: concurrent emits (shared-server mode, several

async emitter threads) shared one connection and prepared statement unguarded,

mixing values across rows and committing each other's batches (TIKA-5007).

EXTRACTION AND FORMATS

* ASX playlists (application/x-ms-asx) are parsed by JSoupParser; XMLParser failed on

many real ones (mixed-case or unclosed tags, raw '&', non-UTF-8 without a declaration).

Texinfo files are parsed as text; they produced no content (TIKA-5029).

* Bare RPM headers (yum *.hdr files), pdf.js binary CMaps, Across Lite crosswords

(application/x-crossword), Commodore 64 tape images (application/x-commodore-tape),

OpenOffice.org 2.x/3.x registry cache files and LibreOffice user-profile backup

(user/pack) files are detected by content; they were application/octet-stream, or

application/x-git-pack for a pack file with two or three entries (TIKA-5029).

* ISO-2022-JP is decoded as x-windows-50221, so NEC/IBM extension characters (circled

numbers, Roman numerals) no longer become U+FFFD. ISO-8859-1, ISO-8859-9, TIS-620 and

ISO-8859-11 decode as windows-1252, windows-1254 and windows-874, as browsers do, when a

detector other than the default chain reports them (TIKA-5045).

* Binary files are no longer detected as text because of their name or a text header.

When content has no magic and at least 5% control bytes, and does not decode as

UTF-16 or UTF-32, a text type from the file name or Content-Type no longer wins: Terminal Reality

.pod archives, Mozilla startupCache bytecode named .js or .xul, CODESYS .meta files and

similar are now application/octet-stream. DefaultDetector no longer re-sniffs the first

512 bytes for text after MimeTypes found the 64 KB prefix binary, so a binary body

behind a short text header (Sass caches, BSB/KAP charts) is no longer text/plain;

zero fill after lines of text (a crashed or preallocated log) does not count as binary.

A UTF-8 BOM followed by binary magic (JPEG, PNG, GIF, ID3, zip, SQLite) is detected by

that magic instead of as text (TIKA-5029).

* Content with no magic is judged text or binary by TextBinaryClassifier, a small

model over byte statistics of the 64 KB prefix, instead of TextDetector's control-byte

check, which only accepted mostly-ASCII or valid UTF-8. Extensionless text in legacy

encodings (windows-1251, GB18030, Shift_JIS, EUC-JP, EUC-KR, KOI8-R and others), logs

with a few control bytes, zero-filled holes or backspace progress bars are now

text/plain; 3.x returned application/octet-stream for these, and 4.0-4.2 rescued

some only through a 512-byte re-sniff. Printable ASCII is always text; inputs under

16 bytes keep the old check. A text name or Content-Type still loses to content with

5% or more control bytes that the classifier calls binary (TIKA-5062).

* New or fixed magic: Emacs bytecode from every Emacs version (only version 19 matched

before, and *.elc is no longer also an octet-stream glob), Lua bytecode

(application/x-lua-bytecode), Qt .qm translations (application/x-qt-translation, were

text/html), SELinux policy packages (application/x-selinux-policy-package, were

text/x-pascal by *.pp), Performance Co-Pilot archives (application/x-pcp-archive), and

binary FrameMaker documents ("<MakerFile"), now application/vnd.framemaker instead of

the text MIF type application/vnd.mif (TIKA-5029).

* Markdown output keeps document code inside its code spans and blocks: a line of

backticks inside <pre> no longer closes the fenced code block, and a line break inside

inline code, alt text or a link destination, or a lone CR inside <pre>, no longer starts

a new markdown line, so none of them can turn the text after it into raw HTML. A

paragraph starting with "+ " stays a paragraph instead of becoming a list, and

elements nested more than 64 levels deep are flattened, so deep nesting can no longer

overflow the stack or make the output grow with the square of the depth (TIKA-5061).

* HTML main-content extraction: tika-readability is a jsoup-only Java port of Mozilla

Readability, and "readability-jsoup-filter" in the parse context runs it in JSoupParser

(off by default; HTML documents only, never message bodies or other HTML inside a

document, so .mht saved pages are not trimmed; tk:readability:* metadata). "jsoup-filters"

runs a list of jsoup filters, including your own JsoupFilter implementations, in order

(TIKA-5054).

* Container formats that were detected only as generic OLE2, OPC or zip are now detected by

their streams, package relationships or entries: Autodesk Revit projects and families,

3ds Max scenes, Ichitaro documents, Windows Thumbs.db caches and newer SolidWorks files

(OLE2); NuGet packages, FictionBook 3 e-books and Windows Hardware Lab Kit packages (OPC);

and Mozilla add-ons with an install.rdf or a Mozilla signature, which are now unpacked

as zip archives (TIKA-5029).

* Zip-based application packages are detected by their entries: MuseScore scores,

compressed MusicXML (by its score), zip-format CorelDRAW drawings, Power BI reports,

FreeCAD documents, SMART Notebook lessons, Anki decks, Fritzing sketches, Scratch 2 and 3

projects (by their project.json), LEGO Digital Designer models, App Inventor projects and

USDZ packages (model/vnd.usdz+zip). They are still unpacked as zip archives, and keep their

detected type in the output (TIKA-5029).

* MP4 and QuickTime files with a box over 2 GB, typically an mdat in front of

moov, keep their metadata: the walk skips such a box instead of stopping (TIKA-5051).

* OneNote FSSHTTPB sections extract each stored file once, not once per reference

to it (TIKA-5057).

* OneNote attachments and pictures carry their stored file name (resourceName) and,

for inserted files, the path they were inserted from (tk:orig-resource-name). .one

sections named none before, and an FSSHTTPB file's icon no longer takes the file's

name and ATTACHMENT type (TIKA-5057).

* OneNote note tag labels ("To Do", "Important", custom tags) are written before the

paragraphs they tag, as <p class="note-tag"> (TIKA-5057).

* OneNote pictures that are hyperlinks are written inside <a href>, so their URLs reach

link handlers (TIKA-5057).

* OneNoteParser parses .onetoc2 notebook tables of contents, listing their section

names (TIKA-5057).

* OneNote sections that fell back to the legacy text dump now parse structurally,

including OneNote 2007 sections, Aspose.Note files and sections with leftover bytes

from earlier saves. OneNote 2003 sections fall back to the dump instead of failing

(TIKA-5057).

* OneNote text that was dropped or garbled is extracted: objects edited in later

revisions, text behind element child references, and paragraphs with hyperlinks

(now <a href>). Extended ASCII is read as windows-1252, and NULs and U+FFFC

placeholders are removed (TIKA-5057).

* OneNote output adds pictures as <img> with alt text, OneNote's OCR text as

<div class="ocr">, and .one handwriting recognition as <span class="handwriting">

(TIKA-5057).

* Password-protected OneNote sections throw EncryptedDocumentException; FSSHTTPB

ones were never detected because their GUIDs were MD5-hashed. dc:creator includes

original authors. A file shorter than its header throws a TikaException

(TIKA-5057).

* OneNote parsing bounds CPU and memory on crafted files with limits scaled to file

size, and rejects more malformed structures up front (TIKA-5057).

* OneNote .one sections allocate far less and small ones parse much faster, reading

through a small page cache instead of one system call per field. Large

multi-revision sections can be slower, as every committed revision is now read

(TIKA-5057).

* OneNote legacy text dump: strings crossing a 1 MB read boundary are written once,

and strings are judged by the text-quality detector (tika-ml-junkdetect) when it is

on the class path (TIKA-5057).

* tika-ml-junkdetect: JunkDetector has a public no-arg constructor, so ServiceLoader

finds it on the class path (TIKA-5057).

* WOFF and WOFF2 fonts (font/woff, font/woff2), SquashFS images

(application/vnd.squashfs), Adobe Swatch Exchange files, PalmDOC e-books and binary

OpenPGP signatures (application/pgp-signature) are detected by content; they were

application/octet-stream. Together they are about a sixth of the Common Crawl files

Tika could not identify. MXF and Java serialization streams are detected by content

too (they had globs only), EOT fonts of version 0x00010000 and 0x00020001 are

detected (only 0x00020002 was), and torrent files whose first key is not "announce"

are detected (TIKA-5029).

* US FEC electronic filings (application/x-fec, a kind of text/plain, so their text is

extracted), gettext message catalogs, Garmin MapSource databases, Blizzard MPQ

archives and Warcraft III maps and replays, VCDIFF and Xdelta patches and Paint Shop

Pro images are detected by content; they were application/octet-stream. SID tunes

in RSID format are detected, and PSID needs a valid version after the magic

(TIKA-5029).

* HTML that opens with a fragment instead of <html> (a <meta>, <script>, <p>, <br>,

<table> or similar tag, as in pages that start with PHP warnings) is detected as

text/html, and Windows Media playlists without an XML declaration are detected as

application/x-ms-asx; they were application/octet-stream or text/plain. A PDF or

XHTML document with such a fragment in front keeps its type (TIKA-5029).

* A new test fails the build on tika-mimetypes.xml rules that load but can never

match as written (TIKA-5029).

* Elasto Mania levels, binary OpenPGP keys, StuffIt (SIT!) archives, MPEG transport

streams (video/mp2t, now also *.m2ts and *.m2t), Symbian 9 installers, FLC and FLI

animations, Microsoft Reader e-books, lrzip archives, Glulx games, Scream Tracker 3

modules and Monkey's Audio files are detected by content, and so are MARC records

with a blank bibliographic level; they were application/octet-stream (TIKA-5029).

* NoteWorthy Composer songs, Finale (Enigma binary) scores, ReCycle REX2 loops, i-mode

MFi melodies, TrackMania/ManiaPlanet GameBox files, PlayStation 3 packages and themes,

Trainz content packages, ZX Spectrum RZX recordings, Quake and Half-Life sprites,

NuFX (ShrinkIt) and Amiga LZX archives, MikroTik RouterOS packages and ClamAV

signature databases and updates, and Casio FX-Interface files are detected by

content; they were application/octet-stream (TIKA-5029).

* DER certificate revocation lists (application/pkix-crl), DER X.509 certificates

signed with any algorithm (only some RSA, DSA and ECDSA ones were matched, so SM2

and Ed25519 certificates were missed) and RFC 3161 time-stamp replies are detected

by content; they were application/octet-stream (TIKA-5029).

* glTF binary models, Blender scenes, Autodesk FBX (binary), Jw_cad and FastCAD drawings,

Valve Source maps, Unreal Tournament compressed packages, SoundFont 2 banks, zsync

control files and PhysioNet WFDB annotations are detected by content; they were

application/octet-stream in a recent Common Crawl sample (TIKA-5029).

* NEXRAD Level III radar products, UCSC bigBed, bigWig and 2bit files, uncompressed BAM

and BAM indexes, OpenStreetMap PBF, LIGO/Virgo frame files, QEMU qcow disks, ImageDisk

floppy images, MAME CHD, devicetree blobs and FIT images, U-Boot legacy images, UBI

images, AmigaOS executables, DirectDraw Surface images (image/vnd.ms-dds), YM chiptunes,

VST presets and older Jw_cad drawings are detected by content; they were

application/octet-stream in a recent Common Crawl sample (TIKA-5029).

* HP printer jobs are detected by content: PJL jobs by the language they enter (PCL,

PCL XL or PostScript), and raw PCL5 and PCL XL streams. So are IBM MO:DCA (AFP)

documents, Windows minidumps and kernel dumps, StarView metafiles, Lotus 1-2-3 files

with format version 0x1005, Promethean ActivInspire flipcharts, Casio calculator

(g1m family) files, Haihaisoft DRM-X protected files and Navitel maps; they were

application/octet-stream (TIKA-5029).

* gcov coverage data and notes, Python bytecode and pickles, Apache Lucene index files,

git packfiles, ACPI tables, TZif time zone files (application/tzif), Firefox mozLz4

files and JPEG XR images (image/jxr) are detected by content; they were

application/octet-stream (TIKA-5029).

* More variants are detected by content: OpenPGP keys and signatures in new-format

packets (as written for RFC 9580 v6 keys), v1 DER certificate revocation lists (no

version field), gettext catalogs with minor revision 1 and VCDIFF deltas in

open-vcdiff's SDCH format; they were application/octet-stream (TIKA-5029).

* EpubParser resolves manifest hrefs as URI references relative to the OPF: a '+' is

no longer read as a space, and '../' is followed, so such chapters and images are no

longer skipped. encryption.xml URIs are resolved the same way. The new

ZipEntryPaths in tika-parser-zip-commons does the resolving (TIKA-5050).

* Canon CR3 raws have a parser of their own, Cr3Parser in the image module, instead

of going through the MP4 parser: their Exif data (IFD0, Exif IFD, Canon maker note,

GPS) is extracted, the JPEG preview (PRVW, 1620x1080 on current bodies) is the

THUMBNAIL and the 160x120 THMB thumbnail is INLINE, so the thumbnail preset serves

a preview (TIKA-5042).

* Blender files (.blend, .blender, application/x-blender, new) yield the preview Blender stores

as the THUMBNAIL embedded document, for every file format version including the one

of Blender 5.0. A gzip- or zstd-compressed file is detected by its content and read

the same way; zstd needs zstd-jni on the classpath (TIKA-5036).

* RAR5 archives are parsed: RarParser no longer refuses them, junrar has read RAR5

since 8.0.0 (TIKA-5043).

* SketchUp models (application/vnd.sketchup.skp, new, detected by their header) yield

the preview SketchUp stores as the THUMBNAIL embedded document, from SketchUp 3 to

2026 (TIKA-5036).

* Jupyter notebooks (.ipynb, new type application/x-ipynb+json) are parsed: cell

sources and plain-text outputs as text, kernel, language and format version as

jupyter:kernel, jupyter:language and jupyter:nbformat, and the image outputs as

embedded documents, as are the images markdown cells carry as attachments. The

thumbnail is picked as nbsphinx picks it: the output a cell marked

nbsphinx-thumbnail names, else the last image in the notebook, given as PNG or

JPEG where the output also holds them as SVG (TIKA-5037).

* FictionBook marks the cover image named in <coverpage> as a THUMBNAIL embedded

resource, so the thumbnail preset serves it (TIKA-5035).

* MP4Parser extracts text subtitle tracks (tx3g/mov_text, QuickTime text) as

embedded text/vtt documents named track-<id>.vtt, synthesized from the samples

and parsed by SubtitleParser, with the track language as dc:language (ISO 639-2;

QuickTime Mac language codes are mapped); chapter tracks are skipped. Samples are

found through the sample tables, and through moof/traf/trun in fragmented files,

and read by seeking. This needs a file, content already in memory (a byte array

or a digest-drained stream), or an archive entry up to 1 MB (more within a cache

memory budget); other inputs are silently not searched. Off with extractSubtitles

on mp4-parser (TIKA-5030).

* New MatroskaParser for Matroska and WebM, which previously had no parser: it

extracts SubRip, ASS/SSA and WebVTT text subtitle tracks the same way, with the

track name as dc:title and the track language (BCP 47 or ISO 639-2; "eng" when

the track names none, as the format specifies). It stops at the first Cluster

when there is no text track; otherwise it reads every block header, seeking past

other tracks' blocks in a file and reading through them in a stream. No container

metadata yet. Off with extractSubtitles on matroska-parser (TIKA-5030).

* MP4Parser extracts CEA-608 closed captions (CC1-CC4) as embedded text/vtt

documents: from QuickTime c608 tracks by default (off with extractClosedCaptions),

and from H.264/HEVC SEI with videoCaptionScan PROBE or FULL. Caption reads have

their own per-file limits (TIKA-5039).

* Parse Mobipocket and Kindle ebooks (.mobi, .prc, .azw, .azw3): EXTH metadata, the

cover as the THUMBNAIL embedded document, the other images, and the text, stored,

PalmDOC- or HUFF/CDIC-compressed, through the HTML parser (of a KF8 book the HTML flow

only). A hybrid file yields its version 6 book; extractKf8 emits the KF8 copy as an

embedded document too. An encrypted book yields its metadata and images, sets

tika:encrypted and records the locked text as a warning. New keys mobi:version,

mobi:compression, mobi:encryption, mobi:cde-type, mobi:hybrid. maxTextLength caps the

text (16 MB, and 32 times the file size), maxFileSize the bytes read (256 MB).

application/vnd.amazon.mobi8-ebook (*.azw3) is new, and with

application/vnd.amazon.ebook a sub-type of application/x-mobipocket-ebook: a .azw file

is now detected as application/vnd.amazon.ebook, not application/x-mobipocket-ebook,

and a .prc or .azw that is not a Mobipocket book yields nothing and a warning

(TIKA-5026).

* Zip-based formats that carry a preview image have it emitted as the THUMBNAIL, so

the thumbnail preset serves it: comic book archives (.cbz, new type

application/vnd.comicbook+zip, the first page in name order), Krita (.kra,

preview.png), OpenRaster (.ora, Thumbnails/thumbnail.png), FreeCAD (.FCStd,

thumbnails/Thumbnail.png), Procreate (.procreate, QuickLook/Thumbnail.png) and

Sketch (.sketch, previews/preview.png). All six are new types and sub-types of

application/zip (TIKA-5032).

* PSD files emit the JPEG thumbnail Photoshop stores in image resource 0x040C as a

THUMBNAIL embedded resource, so the thumbnail preset serves it; one over

maxDataLengthBytes is skipped with a warning (TIKA-5033).

* 3MF packages are parsed as zip packages instead of failing as XML: the type is a

sub-type of application/zip, and the streaming zip detector recognizes it (TIKA-5031).

* 3MF packages have the image their package thumbnail relationship names emitted as the

THUMBNAIL, usually Metadata/thumbnail.png; a model part's own thumbnail is not taken

(TIKA-5032).

* New SubtitleParser for WebVTT (.vtt) and SubRip (.srt). Each cue becomes

<div class="cue"> with data-start/data-end (HH:MM:SS.mmm) and data-id, its text

cleaned of markup; WebVTT NOTE blocks become <div class="note"> and a Language

header sets dc:language. Both were parsed as plain text before, timings and

markup included. Magic fixes: SubRip with CRLF line endings (typo in the rule);

WebVTT with a title or tab after "WEBVTT", with a UTF-8 BOM and a title, or in

UTF-16 (TIKA-5028).

* Detect Python scripts with /usr/bin/env python shebangs as text/x-python

instead of application/x-sh (TIKA-4408).

* Perl, Ruby, Node, PHP, Lua and Tcl scripts are detected by their shebang line

(#!/usr/bin/env ruby, #!/usr/local/bin/node, ...) instead of as application/x-sh.

Magic values with "\ " or "\t" now match a space or tab; the decoder wrote a NUL

byte instead, so about 25 rules never matched, among them Word 6 and Excel 5 at

offset 2080, BibTeX, KDE desktop entries and "#! /bin/sh". Of those, the text/x-diff

rules "*** " and "Only in " and the message/rfc822 rules "Forward to" and "Pipe to"

are dropped, and "diff " needs a "--- " or "*** " line within 1 KB (TIKA-4408).

* PPTX slides are extracted in presentation order (p:sldIdLst in presentation.xml)

instead of relationship-id order, which put rId10 before rId2; embedded-resource

page numbers follow the same order. Slides that are related but missing from the

list are extracted last and counted in msoffice:ppt:num-unlisted-slides (TIKA-5017).

* Superscript and subscript are kept as <sup>/<sub> in XHTML for docx (w:vertAlign,

directly or through a character style), pptx (a:rPr baseline), doc, RTF (\super,

\sub, \nosupersub) and HTML; they were dropped, so "x²" read "x2" and

footnote markers fused with the word before them. .ppt is not covered (TIKA-5016).

* VLM text recognizers (jina-ocr, OpenAI-compatible, Claude, Gemini) no longer corrupt

LaTeX math in model output: $...$, $$...$$, \(...\) and \[...\] are kept verbatim

instead of being read as Markdown, which dropped the backslash from \{, \} and \|

and split GFM table cells on |. Math is emitted as <span class="math inline"> or

<span class="math display">, and the Markdown writer outputs it unescaped (TIKA-5011).

* The OpenDocument parser writes repeated cells and rows (table:number-columns-repeated,

table:number-rows-repeated) once per repeat; a cell was written once with that

colspan and a row once, so values repeated across columns or rows were lost. As in

the Excel parsers, empty rows are dropped and empty cells are written only to pad

out to the next value in their row. Past 10 million characters and elements of

padding and repeated content per document, the rest are written once and padding

is dropped, with a tk:exception:warn entry (TIKA-5014).

* The OpenDocument parsers (open-document-parser, flat-open-document-parser) take

includeMissingRows, as the Excel parsers do: when true, each empty spreadsheet row

before a row with content is written as an empty row, up to 100,000 per document.

The default is false (TIKA-5014).

* VLM text recognizers keep superscripts and subscripts: inline <sup>/<sub> in model

output (x², H₂O, footnote markers) reach the XHTML as sup/sub

instead of plain text ("x2"), and the Markdown writer outputs them as inline HTML.

* Charset detection recognizes ISO-8859-5 (Cyrillic), ISO-8859-6 (Arabic) and ISO-8859-7

(Greek) without a declaration; they were reported as windows-1251/1256/1253, x-EUC-TW

or windows-1252, which garbled the text. When candidates decode identically, the

report is the charset most frequent on legacy-charset web pages, e.g. windows-1252

over ISO-8859-2 for Western text (TIKA-5019).

* Charset detection treats a byte-order mark as a strong hint, not a declaration: it

is honored only when the bytes after it decode in that charset, and a rejected BOM

is dropped rather than left in the text as junk characters (TIKA-5019).

* An HTML meta charset naming UTF-16 (or UTF-32) is read as UTF-8, as the WHATWG

prescan does: a meta readable as ASCII means the bytes are not UTF-16. A page with a

UTF-8 BOM and a UTF-16BE meta was decoded as UTF-16BE, turning its text into CJK

characters (TIKA-5019).

* Charset detection reads UTF-8 with sparse corruption (up to one bad sequence per

ten valid ones, existing U+FFFD not counted) as UTF-8 (TIKA-5019).

* Charset detection reports the windows-125x sibling of an ISO-8859 charset (e.g.

windows-1250 for ISO-8859-2) when the page's 0x80-0x9F bytes are punctuation that

only the windows charset prints (… “ ” €) and no letters decode differently, even

when the page declares the ISO charset (TIKA-5027).

* Charset detection uses a hashed byte n-gram model in place of the byte-bigram Naive

Bayes model, with UTF-16 and UTF-32 detected by the same model instead of a separate

UTF-16 detector. BOM-less UTF-16 with an emoji is no longer misread, UTF-32 is no longer

reported as UTF-16, a surrogate pair cut at the end of the sample no longer rules UTF-16

out, and short BOM-less UTF-16 Thai, Cyrillic or Indic text is no longer read as

windows-1252. Input the model calls binary gets the low-confidence windows-1252

fallback, as undetectable input did before. Adds IBM437 as a detected charset

(TIKA-5052, TIKA-5048).

* The charset model no longer calls valid UTF-8 binary when it has no NUL and no control

character other than tab, line feed, form feed, carriage return and escape (a character

cut at the end of the sample is allowed); its next most likely charset is used. Short

CJK, Greek and Thai samples were often called binary (TIKA-5052).

* Charset detection reads BOM-less UTF-16 with trailing NUL padding, U+0000 separators,

stray control code units or fully vocalized Arabic as UTF-16, not binary; Spanish,

Portuguese and Catalan text with ¡ ¿ º ª as windows-1252, not ISO-8859-3; Kurmanji

as windows-1254; and text with a few stray NUL bytes as text (TIKA-5052).

* Charset arbitration no longer defers to a declared charset whose decode has more

replacement characters than the best-scoring decode, such as a page declared

iso-8859-3 but written in windows-1252 (TIKA-5027).

* Charset detection keeps x-MacRoman instead of relabelling it windows-1252 when the

text has MacRoman curly quotes (TIKA-5052).

* A UTF-32 guess from the statistical model is dropped when the bytes are not valid

UTF-32 code points: binary with a few NULs in its header (a shader file of ASCII 0/1

after an 80 37 00 00 header) was reported as UTF-32BE (TIKA-5052).

* Charset detection no longer reads ASCII text with a few control characters (logs with

escape or form-feed bytes, IRC colour codes, record separators, banners) as BOM-less

UTF-16: the model was retrained with such text, and when it still returns UTF-16 or

UTF-32 as its only candidate, the junk filter now compares that decode against

windows-1252 and keeps whichever reads as text (TIKA-5052).

* The junk model that arbitrates between charset candidates was retrained with UTF-16

misreads of every script (text read as UTF-16, and UTF-16 text read as a single-byte

charset) and with fully vocalized Arabic, so a BOM-less UTF-16 decode of vocalized

Arabic is no longer scored below its windows-1252 misread (TIKA-5027).

* XHTML output no longer contains empty or whitespace-only b/strong/i/em/sup/sub

elements, and whitespace at their edges is written outside them (<b>Note: </b>

becomes <b>Note:</b> ); u and s are unchanged. The Markdown writer does the same for

**, * and ~~, which it wrote as literal asterisks for <b>Note: </b> (TIKA-5018).

* The junk detector, which arbitrates between candidate charsets, decodes numeric

character references in HTML as browsers do: &#128;-&#159; are the windows-1252

characters (&#150; is an en dash, not a C1 control), and &#0;, surrogates and values

past U+10FFFF are U+FFFD. One pass, so &#38;#x41; stays &#x41; (TIKA-5044).

* Charset arbitration uses a new junk model: hashed codepoint n-grams trained on pairs of a

correct and a wrong decode of the same bytes, including the candidates the detector

chain really produces. The CJK family gate and the Latin letter gate are gone; when

the model's margin is thin, the declared or structural charset wins. tika-eval's

languageness comes from the same model and gains tika-eval:languagenessMin (the worst

200-character window, which finds a damaged region in a clean document) and

tika-eval:languagenessCoverage. Languageness is -99 (unknown) for text with no letters

or with under 90% of its letters in scripts the model was trained on (TIKA-5027).

* Big5 text decodes as browsers do: Big5-HKSCS with the 144 byte pairs where the WHATWG

Encoding Standard differs patched to the WHATWG value. A3E1 is the euro sign instead of

U+FFFD; A145, A1E3 and A1FE are the CP950 forms U+2027, U+FF5E and U+FF0F; six pairs in

C6CF-C6DF are radicals and ditto marks, not kana. Detected-charset metadata is unchanged;

tk:decoded-charset is x-Big5-HKSCS-WHATWG, which Charset.forName does not resolve

(TIKA-5052).

* EUC-JP text decodes as browsers do: x-eucJP-Open with nine pairs patched to the WHATWG

value, the Microsoft forms windows-31j already gives for Shift_JIS (A1C1 wave dash is

U+FF5E, A1DD minus U+FF0D, A1F1/A1F2/A2CC cent, pound and not signs U+FFE0-FFE2, A1BD

U+2015, A1C2 U+2225, FCFB/FCFC U+FFE2/U+FFE4). tk:decoded-charset is

x-eucJP-Open-WHATWG (TIKA-5052).

* Charset detection finds non-ASCII text past the first 16 KB, as in an email with a

long ASCII header block, a log or a CSV: those were reported as windows-1252. When the

leading sample is not conclusive, detection reads on (up to 16 MB of a file or byte

array, 1 MB of a stream) and adds the bytes around each non-ASCII byte and each

ISO-2022, HZ or UTF-7 sequence to the sample, stopping at 50 non-ASCII bytes or 16

UTF-8 characters. Fewer than 8 scattered non-ASCII bytes leave the sample as it was.

The junk filter scores the same sample. EncodingProbeCache keys on a probe kind

(TIKA-5046).

OTHER CHANGES

* The file-to-markdown-docker agent skill turns OCR off with an empty text-recognizers

list and a server block (its tesseract-ocr-parser config failed to load since 4.2.0),

and checks tk:parsed-by-full-set, not pdf:ocr-page-count, to confirm OCR ran. A test

loads every config in the user skills in tika-app and tika-server.

* A runtime fetcher or emitter save or delete reaches running pipes workers at

their next fetch, through a change token on file and ignite stores

(ConfigStore.changeToken); before, a worker kept the instance it first built

until it restarted (TIKA-5001).

* PEM-armored CMS/PKCS7 (-----BEGIN PKCS7----- or -----BEGIN CMS-----) is parsed like

DER: Pkcs7Parser and the opt-in pkcs-7-detector report the smime-type and extract

signed content; -----BEGIN CMS----- is now detected (TIKA-5002).

* Markdown output writes a line break inside a table cell as <br>, so the row no

longer splits, and no longer prefixes the text after a line break or at the start

of a paragraph with &#32; (TIKA-5012).

* Markdown output writes a table with merged cells (colspan/rowspan) as an HTML table

instead of a pipe table, which dropped the spans. A nested table is still flattened

into its cell, now with <br> between its rows and a space between its cells; its text

was fused with the cell's before (TIKA-5015).

* A PDF with pages that pages.ocr.maxPages kept from the OCR engine records

skipped:max-pages in tk:ocr-warnings; before, the cap left no trace and

pdf:ocr-page-count still counted those pages (TIKA-5013).

* tika-eval Compare reports text that moved between the two extracts (a reordered

slide, a title written at the end of a page instead of the start): order_* columns

in content_comparisons and the moved_text and moved_text_by_mime reports. The

maxTokens and maxContentLength settings in a tika-eval config are now applied; they

were ignored, and the maxContentLength default is now the 10,000,000 that was in

effect (TIKA-5022).

Release 4.2.0 - 10/25/2026

BREAKING CHANGES AND DEPRECATIONS

* Unpack output is a Frictionless Data Package by default (tika-server /unpack,

tika-app -z/-Z, pipes UNPACK); unpack-config.outputFormat=REGULAR

(--unpack-format=REGULAR) restores the old layout (TIKA-4681). A document without

attachments is still a package, so /unpack returns 200, not 204;

unpack-config.emptyPackage=SKIP restores the 204 (TIKA-4971).

* The package's metadata.json is now rmeta.json and gets the metadata filters; a

package sent to the container's emitter replaces its <emitKey>.json. REGULAR

.metadata.json sidecars are the file's row from the metadata list.

FrictionlessResource.name() is the resource name (the filename moved to

resourceName()); FrictionlessResource.create and DataPackage.addResource are removed

(TIKA-4971).

* Config is untrusted unless the operator says so. @OperatorOnly properties (paths,

executables, class names, URLs, credentials, operator-sized limits) are refused

wherever a request builds a component, and an @OperatorOnly class can't be built

from a request at all. Java code setting one through a JsonConfig wraps it in

JsonConfig.trusted(json); JsonConfig constructors now need tika-serialization.

Newly locked: pdf-parser imageGraphicsEngineFactoryClass and maxMainMemoryBytes,

raw-tiff maxTotalPreviewBytes, VLM completionsPath. The RuntimeConfig subclasses

are removed (TIKA-4965, TIKA-4974).

* Per-request limits and timeouts may tighten the operator's, never loosen them.

pipes.trustedRequests=true marks requests as the operator's own (no clamp, may set

@OperatorOnly properties); PipesForkParser sets it, and tika-server and tika-grpc

refuse to start with it (TIKA-4978, TIKA-4985).

* A duplicate key in configuration JSON is an error, not last-wins. A request's

http-fetcher config may set only httpRequestHeaders; a file-system emitter's runtime

config naming basePath fails as an unknown key. forbidden-apis bans JsonConfig.json()

outside ConfigDeserializer and PluginJson, also in projects inheriting tika-parent

(point json.forbiddenapis.signaturesFile at an empty file to opt out).

ExtensionConfig no longer implements JsonConfig (TIKA-4975).

* Dates: Metadata.getDate() reads zone-less values as UTC, not the JVM's zone.

Archive-entry and mail-attachment file times move from dcterms:* to

fs:created/fs:modified. DateNormalizingMetadataFilter honors offsets, normalizes

every value and completes partial dates (2019 -> 2019-01-01T12:00:00Z).

MailDateParser is replaced by TikaDates; MailUtil moved to tika-parser-mail-module,

leaving tika-parser-mail-commons a deprecated shim, to be removed in 4.3.0

(TIKA-4917).

* OCR engines (tesseract-ocr-parser, tess4j-parser) under "parsers", deprecated in

4.1.0, now fail config load: configure them under "engines" and name them in

"text-recognizers". A VLM parser may stay under "parsers" as a document parser

(ContentEnricher.parsesDocuments()). openai-embedding-filter, jina-embedding-filter

and openai-image-embedding-parser, deprecated in 4.1.0, are removed; use an engine

and an inference binding (TIKA-4985).

* text-recognizers: an inline entry must have exactly one key, setting both

_min-width and min-width is an error, and gated entries load as GatedEnricher, not

SizeGatedEnricher (TIKA-4972).

* tika-server transfer limits: maxRequestReadMillis (408) and maxResponseWriteMillis,

both 5 min; idleTimeoutMillis 120s (was CXF's fixed 200s); maxInFlightRequestBytes

(429). Raise the deadlines for very large or slow transfers; cut-offs are logged at

INFO by the TransferLimitLog logger. Cleartext HTTP/2 (h2c), on by default since

4.0.0, now needs allowHttp2 (TIKA-4991).

* tika-server TLS: a tlsConfig block turns TLS on; "active" is deprecated (true

ignored with a warning, false refused, removed in 4.3.0), and

TikaServerConfig.getTlsConfig() is null without a block. Protocol and cipher suite

policy is the JVM's (jdk.tls.server.protocols, jdk.tls.server.cipherSuites,

jdk.tls.disabledAlgorithms): includedProtocols, excludedProtocols,

includedCipherSuites and excludedCipherSuites, none of which worked as documented,

are refused at startup. clientAuthenticationWanted is removed (true refused, false

ignored with a warning), as tika-server never reads client certificates

(TIKA-4994).

* embedded-limits maxDepth defaults to 20 (was unlimited) and replaces output-limits

maxPackageEntryDepth, which is to be removed in 4.3.0 with its accessors on

OutputLimits and SecureContentHandler (#3319).

* Pipes plugins load only from the zips in plugin-roots; a plugin on the classpath

needs -Dtika.plugins.classpath=true, which PipesForkParser sets when it finds no

plugins directory holding tika-pipes-file-system (TIKA-4918, TIKA-4985). A

plugin-roots naming a directory without the plugin zips, which the classpath used

to mask, now fails with "Unknown type: ... Available: []"; tika-app and

tika-server treat the docs' PLUGINS_PATHS placeholder as unset and use their

bundled plugins directory (TIKA-4984).

* image/vnd.microsoft.icon moves from ImageParser to the new ICOParser, which also

parses cursors (TIKA-4937); animated cursors (RIFF ACON) are

application/x-navi-animation, even when named *.cur (TIKA-4985).

* ISArchiveParser(String location) is removed; pass a file-backed TikaInputStream

(TikaInputStream.get(Path)) (TIKA-4986).

* tika-bundle-standard no longer embeds dependencies that are OSGi bundles

themselves (commons-*, pdfbox, fontbox, bouncycastle, jsoup, asm, xz, xmpcore,

dd-plist); deploy them alongside it (TIKA-4934). Its imports are now mandatory, it

registers its encoding detector, language detector and metadata filter providers,

and it requires Java 17 (TIKA-4955).

SECURITY

* The default zip-bomb ratio (output-limits.zipBombRatio) is 1000, up from 100. It

is measured across the whole embedded tree against the container's bytes, so nested

compression is still caught; at 100, ordinary compressed logs and text tripped it.

Set zipBombRatio to 100 to restore the old limit (TIKA-4985).

* The Gemini VLM parser sends its API key in x-goog-api-key, not in the URL that

error messages and logs quote (TIKA-4985).

* XML parsers are configured through Apache Commons Secure XML (TIKA-4935); Tika's

own entity-expansion limit is retired in favor of the JDK's (TIKA-4940); and

forbidden-apis rejects building XML parsers outside XMLReaderUtils (TIKA-4939).

New tk:xml-external-reference and -count list the external DTDs and entities a

document referenced, which Tika never resolves (TIKA-4941). XFA is read with SAX;

XMLReaderUtils.getXMLInputFactory() is deprecated (TIKA-4938).

* The tika-app GUI's Formatted Text view no longer lets a document make Swing

construct classes or fetch URLs: only plain elements and attributes reach it, and

remote images are not loaded (TIKA-4992).

* tika-server TLS: a keystore configured without "active": true no longer serves plain

HTTP. 2-way TLS accepts client certificates issued by a CA in the truststore, which

it had rejected since 2.4.0; their revocation is checked through the certificate's

OCSP responder or CRL distribution point and fails closed (tlsConfig

checkRevocation=false turns it off). Rejected client certificates are logged at INFO

with the reason (TIKA-4994).

SERVER, PIPES AND OPERATIONS

* The file-system emitter can gzip its output ("compression": "GZIP"); tika-eval reads

.json.gz extracts and measures them uncompressed (TIKA-4989).

* tika-server activates catalog presets with --preset <name> (repeatable) or

TIKA_PRESETS (comma-separated), overriding the config file's entry (TIKA-4988).

* PipesParser.start() and PipesForkParser.start() bring the forks up before the

first parse (TIKA-4953), and tika-server starts its forks before opening its port,

so a fork that can't start fails startup, not the first request (TIKA-4983).

* PipesForkParser: a ParseMode, ContentHandlerFactory or EmbeddedLimits in the

per-call ParseContext overrides PipesForkParserConfig, and the caller's context is

no longer modified (TIKA-4981).

* Forked pipes servers get their config and auth token over stdin, never through the

temp dir; server-manager and PipesServer signatures change (TIKA-4952).

* maxFilesProcessedPerProcess defaults to 100,000, up from 10,000 (TIKA-4950).

* Pipes forks inherit the parent's default locale unless forkedJvmArgs set

user.language (TIKA-4920).

* Pipes UNPACK deletes its temp dir when the fetch or parse fails (TIKA-4985).

* Parser loading and fork startup are about 1.3s faster: parsers no longer build a

default EncodingDetector the loader then replaces (TIKA-4954).

* Quieter pipes logging: per-parse timing lines move to TRACE, lifecycle lines to

DEBUG (TIKA-4949).

* OSGi: tika-serialization is a bundle, so JsonConfig constructors work (TIKA-4982);

tika-core installs without a Service Loader Mediator (TIKA-4945); the

DefaultParser and DefaultDetector services no longer recurse into a

StackOverflowError (TIKA-4942); OOXML, HTTP and GeoGebra parsing and charset

detection work in tika-bundle-standard again (TIKA-4955).

EXTRACTION AND FORMATS

* docx: OMML equations are wrapped in <span class="omml"> (TIKA-4968); REF, PAGEREF,

NOTEREF, HYPERLINK \l and simple fields link to their bookmarks (TIKA-4967);

bold and italics from character styles are applied, with opt-in

includeCharacterStyleClasses (TIKA-4966).

* .doc: stored line and paragraph counts are extracted (TIKA-4694).

* ExecutableParser emits a PE file's icon (its first icon group) as an embedded

THUMBNAIL .ico; extractAllIcons=true emits every group, extractIcons=false none

(TIKA-4936).

* BigTIFF is read through metadata-extractor 2.22.0: EXIF and DNG tags from BigTIFF

DNGs, plain BigTIFF no longer fails, and little-endian BigTIFF is detected by

content (TIKA-4987).

* Previews from Olympus ORF, Panasonic RW2, Epson ERF, Kodak KDC and Leaf MOS; Epson

ERF is detected by content. A metadata-extractor tag descriptor that throws skips

the tag instead of failing the parse (TIKA-4980).

* OneNote: table and list structure in FSSHTTPB files (TIKA-4943); section and page

GUID metadata (TIKA-4944).

* OneNote FSSHTTPB parsing is ~6% faster with ~20% fewer GCs: binary items are read

in place instead of copied, bit reads assemble whole bytes, and GUIDs are cached.

Compat: BinaryItem.content (a List<Byte>) is replaced by getContent()/setContent()

on byte[]; a binary item whose length decodes as 2^63 or more now fails the

FSSHTTPB parse (TIKA-5055).

* Detection: *.rs is text/x-rust (TIKA-4977); a MIME email nesting

multipart/related is message/rfc822, not MHTML (TIKA-4973); XML with an element

like <htmlPath> is no longer HTML (TIKA-4962); XUL is detected by content and

parsed by the HTML parser, as XMLParser fails on its DTD entities (TIKA-4985);

OS/2 bitmap array and TGA detection is tighter (TIKA-4937).

* Image checks (byte-size, min-image-size, blank-image-check; new

tika-image-processing module) keep blank and tiny images from text recognizers;

SizeGatedEnricher is deprecated (TIKA-4972).

* VLM parsers: output gets tk:ocr-warnings (an empty Claude response is a warning, not

an exception), vlm:skipped-reason and vlm:image-pixels; they take temperature and

retry dropped HTTP connections (TIKA-4972). The OpenAI-compatible parser has opt-in

useMaxCompletionTokens for models that reject max_tokens (#3350).

* The embed task writes tk:chunks once per binding, not once per image or page

(TIKA-4970).

* Opt-in "metadata-list": {"typedValues": true} writes numbers, booleans and

tk:chunks as JSON values rather than strings; new Property.ValueType.JSON

(TIKA-4969).

* Parsers no longer carry one document's state into the next, or share it between

concurrent parses: IDML page counts, an ENVI header's multi-line field (#3406), an

IPTC/ANPA wire's agency format and publisher (TIKA-4995), and ISArchiveParser's

ISA-Tab directory and study file name (TIKA-4986).

OTHER CHANGES

* tika-app -J writes the metadata list when the container throws, with the exception

in tk:exception:container-exception on the first record, as /rmeta does; it used to

print nothing. The exit status still reports the failure (TIKA-4985).

* A RAR entry junrar fails to extract is recorded as

tk:exception:embedded-stream-exception on the container; what was extracted before

the failure is kept and the rest of the archive is parsed (junrar 8.1.2, TIKA-4985).

* JSoupParser emits CDATA text again (TIKA-4963).

* OutlookExtractor falls back to the Date: header when a .msg has no MAPI date

(TIKA-4959).

* The MP4 parser no longer warns of an EOFException on every file (TIKA-4957).

* GeoGebraParser no longer needs jackson-databind (TIKA-4956).

* tika-server's welcome page lists endpoints inherited from superclasses (XMP on

/meta) under their real paths (TIKA-4991).

Release 4.1.0 - 9/26/2026

HIGHLIGHTS

* OCR and enrichment engines are selected by name in a new

"text-recognizers" list ("[]" turns enrichment off; no list resolves one

engine per media type at startup); the image and PDF parsers invoke them

rather than the composite dispatching to them. The image/ocr-*

pseudo-types are retired; a third-party engine still advertising them is

a legacy text recognizer with a WARN. VLM parsers gain "textRecognizer"

(TIKA-4872, TIKA-4884).

* New "exception-reporting" parse-context config redacts and bounds

exception text in metadata, tika-server error bodies and pipes/grpc

messages; FileSystemEmitter writes atomically. Compat: tk:exception:*

values may now begin with the TikaException wrapper line (TIKA-4848).

PERFORMANCE

* Detection (magic ~35% faster on unmatched input, override

keys honored before magic, cached type->parser map), markdown output

~30x faster, .doc cleanup, CSV sniffing, zip legacy-method rewinds, pipes

ACK overlap, raw UTF-8 content passback for tika-server

(content-bytes-config). Compat: with a Content-Type override set,

DefaultDetector no longer lets a more specific magic result overrule it

(TIKA-4868).

* PDF incremental-update scanning reads in blocks; ~4x less CPU (TIKA-4898).

* Embedded documents in Office files, PDFs, PST and the zip-family

containers are re-opened from their container on rewind instead of

cached (TIKA-4878).

* Improve spooling/decrease number of spills to disk. The pipes cache

memory budget defaults to a quarter of the fork heap

(-Dtika.pipes.cacheMemoryBudgetBytes overrides) (TIKA-4835).

* Digesting embedded documents no longer spools each to a temp file; a

process-wide CacheMemoryBudget keeps them in memory. Zip-family parsing

uses seekable channels, so hasFile() may be false afterward (TIKA-4828).

BREAKING CHANGES AND DEPRECATIONS

* Pipes IPC carries inline bytes as a raw binary field, not in the tuple;

4.0.0 tuples with an "inline-bytes" parse-context entry are rejected

(TIKA-4829).

* Pipes plugins no longer bundle Jackson; plugin config is parsed by a

shared strict mapper that rejects unknown and duplicate keys (TIKA-4840).

* OpenAIVLMParser no longer auto-registers via SPI; per-request config for

the embedding filters works and locks baseUrl/apiKey/model; inline PDF

page OCR accumulates tk:chunks from every page (TIKA-4871).

* Temp files follow -Djava.io.tmpdir on the parent JVM (Tika, its

libraries and forks); pipes.tempDirectory is deprecated. A fork whose

parent dies deletes its own temp dir; failure-path temp leaks fixed

(TIKA-4877).

* Kafka pipes iterator no longer stops on the first empty poll

(assignmentTimeoutMs, drainIdleMs); groupInitialRebalanceDelayMs is

deprecated (TIKA-4833).

INFERENCE (EXPERIMENTAL)

* tika-inference and tika-vlm are still experimental: classes, config keys and

output may change in minor releases. Stable: the "engines" map, the

"text-recognizers" list and pdf-parser "text" (TIKA-4884, TIKA-4895).

* Refactor OCR and inference to use engines for inference and

page rendering (TIKA-4889).

* Enable audio/video chunking with ffmpeg and inference; the default

segment grid is 25 s windows with 5 s overlap (TIKA-4900).

SERVER, PIPES AND OPERATIONS

* tika-server: named configuration presets, and catalog presets that are

inert until the config names them (TIKA-4856).

* Add Micrometer reporting and opt-in endpoint for tika-server (TIKA-4839).

* Enable environment variable interpolation in tika-config.json (TIKA-4909).

* tika-server and tika-async-cli accept // and /* */ comments in config

during override merging (TIKA-4834).

* New file-system-jsonl-reporter pipes reporter records a batch run's

crashes; tika-eval Profile/Compare read that ledger and the run-info

json to classify NO_EXTRACT_FILE by cause (TIKA-4846, TIKA-4847).

* PipesForkParser no longer drops javaPath and socketTimeoutMillis set in

code; the fork ran java from the PATH with a 60s socket timeout

regardless (TIKA-4931).

* Pipes no longer copies the caller's Content-Type hint back over the

detected type on output (TIKA-4932).

EXTRACTION AND FORMATS

* Improve extraction of tagged PDFs. New tika-eval-structure tool compares

two sets of XHTML extracts block by block (TIKA-4891).

* OneNote: document-order extraction, superseded revisions omitted,

embedded BLOBs extracted, bounded recursion; malformed files fall back

to the legacy string dump via Henry Lindeman (TIKA-4814).

* Raw camera formats (NEF/NRW, PEF/PTX, ARW/SRF/SR2, SRW, DNG, RAF, RW2,

MRW, ORF) are detected by content instead of as image/tiff via Dominik

Schmidt (TIKA-4861).

* AVIF images are parsed by HeifParser (dimensions, EXIF, XMP) via Dominik

Schmidt (TIKA-4870).

* New GeoGebraParser for *.ggb/*.ggs/*.ggt with content-based detection

and THUMBNAIL embedded documents via Dominik Schmidt (TIKA-4831).

* The video of a Google/Android motion photo is emitted as an ATTACHMENT

embedded document via Dominik Schmidt (TIKA-4869).

* RawTiffParser extracts embedded JPEG previews as THUMBNAIL embedded

documents; image/x-raw-* are now subtypes of image/tiff

("extractPreviews": false disables) via Dominik Schmidt (TIKA-4824).

* Apple Mail emlx files are detected as message/x-emlx and parsed by

RFC822Parser; a file name no longer turns plain text into a message type

once the type's magic has rejected the bytes (TIKA-4890).

* Audio cover art (ID3/FLAC/Vorbis front cover, MP4 covr) is emitted as a

THUMBNAIL embedded document rather than INLINE via Dominik Schmidt

(TIKA-4850).

* EpubParser emits the OPF cover image as a THUMBNAIL embedded document

via Dominik Schmidt (TIKA-4852).

* DWGReadParser emits THUMBNAILIMAGE as THUMBNAIL instead of INLINE

via Dominik Schmidt (TIKA-4853).

* iWork '09 and '18 preview images are emitted as THUMBNAIL embedded

documents via Dominik Schmidt (TIKA-4854).

* New poi-metafile-renderer rasterizes EMF/WMF (opt-in "renderImage");

OfficeParser emits the OLE2 SummaryInformation thumbnail as a THUMBNAIL

embedded document ("extractThumbnail": false disables) via Dominik

Schmidt (TIKA-4855).

OTHER CHANGES

* Ogg, Vorbis, Opus and FLAC comment fields are read independently of the

JVM's default locale: on a Turkish JVM TITLE and ARTIST were silently

lost. The OCR image-rotation value is formatted the same way (TIKA-4921).

* The Ignite config store quotes its table name, so lookups work under any