The bug
In NDJSON elements-file mode, a split-PDF chunk that returned HTTP 200 with an empty body was logged and skipped, so the combined elements_file was silently short by that chunk's pages while the call still returned 200.
Nothing downstream could detect it: combine_chunk_files_to_ndjson hands back only the combined path, and split_pdf_allow_failed=False does not cover the case because an empty 200 counts as a successful chunk. The buffered path fails outright on the same response (res.json() raises JSONDecodeError), so enabling NDJSON mode converted a hard failure into silent truncation.
The fix
Recombination now raises EmptyChunkResponseError (a ValueError, matching where the buffered path's JSONDecodeError lands) instead of skipping.
Emptiness is judged against the chunk's own Content-Type, because the two formats disagree about what an empty body means:
- JSON has no empty document — a chunk with no elements is
[] — so an empty body is malformed and raises.
application/x-ndjson encodes zero records as zero lines, so an empty body is well formed. It contributes nothing and does not fail the partition; otherwise a split whose pages are blank would break once a server honors the accept header.- An unknown or missing media type is read as JSON. That is what the deployed API returns, and guessing NDJSON would reinstate the silent truncation.
combine_chunk_files_to_ndjson takes an optional media_types list, positionally matched to chunk_paths; omitting it keeps the strict reading. _elements_from_task_responses collects each chunk's Content-Type before the cached branch overwrites the body with a temp-file path — the header survives both cache branches, so the cached case carries a real media type too.
Tests
_test_unstructured_client/unit/test_ndjson_elements_file.py:
- empty JSON chunk raises, over 3 empty-ish bodies ×
application/json / unset / with-charset
- empty NDJSON chunk is zero records, over 3 bodies × 3 media-type spellings (parameters, casing)
- every chunk empty yields an empty output rather than an error
ValueError parity with the buffered path, and the strict default when media_types is omitted- hook level, both cache modes: an empty JSON chunk fails the operation and leaves no partial, spilled, or combined file behind; an empty NDJSON chunk is accepted with the surrounding chunks' elements intact
Mutating _is_ndjson_media_type to return False fails 12 of these, including both hook-level cache-mode tests, so the coverage is load-bearing.
271 unit + 64 contract tests pass; pylint 10/10 on both changed modules.
Release
Bumped to 0.46.1 with CHANGELOG and RELEASES entries. Consumers that pin unstructured-client >=0.46.0 for NDJSON elements-file mode should raise the floor to >=0.46.1.
🤖 Generated with Claude Code