GeezerVison

What

Jobs the new preflight check rejects no longer break the SDK. Workflows can also opt out of preflight from the SDK.

Before, after a job is rejected:

client.jobs.list_jobs(...)   → ValidationError: 'REJECTED' is not a valid JobStatus   (whole page lost)
client.jobs.get_job(...)     → ValidationError

After:

job.status   → JobStatus.REJECTED
job.status   → "SOME_FUTURE_STATUS"   # a status this SDK version doesn't know comes back as a plain str instead of raising

details.node_stats[0].failure_category → PreflightFailureCategory.AUTH_INVALID_CREDENTIAL
details.node_stats[0].failure_reason   → "The AWS Access Key Id you provided does not exist."

• JobStatus and JobProcessingStatus gain REJECTED. JobInformation.status and JobDetails.processing_status also accept unknown values. Known values are still enum members, so job.status == JobStatus.COMPLETED and .value behave as before.
• JobNodeDetails gains failure_category (Union[PreflightFailureCategory, str]) and failure_reason, so a caller can see why a job was rejected. The server sets them only on the node that failed preflight; before this, pydantic dropped them.
• skip_preflight is now on CreateWorkflow, UpdateWorkflow (unset = unchanged) and WorkflowInformation. For a one-off create_job, the API defines the flag inside the request_data JSON. The docs now say to set it there; the SDK does not rewrite that string.
• Releases as 0.47.0.

Why

The SDK models used to be regenerated from the API spec every night. That generator has been dead since January (removed in #349). Since then, new server values reach the SDK only when someone adds them by hand. REJECTED didn't get added, and a closed enum turns a new value into a hard failure.

This PR also adds a way to catch that kind of drift before a customer does (DTPL-696 acceptance criterion 4):

weekly CI → fetch live OpenAPI specs → compare with SDK models → new missing field/enum value → open/update a GitHub issue
                                                                    (a failed download fails the run but opens no issue)

• make check-spec-drift runs the check locally. Gaps we already know about are recorded in scripts/spec_drift_baseline.json (20 schemas today, e.g. JobInformation.processing_status, the teradata connector type). The check fails only on new drift, or on a baseline entry that has since been covered.
• CONTRIBUTING.md documents how to resolve a drift issue.

Replaces #353, which got here through a custom _missing_ enum hook and code that spliced bytes into request_data. This version uses a plain pydantic union and leaves request_data alone.

Notes for consumers

• Type checkers now see these two status fields as Union[JobStatus, str] / Union[JobProcessingStatus, str]. Code that reads .value or .name on them needs to handle the str case. Runtime behavior for known statuses is unchanged.
• Follow-up, not in this PR: the SDK's other "open" enums (connector types, partition Strategy, etc.) use OpenEnumMeta / validate_open_enum. On Python 3.12+ those return a raw str even for known values, and they emit pydantic serializer warnings. The long-term fix is to move them to the same union pattern used here. That changes their runtime types, so it belongs in its own PR.

Validation

• make test-unit: 419 passed. make test-contract: 72 passed. The new job/workflow contract tests pass on Python 3.11, 3.12 and 3.13, and they fail without the model changes.
• make lint is clean. make check-spec-drift passes against the live specs.
• Two rounds of Codex review. Round 1 found that enums declared inline on a property escaped the drift check; that is fixed and has a test. Round 2 was clean.
• cubic: three findings fixed (no drift issue on a fetch failure, no duplicate issues from overlapping runs, changelog wording). Re-review clean, and full CI is green, including integration on 3.11–3.13.

Maintenance validation (2026-10-19, keepalive base refresh)

• Synchronized main through its 0.46.5 keepalive release while retaining the authored 0.47.0 version and user-agent. Manual conflict resolution touched only the changelog and version file; main runtime and test changes merged automatically.
• Local validation: 419 unit tests and 71 contract tests passed (one existing expected unit failure). Four localhost socket tests needed bind permission and passed without source changes.
• Exact maintenance-head CI: https://github.com/Unstructured-IO/unstructured-python-client/actions/runs/39329146842.

Review follow-up (2026-10-20)

• Added failure_category / failure_reason to JobNodeDetails with a new PreflightFailureCategory enum, and dropped both from the drift baseline. A new contract test covers a known category, an unknown category (plain str) and a passing node (both null); it passes on 3.11, 3.12 and 3.13.
• Removed the JobStatus / JobProcessingStatus entries from overlay_client.yaml. Nothing reads the overlay since the generator was removed, and x-speakeasy-unknown-values would describe the OpenEnumMeta style, not the union shipped here.
• make test-unit 419 passed, make test-contract 72 passed, make lint clean, make check-spec-drift clean against the live specs.