GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — refactor: stop allocating on tuple and list hashing by robinbb · Pull Request #14542 · ocaml/dune

Conversation

@robinbb

@robinbb robinbb commented May 15, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

Summary

Extract the (acc * 31) + x hash accumulator into a new Stdune.Hash module per #14542 review, and route Tuple.T2.hash, Tuple.T3.hash, List.hash, Result.hash, and Resolve.error_hash through it. None of these allocate per call any more. (Option.hash is already optimal on main as of #14543 and is left untouched.)

Drive-by fix: Stdune.Result.hash no longer collides Ok x with Error y whenever h1 x = h2 y (e.g. Ok 0 vs Error 0) — same bug shape Stdune.Option.hash had before #14543.

Also consumes #14605 directly at one site: src/source/opam_switch.ml already included Repr.Poly, but a redundant manual let hash shadowed the inherited one. Now dropped.

A broader sweep of "eliminate manual hash via Repr.Poly" is out of scope here. Repr.Poly.validate_compare rejects Repr.view, which is pervasive in dune's wrapped-string newtypes (paths, lib-names, opam URLs, checksums, package names, ...). The set of types whose Repr.t is record/variant + primitives all the way down is small enough that the broader cleanup is best done as a separate, deliberate PR with its own decisions about whether to drop Repr.view from the affected newtypes.

Related: #14605 (Repr.Poly.hash).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@robinbb robinbb self-assigned this May 15, 2026
@robinbb
robinbb requested a review from Copilot May 15, 2026 02:38
Copilot started reviewing on behalf of robinbb May 15, 2026 02:39
View session
Copilot AI reviewed May 15, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors Stdune tuple and list hash helpers to avoid building intermediate tuples/lists before hashing, replacing polymorphic hashing with arithmetic combiners.

Changes:

  • Replaced Tuple.T2.hash and Tuple.T3.hash tuple allocation with multiplicative integer combination.
  • Replaced List.hash’s mapped-list allocation with a left fold.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
otherlibs/stdune/src/tuple.ml Updates tuple hash implementations to combine component hashes directly.
otherlibs/stdune/src/list.ml Updates list hashing to fold over element hashes without constructing an intermediate list.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread otherlibs/stdune/src/list.ml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread otherlibs/stdune/src/list.ml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@robinbb
robinbb force-pushed the robinbb-stdune-hash-nonalloc branch from cc6bccb to e193cc5 Compare May 15, 2026 02:54
@robinbb
robinbb requested a review from Copilot May 15, 2026 02:55
Copilot started reviewing on behalf of robinbb May 15, 2026 02:56
View session
Copilot AI reviewed May 15, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread otherlibs/stdune/src/list.ml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@robinbb robinbb mentioned this pull request May 15, 2026
fix: Option.hash distinguishes None from Some x when (f x = 0) #14543
Merged
@robinbb
robinbb force-pushed the robinbb-stdune-hash-nonalloc branch from e193cc5 to c566d44 Compare May 15, 2026 03:08
@robinbb
robinbb requested a review from Copilot May 15, 2026 03:09
Copilot started reviewing on behalf of robinbb May 15, 2026 03:10
View session
Copilot AI reviewed May 15, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread otherlibs/stdune/src/list.ml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@robinbb

robinbb commented May 15, 2026

Copy link
Copy Markdown
Collaborator Author

Allocation measurement

To check the magnitude of the effect on dune itself, ran a paired dune-on-dune null build experiment.

Setup. dune-old built from upstream/main at 080d3e680d (parent of this PR's commits); dune-new built from this PR's tip c566d4445a. Workload: a clone of the dune source tree, fully built — dune build is a no-op rebuild that exercises the memo cache scan and rule-evaluation path without doing compilation work. Measurement: OCAMLRUNPARAM=v=0x400 <dune> build, picking the parent-process allocation block by max allocated_words. 5 paired runs alternating old/new.

Results.

run old allocated_words new allocated_words Δ
1 757,503,985 756,686,557 −817,428
2 757,587,297 756,756,752 −830,545
3 757,503,216 756,687,417 −815,799
4 757,655,668 756,604,684 −1,050,984
5 757,587,355 756,759,976 −827,379

Mean Δ: −868,427 words ≈ −7.0 MB per null build (−0.115%). All 5 paired deltas have the same sign; range of deltas (~235k) is well below the magnitude (~870k). minor_collections and major_collections were unchanged between binaries, so the effect is purely on minor-heap throughput, not GC pressure.

Treating this as a defensive/clarity change rather than a perf win — the saving is real but unlikely to be observable in wall time on a realistic workload.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@robinbb
robinbb requested a review from rgrinberg May 15, 2026 16:15
@robinbb robinbb assigned rgrinberg May 15, 2026
@robinbb
robinbb marked this pull request as ready for review May 15, 2026 16:16
rgrinberg pushed a commit that referenced this pull request May 15, 2026
@robinbb
fix: Option.hash distinguishes None from Some x when (f x = 0) (#14543)
3aa5afc
## Summary

`Option.hash f None` and `Option.hash f (Some x)` previously both went
through `Stdlib.Hashtbl.hash`, which gave equal results whenever `f x =
0` because `None` shares OCaml's tag-0 runtime representation with the
integer 0. So under stdune's existing hashers:

- `Option.hash Bool.hash (Some false)` collided with `Option.hash
Bool.hash None` (`Bool.hash false = 0`).
- `Option.hash Unit.hash (Some ())` collided with `Option.hash Unit.hash
None` (`Unit.hash () = 0`).

This PR replaces the structural hash with a tagged recurrence — `None ->
0`, `Some s -> (f s * 31) + 1` — so the two cases land in disjoint
buckets. Added an expect test in `otherlibs/stdune/test/option_tests.ml`
covering the `Bool` and `Unit` cases that previously collided.

Discovered while surveying allocation-heavy hash sites for a follow-up
to #14542.

Signed-off-by: Robin Bate Boerop <[email protected]>
@rgrinberg

rgrinberg commented May 15, 2026

Copy link
Copy Markdown
Member

It's ok to deviate from Poly.hash. We do not rely on hashes for correctness, only for performance. However, I'd rather not introduce code like this unless we can measure the impact on dune itself.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@robinbb

robinbb commented May 15, 2026

Copy link
Copy Markdown
Collaborator Author

It's ok to deviate from Poly.hash. We do not rely on hashes for correctness, only for performance. However, I'd rather not introduce code like this unless we can measure the impact on dune itself.

I think I have done that, at least for dune-on-dune builds. It has an almost-imperceptible performance increase of 0.1%, and a reduction in memory allocations of 0.1%, also. An admittedly minor improvement. Simply close the PR if you don't want the change, @rgrinberg. I'm not concerned either way. Just offering a small improvement in performance; I understand it comes with risks related to hash bits entropy.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rgrinberg

rgrinberg commented May 19, 2026

Copy link
Copy Markdown
Member

Here's a related for PR that allows us to replace poorly written hash functions with a Poly.hash #14605

For this PR, I'm ok with the approach in principle, but I would prefer the underlying implementation to live in a separate module. E.g. Stdune.Hash with:

type t
val create : unit -> t
val feed : t -> int -> t
val hash : t -> int
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@robinbb
robinbb force-pushed the robinbb-stdune-hash-nonalloc branch 4 times, most recently from 7f6dc75 to 0bf3b93 Compare May 29, 2026 22:53
@robinbb

robinbb commented May 29, 2026

Copy link
Copy Markdown
Collaborator Author

@rgrinberg Is what's in this PR now what you had in mind? (Forgive my slowness getting back to this, but I think it's low priority for both of us.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

rgrinberg
rgrinberg approved these changes Jun 3, 2026
View reviewed changes

@rgrinberg rgrinberg left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the primitive should be val hash_feed: t -> int -> t.? This is more inline with core and base.

LGTM.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

robinbb reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread src/dune_rules/resolve.ml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@robinbb
refactor: stop allocating on structural hashing; extract Stdune.Hash
4dced4f
Extract the multiplicative `(acc * 31) + x` accumulator into a new
`Stdune.Hash` module with the API requested in review:

    type t
    val create : unit -> t
    val feed   : t -> int -> t
    val hash   : t -> int

Route the structural hash combinators through it:

- `Stdune.Tuple.T2.hash`, `Stdune.Tuple.T3.hash`, `Stdune.List.hash`
  drop the allocations (a fresh inner tuple / a fresh `int list`)
  that they previously paid before handing off to `Hashtbl.hash`.

- `Stdune.Result.hash` likewise stops allocating and gains explicit
  discriminator bytes. This is a drive-by bugfix: the previous
  implementation collided `Ok x` with `Error y` whenever the inner
  hashes were equal (e.g. `Ok 0` and `Error 0`), the same shape as
  the bug `Stdune.Option.hash` had pre-ocaml#14543.

- `Resolve.error_hash` (`src/dune_rules/resolve.ml`) drops its 2-tuple
  allocation and its per-call `List.map ~f:Lazy.force` (which builds
  a fresh `int list` of length |stack_frames|), folding the forced
  frames into the accumulator instead.

`Stdune.Option.hash` is already optimal on main as of ocaml#14543 and is
left unchanged.

Direct consumption of ocaml#14605: `src/source/opam_switch.ml` already
included `Repr.Poly`, but a redundant manual `let hash` shadowed the
inherited one. That definition is now dropped. This is a single
demonstration site for the elimination-by-Repr.Poly direction
referenced in the review; a broader sweep is out of scope here
because `Repr.Poly`'s `validate_compare` rejects `Repr.view`, which
pervades dune's wrapped-string newtypes (paths, lib-names, opam
URLs, checksums, package names, ...).

Same `'a -> 'a -> int` callsite API; callers don't change. Hash
values shift, but these functions feed `Hashtbl` / `Memo` bucket
selectors only — no persistent serialization, no cross-process
invariants, so the shift is invisible to consumers.

Signed-off-by: Robin Bate Boerop <[email protected]>
@robinbb
robinbb force-pushed the robinbb-stdune-hash-nonalloc branch from 0bf3b93 to 4dced4f Compare June 3, 2026 22:28
@robinbb
robinbb requested a review from Copilot June 3, 2026 22:33
Copilot started reviewing on behalf of robinbb June 3, 2026 22:33
View session
Copilot AI reviewed Jun 3, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@robinbb
robinbb merged commit 281990d into ocaml:main Jun 4, 2026
30 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@robinbb
robinbb deleted the robinbb-stdune-hash-nonalloc branch June 4, 2026 13:29
@Alizter Alizter added this to the 3.25.0 milestone Jul 8, 2026
@shonfeder shonfeder mentioned this pull request Oct 2, 2026
[new release] dune (16 packages) (3.25.0~alpha0) ocaml/opam-repository#30850
Closed
@shonfeder shonfeder mentioned this pull request Nov 3, 2026
[new release] dune (16 packages) (3.25.0~alpha1) ocaml/opam-repository#31082
Closed
@shonfeder shonfeder mentioned this pull request Nov 12, 2026
[new release] dune (16 packages) (3.25.0) ocaml/opam-repository#31193
Merged
raphael-proust pushed a commit to ocaml/opam-repository that referenced this pull request Nov 12, 2026
@shonfeder
[new release] dune (16 packages) (3.25.0)
5493d2f
CHANGES:

### Fixed

- Fix crashes when expanding module artifact pforms (`%{cmi:...}` etc.) with
  `(include_subdirs qualified)`. Qualified modules can now be referenced by
  dotted logical paths such as `%{cmi:Sub_a.Foo}`.
  (ocaml/dune#14498, ocaml/dune#16214, fixes ocaml/dune#14496, @Alizter, @anmonteiro)

- `dune show targets` now accepts build-only directories such as
  `_build/default/.simple.objs`. (ocaml/dune#14664, fixes ocaml/dune#13782, @Alizter)

- Reword the error raised when `%{target}` is used in a rule that omits the
  `(target ...)` (or `(targets ...)`) field, so it points at the missing
  field instead of mentioning "inferred rules". (ocaml/dune#14668, fixes ocaml/dune#12439,
  @Alizter)

- Fix watch mode on Windows by creating the file-watcher sync directory before
  using it, avoiding `open(_build/.sync\0): No such file or directory` when an
  RPC client requests a build. (ocaml/dune#14678, fixes ocaml/dune#14632, @rgrinberg)

- Add file watcher trace events for the Mac and Windows backends (ocaml/dune#14722, @rgrinberg)

- Fix decoding coalesced MacOS file notification events. (ocaml/dune#14725, @rgrinberg)

- Handle directory invalidations correctly in watch mode on MacOS (ocaml/dune#14756, @rgrinberg)

- Update ctime when restoring from the shared cache in copy mode. This prevents
  the trimmer from deleting the file. (ocaml/dune#14777, @rgrinberg)

- Allow the use of all the available variables in the `(enabled_if)` field in
  `(executable ...)` and `(install ...)` stanzas, similar to other stanzas like
  `(library ...)`. (ocaml/dune#14860, fixes ocaml/dune#14789, @punchagan)

- Linux: dune will now wait for the process group to be dead after the group
  leader is dead. This tightens dune's policy for what constitutes
  a successful action. Actions for which dune cannot be clean up the process
  group will now fail. (@rgrinberg, ocaml/dune#14868)

- Fix caching of package sources with symlinks by resolving symlinks after
  fetching the sources, similar to the directory symlinks resolution introduced
  in ocaml/dune#13792 (ocaml/dune#14871, fixes ocaml/dune#14509, @punchagan)

- Fixed wrong suggestion for duplicate entries in `conflicts` field (ocaml/dune#14956,
  fixes ocaml/dune#14903, @Leonidas-from-XIV)

- Fix building Dune on Solaris/illumos by handling its `wait4` semantics
  and retrying interrupted reads when spawning processes. (ocaml/dune#14980, fixes ocaml/dune#14182,
  @jperkin)

- Fix `dune-rpc-lwt` clients so disconnecting shuts down socket output
  without invalidating pending input reads, avoiding `EBADF` errors. (ocaml/dune#14983,
  fixes ocaml/dune#7624, @rgrinberg)

- Dune now restores dependency cutoffs concurrently when they were originally
  computed in parallel, instead of recomputing them one at a time. The
  `cutoffs_that_reduce_concurrency_in_watch_mode` setting, which existed to
  avoid this serialisation, is no longer needed and has been removed.
  (ocaml/dune#15012, @snowleopard)

- Validate project module names before creating the project directory in
  `dune init project`, so aborted initializations no longer leave empty
  directories behind. (ocaml/dune#15014, fixes ocaml/dune#9963, @rgrinberg)

- Fix cram test output sanitization when the project path contains a space.
  Absolute paths are now rewritten to `$TESTCASE_ROOT` even when the path
  contains spaces, and lines with multiple paths have every path sanitized.
  (ocaml/dune#15017, fixes ocaml/dune#14943, @yussypu)

- Improve the error message for private libraries that cannot be dependencies of
  public libraries by mentioning the `(package ...)` field as an alternative to
  giving the private library a public name (ocaml/dune#15070, fixes ocaml/dune#10230, @rgrinberg).

- Clarify the module conflict error shown when a module is used in multiple
  stanzas. The message now explains implicit module ownership and lists all
  relevant stanza kinds (ocaml/dune#15072, fixes ocaml/dune#6598, @rgrinberg).

- Emit relative artifact references in installed `dune-package` files when using
  an absolute external `DUNE_BUILD_DIR`, avoiding build-directory-dependent
  archive and module paths (ocaml/dune#15077, fixes ocaml/dune#11774, @rgrinberg).

- Simplify away `(when false _)` expressions (encoded `Nil`s) that previously
  survived inside `concat` and `when` forms, producing cleaner generated lock
  directories. (ocaml/dune#15092, fixes ocaml/dune#15090, @chizy7)

- Drop `(when false _)` arguments from `run` actions in generated lock
  directories instead of keeping them as no-op entries that were re-encoded as
  `(when false "")`. (ocaml/dune#15128, fixes ocaml/dune#15099, @chizy7)

- Make dune-configurator pass `--personality` to `pkgconf` only for
  version 1.7.0 and later. (ocaml/dune#15142, fixes ocaml/dune#11536, @dra27)

- `dune utop` now accepts an absolute path to a workspace directory instead of
  crashing with an internal error. (ocaml/dune#15272, fixes ocaml/dune#15108, @chizy7)

- Fix watch-mode status lines after source changes interrupt a build. Dune now
  shows the restart status while cancelling the current build and resumes the
  normal progress counter when the next build starts. (ocaml/dune#15289, fixes ocaml/dune#11134,
  @rgrinberg)

- Short-circuit AND and OR expressions in blang. This makes their evaluation
  lazy by default.
  (ocaml/dune#15291, @Sudha247)

- Fix the `%{ppx:ppx1+..+ppxn}` form. Previously, it was just broken
  and pointed at the wrong binary (ocaml/dune#15361, @rgrinberg)

- Remove ppx runtime dependencies from the list of dependencies provided via
  the `root_module` field. If such a runtime dependency is needed, it should be
  written in the `libraries` field. (ocaml/dune#15419, @rgrinberg)

- Avoid rebuilding `js_of_ocaml` library archives on every invocation for
  interface-only and empty libraries. (ocaml/dune#15427, @anmonteiro)

- Do not crash when `~/.config/dune` is a file instead of a directory. Just
  pretend it doesn't exist (ocaml/dune#15475, fixes ocaml/dune#15406, @rgrinberg)

- Dune will now terminate all launched sub-processes even if it received
  `SIGKILL` itself (ocaml/dune#15483, @rgrinberg)

- Account for `ctime` in stat based digest cache (ocaml/dune#15495, @rgrinberg)

- `$ dune install --dry-run` now checks for non empty directories as part of its
  validation (ocaml/dune#15501, @rgrinberg)

- Fix a target-less rule attached to multiple aliases (`(rule (aliases a b)
  ...)`) leaking unrelated contributions between those aliases. Building one
  alias no longer pulls in actions or dependencies that were added only to
  another of the rule's aliases. (ocaml/dune#15581, @NatKarmios)

- Make `dune pkg enabled` query a running Dune RPC server, so it observes
  transient server-side `--pkg enabled` and `--pkg disabled` settings without
  trying to acquire the build lock (ocaml/dune#15589, fixes ocaml/dune#15587, @Alizter)

- Make generated MDX executables inherit `link_flags` from `env` stanzas.
  (ocaml/dune#15614, fixes ocaml/dune#15613, @Alizter)

- Return Unix errors from `Fs_io.read_file` rather than raising them. (ocaml/dune#15641,
  @rgrinberg)

- Correct filesystem trace metrics for file writes and directory scans.
  (ocaml/dune#15686, @rgrinberg)

- Preserve runtime-event ring IDs in Dune traces so consumers can distinguish
  interleaved event streams. (ocaml/dune#15687, @rgrinberg)

- BREAKING CHANGE: in dune-project, (warnings (duplicate_deps ..)) requires
  a dune lang version of 3.22, instead of 3.18 (ocaml/dune#15696, @v-gb).

- Preserve spaces in unquoted filenames when applying Git patches instead of
  truncating paths at the first space (ocaml/dune#15781, @rgrinberg)

- Respect `enabled_if` on `install` stanzas when adding their files and
  directories to the `@all` alias. (ocaml/dune#15895, fixes ocaml/dune#15825, @aryemans)

- Fix Melange emission with installed libraries whose `.cmj` and `.cmi` files
  are stored in the `melange` object directory (ocaml/dune#15949, fixes ocaml/dune#14820,
  @anmonteiro)

- Reject package dependencies in named dependency bindings while decoding
  dependency fields, including for disabled stanzas. (ocaml/dune#15966, @rgrinberg)

- Fix Merlin definition lookup for libraries with private modules
  (ocaml/dune#15977, fixes ocaml/dune#4892, @anmonteiro)

- Fix a crash when generating install rules for a Melange-only implementation
  of an installed virtual library (ocaml/dune#16033, @anmonteiro)

- Prevent build cancellation from crashing Dune when macOS reports `EPERM`
  while signaling a process group (ocaml/dune#16059, fixes ocaml/dune#16053, @rgrinberg)

- Fix `dune pkg outdated` so it fails early with a clear user message if it is
  run when package management is not enabled. (ocaml/dune#16065, @shonfeder)

- Fix watch mode builds remaining stale after the file watcher event queue
  overflows (ocaml/dune#16068, fixes ocaml/dune#16052, @rgrinberg)

- Reject Windows path separators in validated filename components (ocaml/dune#16076,
  @rgrinberg)

- Preserve output following malformed ANSI escape sequences and erase-line
  controls (ocaml/dune#16116, @rgrinberg).

- Resolve relative paths passed to `dune ocaml top-module` from the directory
  where Dune was started, allowing use from workspace subdirectories. (ocaml/dune#16135,
  @rgrinberg)

- Allow `dune ocaml top` to accept absolute directory paths inside the
  workspace. (ocaml/dune#16142, @rgrinberg)

- Allow Dune promotion commands to accept absolute source paths inside the
  workspace. (ocaml/dune#16143, @rgrinberg)

- Allow `dune describe workspace` to accept absolute directory filters inside
  the workspace. (ocaml/dune#16144, @rgrinberg)

- Reject empty `module_systems` fields in `melange.emit` stanzas instead of
  silently producing no JavaScript files (ocaml/dune#16162, @anmonteiro)

- Sandbox PPX lint actions starting with Dune language 3.25 so linters cannot
  observe undeclared build targets. (ocaml/dune#16176, @rgrinberg)

- Prevent diagnostic source locations at compact-position field boundaries from
  being truncated (ocaml/dune#16180, @rgrinberg)

- Sandbox `ocaml-mdx deps` when using MDX extension 0.6 so dependency
  discovery cannot observe undeclared workspace inputs. (ocaml/dune#16183, @rgrinberg)

- Fix missing Odoc generation of OxCaml parameter implementations (ocaml/dune#16200, @art-w)

- Make `Dune_rpc_lwt.V1.Action_plugin.read_directory_with_glob` return an empty listing
  for absent directories (ocaml/dune#16207, fixes ocaml/dune#4445, @rgrinberg)

- Fix compiling Melange implementations of installed virtual libraries, both
  with and without binary annotations, including transitive private module
  dependencies (ocaml/dune#16211, ocaml/dune#16212, ocaml/dune#16272, @anmonteiro)

- Run Ctypes stub generation in a sandbox and honor its declared dependencies.
  (ocaml/dune#16223, @rgrinberg)

- Fix `@opam` failing to promote a generated opam file after it is deleted
  (ocaml/dune#16234, fixes ocaml/dune#16233, @Alizter)

- Rebuild generated Ctypes helpers when headers from Ctypes' implicit include
  directories change (ocaml/dune#16243, @rgrinberg)

- Collect action-provided trace events when an action fails instead of dropping
  them during result validation. (ocaml/dune#16250, @rgrinberg)

- Report the usual program-not-found error when a `dynamic-run` executable is
  missing instead of failing with an assertion. (ocaml/dune#16251, @rgrinberg)

- Prevent rules with duplicate lock declarations from deadlocking by acquiring
  each lock only once. (ocaml/dune#16252, @rgrinberg)

- Prevent concurrent action children from closing shared redirections before
  their siblings use them (ocaml/dune#16257, @rgrinberg)

- Resolve relative paths in forwarded build, runtest, and exec commands from
  the client's directory instead of the RPC server's directory (ocaml/dune#16267,
  @rgrinberg)

- `(env (env-vars ...))` in a `(context ...)` stanza of `dune-workspace` now
  takes precedence over the root `env` stanza, as documented. This affects
  commands such as `dune exec`; rules already used the documented order.
  (ocaml/dune#16290, @punchagan)

- Fix a crash when an opam filter is not a boolean (ocaml/dune#16313, @art-w)

- Correct module paths for generated interfaces of nested qualified module
  groups. (ocaml/dune#16322, @anmonteiro)

- Resolve ``%{cmi:...}``, ``%{cmt:...}``, and ``%{cmti:...}`` against Melange
  module sets when no OCaml module is selected, while preferring OCaml for
  modules selected in both modes. (ocaml/dune#16339, @anmonteiro)

- Reject `.` and `..` as `melange.emit` targets instead of raising an internal
  error (ocaml/dune#16371, @anmonteiro)

- Validate custom toplevel names as filenames, reporting an error for empty
  names and avoiding crashes for names containing paths (ocaml/dune#16373, @anmonteiro)

- Fix Menhir type inference with package-managed relocatable compilers on macOS
  by making the compiler standard library available in the sandbox (ocaml/dune#16374,
  fixes ocaml/dune#15642, @Alizter, @shonfeder)

- Reject invalid Menhir `merge_into` names during parsing to prevent internal
  errors (ocaml/dune#16387, @anmonteiro)

- Reject conflicting module sources in unqualified directory groups,
  including generated sources, instead of silently discarding one
  (ocaml/dune#16414, @anmonteiro)

- fix copying of directory targets on windows (ocaml/dune#16434, @kentookura)

- Track local module dependencies used by Menhir's type inference so generated
  parsers and dependent modules build and rebuild correctly
  (ocaml/dune#16447, fixes ocaml/dune#14450, @anmonteiro)

- Fix Menhir type inference for qualified module group interfaces while keeping
  dependency lookup in terms of logical module names (ocaml/dune#16464, ocaml/dune#16489, ocaml/dune#16497,
  fixes ocaml/dune#8989, @anmonteiro)

- Fix intermittent crashes on Windows caused by unstable file descriptor
  hashing (ocaml/dune#16476, fixes ocaml/dune#16459, @Alizter)

- Explain that forwarded build requests require a watch-mode Dune server instead
  of reporting an internal RPC error. (ocaml/dune#16486, fixes ocaml/dune#16219, @Alizter)

- `(deps (package ...))` now tracks installed files of lock-built packages
  and their package dependencies, rebuilding actions when they change.
  (ocaml/dune#16507, @Alizter)

- Share Rocq plugin layout dependencies between discovery and compilation
  rules, reducing repeated dependency processing in unchanged builds.
  (ocaml/dune#16622, @Alizter)

- Support directory dependencies in hardlink sandboxes by linking their files
  rather than the directories themselves (ocaml/dune#16675, @Alizter).

- Fix recursive copy actions rejecting symbolic links, including file links
  created by symlink sandboxes (ocaml/dune#16686, @rgrinberg).

- Fix copy sandbox failures caused by overlapping directory and file
  dependencies (ocaml/dune#16806, fixes ocaml/dune#16547, @Alizter)

- Fix Menhir inference producing invalid module paths for types imported from
  other modules (ocaml/dune#16952, @anmonteiro)

### Added

- Add `(dirs ...)` to `(include_subdirs (mode qualified) ...)`, allowing
  qualified source directories to be renamed in the module path, including
  when bootstrapping. Mappings can use variables, including reads from generated
  files outside the qualified group (ocaml/dune#13691, ocaml/dune#16418, ocaml/dune#16420, fixes ocaml/dune#7605,
  @anmonteiro).

- Add opt-in action runners via `--action-runner`, and add
  `--sandbox-actions` to run eligible process-spawning actions in a
  bubblewrap-wrapped worker. (ocaml/dune#14240, @rgrinberg)

- Add platform support for DragonFly BSD (ocaml/dune#14774, fixes same issue with lev
  as seen on NetBSD in ocaml/dune#14484, @mneumann).

- Added command, subcommand, and flag completion in the command line.
  Supported shells are `bash`, `zsh`, and `pwsh`.
  See `dune completion [shell] --help`.
  (ocaml/dune#14779, @ElectreAAS)

- Add ``%{git-sha}`` variable that expands to the short git SHA of the current
  HEAD commit, or the empty string when git is unavailable or the workspace is
  not inside a git repository (ocaml/dune#14810, @toots).

- Add a `?static` optional argumentin `Configurator.V1.Pkg_config.get` to instruct pkg-config to get static linking options. (ocaml/dune#14947, @raphael-proust)

- Only generate Melange rule for multi-mode libraries if `melc` is installed
  (ocaml/dune#15021, @anmonteiro)

- Add `Configurator.V1.Pkg_config.query_variable` to query package variables
  through `pkg-config --variable` (ocaml/dune#15078, fixes ocaml/dune#3332, @rgrinberg).

- Add `env_vars` as an alias for the `env-vars` field in `env` stanzas
  (ocaml/dune#15187, fixes ocaml/dune#11424, @rgrinberg).

- Add `@rocqproject` alias to generate all `_RocqProject files
  (ocaml/dune#15221, @rlepigre-skylabs-ai).

- Tab completion for `dune promote` now lists pending promotion files.
  (ocaml/dune#15338, fixes ocaml/dune#7254, @Alizter).

- Add build time information to watch mode (ocaml/dune#15384, @rgrinberg)

- Added the missing `dune tools exec utop` and `dune tools exec odoc` options.
  (ocaml/dune#15388, @Leonidas-from-XIV)

- Display build parallelism in watch mode. (ocaml/dune#15390, @rgrinberg)

- Add process counts, system cpu, user cpu, and wall clock cpu times to trace
  events (ocaml/dune#15401, @rgrinberg)

- Support `--sandbox-actions` on macOS using sandbox-exec. (ocaml/dune#15500,
  @rgrinberg)

- Add `fish` to the shells supported by `dune completion`, enabling
  command, subcommand, and flag completion in the fish shell (ocaml/dune#15515,
  fixes ocaml/dune#15334, @chizy7)

- Display build duration and average process parallelism in the status line for
  batch builds. (ocaml/dune#15564, @rgrinberg)

- Add a `--split-public-names` flag to `dune describe workspace` and `dune
  describe external-lib-deps`. When enabled, the `name`/`names` fields report
  only private names and a new `public_name`/`public_names` field reports the
  public names. (ocaml/dune#15582, @NatKarmios)

- Add a `--include-empty` flag to `dune describe external-lib-deps` to also
  print libraries that have no internal or external dependencies (ocaml/dune#15585,
  @NatKarmios)

- Support qualified module references in fields such as `per_module` when using
  `(include_subdirs qualified)` (ocaml/dune#15633, fixes ocaml/dune#15578, @anmonteiro)

- Display the current build run at the end of watch-mode status lines.
  (ocaml/dune#15673, @rgrinberg)

- Record per-build Memo compute, restore, and cycle-detection metrics in Dune
  traces. (ocaml/dune#15698, @rgrinberg)

- Record discovered, validated, and failed rule counts in Dune traces.
  (ocaml/dune#15701, @rgrinberg)

- Add `%{melange.emit:...}` to reference a Melange output directory and
  depend on its emitted files (ocaml/dune#15751, @anmonteiro)

- Add `(flags ...)` to the `ocamllex` and `ocamlyacc` stanzas to pass custom
  options to parser generators. (ocaml/dune#15907, @rgrinberg)

- Added an opt-in `sat` trace category emitting a `sat/solve` event with
  the SAT solver's stats (variables, clauses, decisions, conflicts) and
  duration. Enable with `DUNE_TRACE=+sat`. (ocaml/dune#15923, @Alizter)

- Add `dune ocaml inferred-mli` to infer an interface with a module's build
  configuration and register it for promotion. (ocaml/dune#16111, fixes ocaml/dune#12727,
  @rgrinberg)

- Add `--no-actions` and `--only-actions` to `dune trace cat` to filter
  events collected from actions. (ocaml/dune#16245, @rgrinberg)

### Changed

- BREAKING CHANGE: Actions no longer see the shared install staging area
  (`_build/install/<context>/{bin,lib,lib/stublibs}`) on `PATH`, `OCAMLPATH` and
  `CAML_LD_LIBRARY_PATH`. An action's environment only contains the packages it
  declares with `(deps (package <pkg>))` (and their library closure, see below).
  Actions that rely on packages being visible (such as tests that `#require` the
  package under test from an mdx block, steps that run `ocamlfind`, a nested `dune`
  build, or load a bytecode stub library) must now declare the package
  dependency explicitly.
  (ocaml/dune#14373, ocaml/dune#16617, @Alizter)
- `(package <pkg>)` now also includes the packages owning the library closures
  of libraries in workspace and installed packages. Installed packages reached
  through those closures also work with Dune language versions older than 2.9.
  (ocaml/dune#14373, ocaml/dune#16362, ocaml/dune#16529, @Alizter, @anmonteiro)

- Extract the structural-hash accumulator into a new `Stdune.Hash`
  module and route `Stdune.Tuple.T2.hash`, `Stdune.Tuple.T3.hash`,
  `Stdune.List.hash`, `Stdune.Result.hash`, and `Resolve.error_hash`
  through it. Hashing these structural shapes no longer allocates per
  call.
  (ocaml/dune#14542, @robinbb)

- Remove `(using melange 0.1)` in Dune 3.25 in favor of version 1.0 of the
  extension. (ocaml/dune#14574, @anmonteiro)

- `$ dune rpc build` invocations no longer queue. All build requests are built
  in parallel. Watch servers started by `$ dune exec --watch` can now also
  handle forwarded build and runtest requests (ocaml/dune#14954, @rgrinberg)

- Report rule build progress as the number of rules built against the total
  number of live rules, rather than rules executed against rules discovered
  during the current run. This affects both the status line and the progress
  reported over RPC. (ocaml/dune#14985, @snowleopard, @pWhite)

- With `--error-reporting=twice`, print an `==== Error Summary ====` separator
  before the deterministic list of errors reported at the end of the build. The
  separator is omitted when there are no errors to report.
  (ocaml/dune#15016, fixes ocaml/dune#14757, @yussypu)

- Enable fuzzy patch application by default for Dune. This is similar to OPAM's
  behaviour, and this makes it easier for packages like OxCaml to build.
  (ocaml/dune#15146, fixes ocaml/dune#15145, @Sudha247)

- Odoc rules are now sandboxed (ocaml/dune#15190, @rgrinberg)

- In watch mode, `$ dune rpc build` requests now complete as soon as their
  requested build finishes, instead of waiting for unrelated concurrent RPC build
  requests in the same build iteration. (ocaml/dune#15191, fixes ocaml/dune#13788, @rgrinberg)

- Cleanup all build processes spawned by dune before starting a fresh build on
  Linux. (ocaml/dune#15239, ocaml/dune#15274, @rgrinberg)

- Show partial output timed out cram tests (ocaml/dune#15269, @rgrinberg)

- In watch mode, `$ dune rpc build` now exits with status 1 when the
  requested build fails. (ocaml/dune#15341, @rgrinberg)

- Debounce file change events in watch mode (ocaml/dune#15344, @rgrinberg)

- Align dune's C compiler vendor detection with the OCaml compiler's own
  identifiers (`msvc`, `icc`, `mingw`, `clang`, `gcc`, `xlc`, `sunc`), so more C
  compilers are recognized without emitting a spurious warning. (ocaml/dune#15423,
  fixes ocaml/dune#14870, @chizy7)

- Never use `$ lsof` for counting file descriptors, as it is not allowed
  to reap processes outside of the scheduler. We have better mechanisms
  on MacOS and Linux for counting file descriptors. So this is only an
  issue on other Unices. (ocaml/dune#15477, @rgrinberg)

- `dune describe workspace` now reports an `origin` field for every module,
  indicating whether its source was written by hand (`source`), generated by a
  `rule` stanza (`dune`, with the generating dune file), produced by `ocamllex`,
  `ocamlyacc`, or `menhir`, or synthesised by dune itself for wrapping
  (`wrapper`). Generated modules point at the responsible file. The
  `--with-origin-deps` and `--with-origin-deps-expanded` flags additionally
  report the dependencies of a `rule`-generated module, as declared and expanded
  to concrete files respectively.
  (ocaml/dune#15478, @nat-karmios)

- Numbers with unit now allow dropping the unit if the number is zero ("0").
  For example, `dune cache trim --trimmed-size 0` and `dune cache trim
  --trimmed-size 0TiB` are now both accepted and equivalent. (ocaml/dune#15498, @reynir)

- Run artifact substitution in background workers. (ocaml/dune#15559, @anmonteiro)

- `dune describe` now reports the public name of an executable (from
  `public_name`/`public_names`) instead of its private name when one is
  available, falling back to the private name otherwise. (ocaml/dune#15582, @NatKarmios)

- Calculate status-line build parallelism from process CPU time rather than
  wall-clock action time; Windows continues to use wall-clock time.
  (ocaml/dune#15685, @rgrinberg)

- Use sha256 instead of md5 for source archives that lack a checksum
  (ocaml/dune#15824, @shunueda)

- Starting with Dune language 3.25, restrict sandboxed actions on Linux
  systems with Landlock ABI 3 or newer from writing outside their sandbox,
  temporary directory, action-trace directory, and `/dev`. Set
  `DUNE_CONFIG__LANDLOCK=disabled` to opt out. (ocaml/dune#15858, @rgrinberg)

- `dune pkg lock` now fails without writing a lock directory when any platform
  requested by `solve_for_platforms` cannot be solved, instead of producing a
  partial lock directory containing only the successful platforms. (ocaml/dune#15981,
  @Alizter)

- Portable lock directories now select one package version across all requested
  platforms, and fail when no common version exists. (ocaml/dune#15982, fixes ocaml/dune#13645,
  fixes ocaml/dune#13647, @Alizter)

- **BREAKING CHANGE:** Reject `:` in context names so they remain portable path
  components (ocaml/dune#16085, @rgrinberg)

- Keep `.t` suffixes in cram test names, including aliases and `applies_to`,
  starting with Dune language 3.25 (ocaml/dune#16146, @rgrinberg)

- Sandbox `ocaml-mdx dune-gen` when using MDX extension 0.6, available from
  Dune 3.25. (ocaml/dune#16177, @rgrinberg)

- Sandbox `pkg-config` queries starting with Dune language 3.25 so they cannot
  observe undeclared workspace inputs. (ocaml/dune#16178, @rgrinberg)

- Remove the experimental OCamlFDO integration, including the `(fdo ...)`
  workspace field (ocaml/dune#16198, @rgrinberg)

- BREAKING CHANGE: Generated files are now read-only even with Dune language
  versions before 2.4, regardless of whether the shared cache is enabled.
  Executable permissions are preserved. (ocaml/dune#16388, @rgrinberg)

- Make newly written shared-cache metadata files read-only to guard against
  accidental modification. (ocaml/dune#16393, @rgrinberg)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@rgrinberg rgrinberg rgrinberg approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@rgrinberg rgrinberg

@robinbb robinbb

Labels

None yet

Projects

None yet

Milestone

3.25.0

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@robinbb @rgrinberg @Alizter
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.