Optuna for .NET

OptunaSharp is a faithful and complete .NET port of Optuna, an automatic hyperparameter optimization software framework, particularly designed for machine learning.

OptunaSharp is thread-safe and truly parallel.
nJobs runs trials on all your cores at once, with no GIL in the way, and workers on
many machines share one study, Python Optuna workers included.
OptunaSharp supports parallelism powered by:
single-threaded, reused dedicated threads and ThreadPool async-await.

OptunaSharp is cloud-ready, with supported integrations for:
Artifact storage - Amazon S3 · Azure Blob Storage · Google Cloud Storage · Cloudflare R2
Study storage - PostgreSQL · MySQL · Redis · gRPC storage proxy

The compatibility target is Optuna 5.x. When OptunaSharp behavior and Optuna
behavior differ, Optuna is treated as the source of truth and OptunaSharp aligns to it.

OptunaSharp is a complete 100% port of Optuna 5.x with full and backwards compatability with optuna's integrations.

• Define-by-run API - Study, Trial and FrozenTrial with every suggest_*, Optimize, ask-and-tell, enqueued trials, multi-objective directions, constraints, user attributes and callbacks; float, int and categorical distributions with search-space inference. All of it lives in the pure-managed Core package, built on NumSharp.

• Samplers - Optuna's Random, Grid, TPE, CMA-ES, GP, QMC, BruteForce, PartialFixed, NSGA-II and NSGA-III, plus samplers Optuna does not have: AGE-MOEA-2, SMS-EMOA, MOEA/D, RVEA, LM-CMA, VkD-CMA, differential evolution, PSO, the natural evolution strategies (OpenAI-ES, SNES, xNES), TuRBO, REMBO, SAASBO and JANUS (with its DE and multi-objective variants).

  • Integrations - a native scikit-learn-style OptunaSearchCV, LightGBMTuner, ML.NET, training-loop pruning callbacks, Microsoft.Extensions.DependencyInjection registrations, and the optunasharp command-line tool, a port of the optuna CLI.

• Python bridges - OptunaHub packages, BoTorch, pycma and pymoo, driven in-process through pythonnet.

• Pruners - Median, Percentile, SuccessiveHalving, Hyperband, Threshold, Patient, Wilcoxon and Nop.

• Analysis - parameter importance (fANOVA, mean decrease impurity, PED-ANOVA and SHAP, all pure-managed and byte-exact), terminators, and the data behind all 12 plot_* functions, which the Plotly.NET package draws as interactive figures.

• Storage - in-memory and journal-file storage in Core; SQLite, PostgreSQL, MySQL, Redis journals and a gRPC storage proxy as companion packages. Python Optuna can work on the same study through the journal file, any of the databases, Redis or the gRPC proxy. On the SQLite, PostgreSQL and MySQL storages, heartbeats with RetryFailedTrialCallback retry the trials of a worker that died.

• Artifacts - a file-system store in Core; Amazon S3, Google Cloud Storage, Azure Blob Storage and Cloudflare R2 as companion packages.

• Parallel and thread-safe - Optimize(nJobs: n) runs n trials at once, with no GIL in the way. Measured on 32 logical CPUs, 64 trials of a ~50 ms CPU-bound objective finished 6.7× faster at nJobs: 8 and 9.2× faster at nJobs: 16 than at nJobs: 1, while Python Optuna's time stays flat because the GIL runs one Python thread at a time (Performance).

  • Trials run on OptunaSharp's own dedicated threads by default, or on the .NET ThreadPool.
  • Sampling is serial per sampler, as in Optuna, unless SamplerConcurrency.PerWorker gives each worker its own Random, TPE, CMA-ES, VkD-CMA or Grid sampler.
  • A CancellationToken, or StopSignals for Ctrl+C and SIGTERM, stops a run gracefully: no new trial starts, and the trials in flight finish and are stored.
  • Studies, trials, samplers, pruners and storages are safe to share between threads; docs/THREADING.md has the design, its guarantees and its limits.

• Reproducible and tested - a seeded run is bit-identical to Python wherever the path is deterministic, enforced by a live-Optuna parity harness (pythonnet) that asks real Optuna for each answer in-process and asserts that OptunaSharp matches. Everything builds and tests on net8.0 and net10.0 across Windows, Linux and macOS, behind strong-naming and package gates, and all 20 packages release together from one tag.

OptunaSharp has reimplemented byte-for-byte many cutting-edge papers and state of the art algorithms and will continue to doing so.
Found an incredible paper? You believe an optimizer/sampler/technique has achieved a breakthrough or has been referenced more than 50 times?

AGE-MOEA-2, SMS-EMOA, MOEA/D, RVEA, LM-CMA, VkD-CMA, differential evolution, PSO, the natural evolution strategies (OpenAI-ES, SNES, xNES), TuRBO, REMBO, SAASBO and JANUS (with its DE and multi-objective variants).

And we would love to accept more!

Optuna is an automatic hyperparameter optimization framework with an imperative, define-by-run API. The two core concepts:

• Study - an optimization session over an objective function.
• Trial - a single evaluation of the objective, during which hyperparameters are suggested.

# Optuna (Python) - the behavior OptunaSharp targets
import optuna

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2

study = optuna.create_study(sampler=optuna.samplers.TPESampler(seed=42))
study.optimize(objective, n_trials=20)
print(study.best_value, study.best_params)

The C# quick start above is the same study, written against the same define-by-run shape: snake_case names become PascalCase (suggest_float → SuggestFloat, n_trials → nTrials), while parameter order, defaults and error messages are kept as Optuna has them.

The documentation site, scisharp.github.io/OptunaSharp:

• Start here - Introduction · Getting Started · Key Features · FAQ
• Guides - Study and Trials · Distributions · Search Spaces · Samplers · Pruners · Multi-objective Optimization · Storage · Artifacts · Terminators · Parameter Importance · Visualization · Logging · Exceptions
• Recipes - Overview · Saving/Resuming with RDB · Multi-objective Optimization · User Attributes · Command-Line Interface · User-Defined Sampler · User-Defined Pruner · Callback for Study.Optimize · Specify Hyperparameters Manually · Ask-and-Tell Interface · Re-use the Best Trial · Journal-File Storage · Artifacts · Wilcoxon Pruner · Ablation Study with Brute Force
• Integrations - Overview
  • Model training - scikit-learn-style search · ML.NET · LightGBM · Training-loop pruning
  • Storage - SQLite · EF Core (PostgreSQL / MySQL) · Redis · gRPC
  • Python bridges - Python.NET runtime · OptunaHub · BoTorch · PyCma · pymoo
  • Cloud artifacts - Amazon S3 · Google Cloud Storage · Azure Blob Storage · Cloudflare R2
  • Tools - Plotly.NET figures · Command-line tool
• Methods - Overview: a page per algorithm, each with its paper - the samplers, pruners, importance evaluators and terminators, and the operators and kernels they are built from.
• API reference - OptunaSharp, generated from the XML documentation of the Core package.

Measured on an Intel Core i9-13900K (8 performance and 16 efficiency cores, 32 logical CPUs) under Windows 11: OptunaSharp in .NET 10 Release builds, and Python 3.12.12 with Optuna 5.0.0 on the same machine. Each configuration ran in its own process, and times are medians of three runs unless a table says otherwise. Every benchmark process opted out of Windows power throttling, which otherwise moves a busy console process onto the efficiency cores after a few seconds. Absolute times depend on the hardware, so compare within a row.

64 trials of a CPU-bound objective that takes about 50 ms on one core, with RandomSampler. Each language's loop was calibrated to ~50 ms on its own; OptunaSharp ran the sweep twice.

Python's time stays flat because the GIL runs one Python thread at a time. Trials that wait rather than compute run all at once in both: 640 trials sleeping 100 ms took 1.26 s at nJobs: 64, and 1,280 took 1.27 s at nJobs: 128 (Python: 1.06 s and 1.13 s; ideal 1.0 s).

One trial at a time, the same seeded study in both languages, on a cheap sphere objective so that the sampler's own work dominates. OptunaSharp's times include the .NET JIT warming up.

Random sampling over 1,000 parameters is the one measured case where OptunaSharp is slower.

Trials run on OptunaSharp's own dedicated threads by default, or on the .NET ThreadPool (TrialExecutorKind.DedicatedThreads or TrialExecutorKind.ThreadPool). Three runs of each:

The two match on throughput. Once nJobs reaches the pool's ready threads (32 on this machine), the ThreadPool runs fewer trials at once and delays the rest of the application, so it suits an nJobs well below the core count.

nJobs: 8, best of three runs: one sampler shared by every worker, taking one call at a time (SamplerConcurrency.Shared, Optuna's behaviour and the default), against one sampler per worker (SamplerConcurrency.PerWorker):

A sampler per worker pays where sampling dominates and changes nothing when the objective does. Slow model-based samplers stay shared: a GPSampler study of 120 trials with a 20 ms objective took 7.53 s at nJobs: 1 and 9.30 s at nJobs: 8.

dotnet add package OptunaSharp

The package targets .NET 8 and .NET 10. Storages, cloud artifact stores, Plotly figures, the CLI and the Python bridges are separate OptunaSharp.* packages (listed in the package README).

using OptunaSharp;
using OptunaSharp.Samplers;

var study = Optuna.CreateStudy(sampler: new TPESampler(seed: 42));

study.Optimize(trial =>
{
    double x = trial.SuggestFloat("x", -10, 10);
    return (x - 2) * (x - 2);
}, nTrials: 20);

Console.WriteLine(study.BestValue);        // 0.0004973004749973951
Console.WriteLine(study.BestParams["x"]);  // 2.0223002348641756

Those two numbers are exactly what Python Optuna 5.0.0 prints for the same seeded study.

OptunaSharp.Core depends on NumSharp and its native LAPACK/CBLAS backend NumSharp.Interop.OpenBLAS, both consumed as NuGet packages from nuget.org (version pinned by the NumSharpVersion property in Directory.Build.props). Build and CI restore them from nuget.org; the OpenBLAS package ships the native LAPACK binaries (in runtimes/<rid>/native/), so np.linalg computes through LAPACK and is bit-identical to NumPy - no extra setup.

The matching NumSharp source is vendored reference-only as a pinned git submodule at refs/NumSharp (the same release tag), for browsing - it is not part of the build. Restore it, if you want it, with git submodule update --init refs/NumSharp.

# Build everything
dotnet build SciSharp.OptunaSharp.sln -c Release

# Unit tests (both TFMs), excluding known-failing/heavy/slow categories as per-push CI does
# (drop `&TestCategory!=Slow` for the full run)
dotnet test test/OptunaSharp.Tests/OptunaSharp.Tests.csproj -c Release \
  --filter "TestCategory!=OpenBugs&TestCategory!=HighMemory&TestCategory!=RequiresPython&TestCategory!=Slow"

# Live-Optuna interop: needs a discoverable Python with the vendored Optuna installed, which is what
# the parity is measured against (`git submodule update --init refs/optuna`, then
# `pip install -e refs/optuna`)
dotnet test test/OptunaSharp.Tests.Interop/OptunaSharp.Tests.Interop.csproj -c Release

# Pack the OptunaSharp package
dotnet pack src/OptunaSharp.Core/OptunaSharp.Core.csproj -c Release -o packages

Test categories (OpenBugs, Misaligned, RequiresPython, HighMemory, WindowsOnly, Slow) are typed attributes in test/OptunaSharp.Tests/TestCategory.cs, so CI filter strings stay stable as the suite grows.

Internal members are exposed to a dotnet run script that overrides its assembly name to a declared friend (see src/OptunaSharp.Core/Assembly/Properties.cs). A script under the repo root inherits Directory.Build.props and is signed automatically; one run from elsewhere must pass the signing properties itself:

#:project <path-to-your-clone>/src/OptunaSharp.Core/OptunaSharp.Core.csproj
#:property AssemblyName=OptunaSharp.DotNetRunScript
#:property PublishAot=false
#:property SignAssembly=true
#:property AssemblyOriginatorKeyFile=<path-to-your-clone>/Open.snk

MIT - see LICENSE. OptunaSharp is a derivative of Optuna (© Preferred Networks, Inc.) and retains its copyright notice as the MIT License requires.

• NumSharp - NumPy for .NET (OptunaSharp's numerical foundation)
• The SciSharp STACK - open-source machine learning for .NET