GeezerVison is showing codspeed.io — CodSpeed CLI - CodSpeed Docs The CodSpeed CLI allows you to benchmark any executable program and collect
performance data.
It is open source, and its source code is available on
GitHub.
Getting Started
Installation
Install the CodSpeed CLI using the installation script:
curl -fsSL https://codspeed.io/install.sh | sh
codspeed auth login.
Installing a specific version
To install a specific version:curl -fsSL https://codspeed.io/v5.1.0/install.sh | sh
v5.1.0 with the version you want to install.You can find all available versions on the
releases page. The CodSpeed CLI officially supports Ubuntu 22.04, 24.04, 26.04 and Debian 12.
Other Linux distributions can be used as long as the required tools for your
chosen instruments are manually installed and
available in your PATH. Run your first command
Let’s run a simple benchmark using the walltime instrument:
$ codspeed exec -- sleep 1
►►► Running the benchmarks
Executing: sleep 1
Completed 2 warmup rounds
Warmup done, now performing 3 rounds
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
sleep 1: 1.01 s
To see the full report, visit: https://app.codspeed.io/[...]
Configuration
You can define benchmarks and options in a codspeed.yml configuration file.
Most importantly, items in the benchmarks list describe the commands that are
run when using codspeed run.
Other supported filenames
The CodSpeed CLI looks for configuration files with the following names, by order of priority:
codspeed.ymlcodspeed.yaml.codspeed.yml.codspeed.yaml You can also specify a custom path with --config <path>. Benchmark fields
The command to execute. Can include arguments and flags. A descriptive name for the benchmark. This is used to identify the benchmark
in reports and results.When omitted, the command is used as the benchmark name. Override global instrument options for this specific benchmark.See the Instruments section for available options for each
instrument and their effects. Running benchmarks
To run all benchmarks defined in your configuration file, specify the instrument
with the -m flag:
$ codspeed run -m walltime
►►► Running the benchmarks
Executing: <commmand 1>
Completed 13 warmup rounds
Warmup done, now performing 20 rounds
Executing: <command 2>
A single warmup execution (517.ms) exceeded or met max_time (200ms).
Executing: <command 3>
Completed 23 warmup rounds
Warmup done, now performing 68 rounds
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
┌──────────────┬─────────────┐
│ Benchmark │ Measurement │
├──────────────┼─────────────┤
│ <command 1> │ 517.23 ms │
├──────────────┼─────────────┤
│ <command 2> │ 14.10 ms │
├──────────────┼─────────────┤
│ <command 3> │ 6.91 ms │
└──────────────┴─────────────┘
To see the full report, visit: https://app.codspeed.io/[...]
codspeed exec to run a single command:
$ codspeed exec -m walltime --warmup-time 1ms --max-time 2s -- ./my-binary --arg1 value --arg2
►►► Running the benchmarks
Executing: ./my-binary --arg1 value --arg2
Completed 1 warmup rounds
Warmup done, now performing 115 rounds
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
./my-binary --arg1 value --arg2: 14.01 ms
To see the full report, visit: https://app.codspeed.io/[...]
Example configuration
$schema: https://raw.githubusercontent.com/CodSpeedHQ/codspeed/refs/heads/main/schemas/codspeed.schema.json
# Global options applied to all benchmarks
options:
warmup-time: "0.2s"
max-time: 1s
# List of benchmarks to run
benchmarks:
- name: "<command 1>"
exec: ./my_binary --arg1 value --arg2
# Override global options for this benchmark
options:
max-rounds: 20
- name: "<command 2>"
exec: ./my_binary2
options:
max-time: 200ms
- name: "<command 3>"
exec: ./my_binary3 --flag
options:
min-time: 200ms
As displayed in the above example, we provide a schema for the configuration file
here.If your editor supports it, you can use the $schema field to enable
autocompletion and validation for the codspeed.yml file.Please note that by default, the VSCode yaml language server does not support
declaring the schema through the $schema field. One of the ways to declare the
schema as is to add# yaml-language-server: $schema=https://raw.githubusercontent.com/CodSpeedHQ/codspeed/refs/heads/main/schemas/codspeed.schema.json
Instruments
CodSpeed supports three instruments for measuring different aspects of your
program’s performance.
Walltime
The walltime instrument measures real-world execution time. It runs your
benchmark multiple times and collects timing data. See the
Walltime documentation for more details.
How it works
-
Warmup phase: The benchmark runs repeatedly until the warmup time is
reached. This allows the system to reach a steady state.
-
Measurement phase: Based on the average time from warmup, the CodSpeed
CLI calculates how many rounds to run within the configured time bounds, then
collects timing data for each round.
The walltime instrument requires sudo privileges to run because it needs to
configure the kernel so that linux
perf can collect all the
necessary events.We plan to fine tune permissions in the future to reduce permissions to the
strict minimum. Profiling for interpreted languages (e.g., Python, JavaScript) is not
currently supported with the walltime instrument.We are currently working on it and it’s coming soon! Walltime Options
Time spent in warmup runs. Set to 0 to disable warmup. Minimum time spent running measurement rounds. Maximum time spent running measurement rounds. Minimum number of measurement rounds. Maximum number of measurement rounds. If there is a conflict between time and round constraints, the CodSpeed CLI
prioritizes satisfying the maximum bounds first. Example
$ codspeed exec -m walltime --warmup-time 100ms --max-time 1s -- <command>
►►► Running the benchmarks
Executing: <command>
Completed 7 warmup rounds
Warmup done, now performing 63 rounds
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
<command>: 14.15 ms
To see the full report, visit:
https://app.codspeed.io/[...]
benchmarks:
- name: My command
exec: <command>
options:
warmup_time: 100ms
max_time: 1s
$ codspeed run
►►► Running the benchmarks
Executing: My command
Completed 7 warmup rounds
Warmup done, now performing 63 rounds
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
<command>: 14.15 ms
To see the full report, visit:
https://app.codspeed.io/[...]
Memory
The memory instrument tracks heap allocations during benchmark execution.
Unlike walltime, the benchmark runs only once while memory usage is
recorded. See the Memory documentation for more details.
It measures:
- Peak memory usage
- Total allocated memory
- Allocation count
- Average allocation size
See Supported Allocators for the
list of allocators that can be tracked.
The memory instrument requires sudo privileges to run because it uses
eBPF to track memory allocations.We plan to fine tune permissions in the future to reduce permissions to the
strict minimum. Example
$ codspeed exec -m memory -- <command>
►►► Running the benchmarks
Executing: <command>
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
<command>: 7.8 MB
To see the full report, visit: https://app.codspeed.io/[...]
Simulation
The simulation instrument uses CPU simulation to measure performance. The
benchmark runs only once and CPU behavior is simulated, providing consistent
measurements independent of system load. See the
CPU Simulation documentation for more details.
This is the same instrument used by
CodSpeed integrations when running in CI.
Profiling for interpreted languages (e.g., Python, JavaScript) is not
currently supported with the simulation instrument.We are currently working on it and it’s coming soon! Limitations
Statically linked executables are not currently supported with the simulation
instrument. If this is a limitation for your use case, please
open a GitHub issue.
Additionally, the simulation instrument currently does not follow child
processes. Support for this is planned for a future release.
Example
$ codspeed exec -m simulation -- <my_command>
►►► Running the benchmarks
Executing: <my_command>
►►► Uploading results
Linked repository: [...]
Performance data uploaded
►►► Benchmark results
<my_command>: 26.74 ms
To see the full report, visit: https://app.codspeed.io/[...]
Setting instrument with codspeed use
If you do not want to always explicitly specify the instrument with -m when
running benchmarks, you can set an instrument for your whole shell session by
using
$ codspeed use simulation
codspeed run and codspeed exec commands in the current shell session.
You can check the currently selected instrument with
$ codspeed show
Simulation
Even if you ran codspeed use <instrument>, you can still override the
instrument for a specific command by using the -m flag with codspeed run
or codspeed exec. Running the benchmarks in your CI
To generate performance reports, you need to run the benchmarks in your CI. This
allows CodSpeed to automatically run benchmarks and warn you about regressions
during development.
Here is an example of a GitHub Actions workflow that runs the benchmarks and
reports the results to CodSpeed on every push to the main branch and every
pull request:
.github/workflows/codspeed.yml name: CodSpeed Benchmarks
on:
push:
branches:
- "main" # or "master"
pull_request:
# `workflow_dispatch` allows CodSpeed to trigger backtest
# performance analysis in order to generate initial data.
workflow_dispatch:
jobs:
benchmarks:
name: Run benchmarks
runs-on: ubuntu-latest
permissions: # optional for public repositories
contents: read # required for actions/checkout
id-token: write # required for OIDC authentication with CodSpeed
steps:
- uses: actions/checkout@v7
# ...
# Setup your environment here:
# - Configure your Python/Rust/Node version
# - Install your dependencies
# - Build your executable (if using a compiled language)
# ...
- name: Run the benchmarks
uses: CodSpeedHQ/action@v5
with:
mode: simulation
Contrary to other CI usages, the run input is intentionally omitted here.This is what defines whether the action will run benchmarks defined in your
configuration file, or a benchmark that makes use of one of the
CodSpeed integrations. Manual tool installation
On Ubuntu and Debian, the CodSpeed CLI automatically installs the tools required
by each instrument. On other distributions, you need to install them manually.
Walltime
The walltime instrument uses
linux perf for profiling,
which is enabled by default. Refer to your distribution’s documentation for how
to install the perf tool.
If you don’t need profiling data, you can disable it by setting the
CODSPEED_PERF_ENABLED environment variable to false.
Simulation
The simulation instrument requires
CodSpeed’s fork of Valgrind.
You need to build and install it from source.
Prerequisites: git, autoconf, automake, make, and a C compiler.
git clone https://github.com/CodSpeedHQ/valgrind-codspeed.git
cd valgrind-codspeed
./autogen.sh
./configure
make -j
make install
PATH:
$ valgrind --version
valgrind-3.26.0.codspeed
valgrind binary in your PATH must be the CodSpeed version for the
simulation instrument to work correctly.
Memory
The memory instrument requires a Linux kernel with eBPF support. No additional
tool installation is needed, the CLI will download and load the necessary tools
automatically.
Next Steps
ProfilingLearn how to use flamegraphs and profiling data to optimize your code InstrumentsDive into instruments and learn how to measure different aspects of your
program’s performance |