GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing docs.flashinfer.ai — flashinfer.testing - FlashInfer 0.7.4 documentation

flashinfer.testing¶

This module provides comprehensive testing utilities for benchmarking and performance analysis in FlashInfer.

Test Environment Setup¶

`set_seed`(random_seed)	Set random seed for reproducibility during testing.
`sleep_after_kernel_run`(execution_time)	Sleep after kernel run.

Performance Analysis¶

FLOPS Calculation¶

`attention_flops`(batch_size, qo_seqlen, ...)	Calculate FLOPs for a given attention layer.
`attention_flops_with_actual_seq_lens`(...)	Calculate FLOPs for a given attention layer with actual sequence lengths where actual sequence lengths are provided as 1D tensors.
`attention_tflops_per_sec`(batch_size, ...)	Calculate TFLOPS per second for a given attention layer.
`attention_tflops_per_sec_with_actual_seq_lens`(...)	Calculate TFLOPS per second for a given attention layer with actual sequence lengths.

Throughput Analysis¶

`attention_tb_per_sec`(batch_size, qo_seqlen, ...)	Calculate the TB-per-second performance achieved for a given attention layer.
`attention_tb_per_sec_with_actual_seq_lens`(...)	Calculate the TB-per-second performance achieved for a given attention layer with actual sequence lengths.

GPU Benchmarking¶

`bench_gpu_time`(fn[, dry_run_iters, ...])	Unified GPU benchmarking interface with configurable timing backends.
`bench_gpu_time_with_cuda_event`(fn[, ...])	Benchmark kernel execution time using CUDA events (no CUDA graphs).
`bench_gpu_time_with_cudagraph`(fn[, ...])	Benchmark GPU time using CUDA graphs with amortized kernel launch overhead.
`bench_gpu_time_with_cupti`(fn[, ...])	Benchmark GPU time using CUPTI activity tracing for precise kernel timing.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.