Features¶

The easiest way to track the feature set of RADV (and other Vulkan drivers in Mesa) is to take a look at the Mesa matrix.

Contributing to RADV¶

RADV is part of Mesa, so we recommend reading Mesa’s guidelines for submitting patches.

Additionally, the RADV team agreed on the following workflow for contributions:

• Do NOT merge any MRs without at least one approval from someone familiar with the code.

• There are different types of approvals:

  • Reviewed-by tag

  • MR approval button

  • Acked-by tag

• After receiving the first approval, please wait at least 24 hours (excluding weekends) before merging the MR. This is to make sure that anyone who may be interested had opportunity and time to look at the MR (eg. timezone differences). The wait is not required in the following cases:

  • The MR is trivial (eg. very simple cleanups, typos, CI flakes updates)

  • The MR has been approved by two or more developers

• Even if your MR has been approved and is ready to be merged, please make sure to resolve any open conversations about potentially controversial commits.

In case your MR needs to be merged immediately (should be rarely needed), feel free to ping the developers to get more approvals.

If you find a bug or shortcoming in RADV and cannot address it yourself, file an issue on the tracker. LLM-assisted MRs require a good understanding of the codebase, and reviewing them has a real cost. Reviewers/maintainers may close such MRs at their discretion.

LLM-written issues are discouraged: they tend to be verbose and hard to read. Please describe the issue in your own words. LLM-generated reproducers are fine as long as you have reviewed them and they demonstrate the bug.

Supported hardware¶

All GCN and RDNA GPUs that are supported by the Linux kernel (and capable of graphics) are also supported by RADV, starting from GCN 1. We are always working on supporting the very latest GPUs too.

Vulkan API support:

• GFX6-7 (GCN 1-2): Vulkan 1.3

• GFX8 and newer (GCN 3-5 and RDNA): Vulkan 1.4

The exact list of Vulkan conformant products can be seen here.

Each GPU chip can contain various hardware blocks (also known as IP blocks), and each of those are separately versioned. We usually refer to hardware generations by the main version number of the GFX (graphics) hardware block.

Each hardware generation has different chips which usually only have minor differences between each other, such as number of compute units and different minor versions of some IP blocks. We refer to each chip by the code name of its first release, and we don’t differentiate between refreshes of the same chip, because they are functionally exactly the same.

For more information about which GPU chip name corresponds to which GPU product, see the src/amd/common/amd_family.h file.

Notes about GFX6-7 (GCN 1-2) GPUs:

• RADV should work out of the box on GFX6-7 dGPUs as of Linux 6.19 as the default kernel driver was changed to amdgpu.

• RADV should work out of the box on GFX7 APUs as of Linux 7.1 as the default kernel driver was changed to amdgpu.

• RADV is not supported by the older radeon kernel driver, so it won’t work out of the box on older kernels where the default driver is radeon. On these kernel versions, users need to switch to amdgpu manually by adding the following to the kernel command line: radeon.si_support=0 radeon.cik_support=0 amdgpu.si_support=1 amdgpu.cik_support=1

Basics¶

The RADV source code is located in src/amd/vulkan.

RADV is a userspace driver, compiled to a shared library file. On Linux, typically libvulkan_radeon.so (equivalent to a .dll on Windows).

When you start a Vulkan application, the Vulkan loader (in userspace) will find a set of Vulkan drivers (also known as Vulkan implementations), all of which are technically shared libraries. If you are running on a system with a supported AMD GPU and have RADV installed, the loader will find libvulkan_radeon.so and load that. The Vulkan application can then choose which available Vulkan implementation to use.

With RADV, when the application makes Vulkan API calls (aka. entry points), the vk* functions will end up calling radv_* functions. For example, vkCmdDrawMeshTasksEXT will actually call radv_CmdDrawMeshTasksEXT.

Responsibilities of RADV vs. the kernel driver¶

Due to the complexity of how modern GPUs work, the graphics stack is split between kernel-mode drivers (KMD) and user-mode drivers (UMD). All Graphics APIs such as Vulkan, OpenGL, etc. are implemented in userspace.

RADV is a UMD that currently works with the amdgpu KMD in the Linux kernel. Interacting with the KMD is done by RADV’s winsys code.

The KMD is responsible for:

• Talking to the GPU through a PCIe port and power management (such as choosing voltage, frequency, sleep modes etc.)

• Display functionality

• Video memory management (and GTT)

• Writing submitted commands to the GPU’s ring buffers

The UMD (in our case, RADV) is responsible for:

• Recording commands in a binary format that the GPU can understand

• Programming GPU registers for correct functionality

• Compiling shaders to the GPU’s native ISA (instruction set architecture) and uploading them to memory accessible by the GPU

• Submitting recorded commands to the kernel through a system call

To communicate with amdgpu, RADV relies on the DRM userspace API (uAPI) of the Linux kernel, which is a set of system calls. RADV depends on libdrm for some functionality and for others it uses the system calls directly.

Command submission and PM4 packets¶

Vulkan applications record a series of commands in command buffers and later submit these command buffers to one of the queues in the GPU.

Command buffer recording in RADV is implemented by emitting packets in a buffer, which is called a command stream (CS).

Command packets are more or less analogous to Vulkan API calls, which means that each Vulkan command (such as draw or dispatch) corresponds to one or more packets. However, depending on how closely the hardware follows Vulkan spec some commands will require a more complex implementation. For example, a simple vkCmdDraw call will result in just one packet, however emitting draw state (before the draw) may take dozens of packets.

For the graphics queue (GFX) and async compute queue (ACE), the PM4 packet format is used; other queues such as SDMA and the various video queues have their own format. Commands typically vary between different GPU generations.

When submitting to a queue, several CSs are submitted to the kernel at once. The kernel terminology calls these indirect buffers (IB) because the kernel typically uses the INDIRECT_BUFFER PM4 command to execute them. Modern AMD GPUs have several queues, which more or less map to the Vulkan queue types, though sometimes we need to submit to more than one HW queue at the same time.

After command submission, the packets in the IBs will be executed by the command processor (CP) of the queue that the buffer was submitted to. The exact capabilities and supported commands of the CP depend on HW generation and queue type.

Each CP has a collection of registers that control how the CP behaves. Registers in the CP are not to be confused with registers in shaders. For example, the addresses of shader programs, some details of shader execution, draw call information etc. have a corresponding register. These registers can be typically set by PM4 packets.

Shader compilation¶

One of the main responsibilities of RADV is compiling shaders for Vulkan applications.

RADV relies on the Mesa shader compiler stack. Here is a rough overview of how it works:

• Vulkan applications pass shaders to RADV in the SPIR-V format

• RADV calls spirv_to_nir which translates the shader into the NIR intermediate representation

• We perform various optimizations and lowerings on the NIR shader

• The shader is linked with the other shaders it is compiled with (if any)

• Still using NIR, the shader is further lowered to be closer to the hardware

• Finally, we pass the lowered NIR shader to ACO, our compiler backend, which compiles it into GPU specific ISA

ACO is the default shader-compiler used in RADV. Read its documentation here.

We still maintain an LLVM based compiler backend too, which is these days solely used for testing and hardware bringup. Users are recommended NOT to use the LLVM backend.

Shader execution¶

Some commands (such as draws and dispatches) ask the CP to launch shaders. Shader launch is handled by the firmware, based on registers that control shader programs. Additionally, draw commands will also automatically use the appropriate fixed function units in the hardware.

Shaders are executed by a so-called compute unit on the GPU, which is a SIMD machine. A shader invocation is a single SIMD lane (AMD also calls it thread, not to be confused by a CPU HW thread), and a subgroup is all 64 or 32 SIMD lanes together (also known as a wave). Each wave is a separate running instance of a shader program, but multiple waves can be grouped together into workgroups.

Registers in shaders (not to be confused with registers in the CP):

• VGPR - vector general purpose register: each SIMD lane has a different value for this register

• SGPR - scalar general purpose register: same value within a wave

For further reading, AMD has publised whitepapers and documentation for the GCN and RDNA GPU architectures. These can be found on their GPUOpen site.

Debugging¶

Performance¶

Features¶

Miscellaneous¶