GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing www.kernel.org — drm/i915 Intel GFX Driver — The Linux Kernel documentation drm/i915 Intel GFX Driver¶The drm/i915 driver supports all (with the exception of some very early models) integrated GFX chipsets with both Intel display and rendering blocks. This excludes a set of SoC platforms with an SGX rendering unit, those have basic support through the gma500 drm driver. Core Driver Infrastructure¶This section covers core driver infrastructure used by both the display and the GEM parts of the driver. Runtime Power Management¶The i915 driver supports dynamic enabling and disabling of entire hardware blocks at runtime. This is especially important on the display side where software is supposed to control many power gates manually on recent hardware, since on the GT side a lot of the power management is done by the hardware. But even there some manual control at the device level is required. Since i915 supports a diverse set of platforms with a unified codebase and hardware engineers just love to shuffle functionality around between power domains there’s a sizeable amount of indirection required. This file provides generic functions to the driver for grabbing and releasing references for abstract power domains. It then maps those to the actual power wells present for a given platform.
Parameters
Description This is the unlocked version of intel_display_power_is_enabled() and should only be used from error capture and recovery code where deadlocks are possible. This function grabs a device-level runtime pm reference (mostly used for asynchronous PM management from display code) and ensures that it is powered up. Raw references are not considered during wakelock assert checks. Any runtime pm reference obtained by this function must have a symmetric
call to Return the wakeref cookie to pass to
Parameters
Description This function grabs a device-level runtime pm reference (mostly used for GEM code to ensure the GTT or GT is on) and ensures that it is powered up. Any runtime pm reference obtained by this function must have a symmetric
call to Return the wakeref cookie to pass to
Parameters
Description This function grabs a device-level runtime pm reference if the device is already active and ensures that it is powered up. It is illegal to try and access the HW should intel_runtime_pm_get_if_active() report failure. If ignore_usecount is true, a reference will be acquired even if there is no user requiring the device to be powered up (dev->power.usage_count == 0). If the function returns false in this case then it’s guaranteed that the device’s runtime suspend hook has been called already or that it will be called (and hence it’s also guaranteed that the device’s runtime resume hook will be called eventually). Any runtime pm reference obtained by this function must have a symmetric
call to Return the wakeref cookie to pass to
Parameters
Description This function grabs a device-level runtime pm reference (mostly used for GEM code to ensure the GTT or GT is on). It will _not_ power up the device but instead only check that it’s powered on. Therefore it is only valid to call this functions from contexts where the device is known to be powered up and where trying to power it up would result in hilarity and deadlocks. That pretty much means only the system suspend/resume code where this is used to grab runtime pm references for delayed setup down in work items. Any runtime pm reference obtained by this function must have a symmetric
call to Return the wakeref cookie to pass to
Parameters
Description This function drops the device-level runtime pm reference obtained by
Parameters
Description This function drops the device-level runtime pm reference obtained by
This function exists only for historical reasons and should be avoided in
new code, as the correctness of its use cannot be checked. Always use
Parameters
Description This function drops the device-level runtime pm reference obtained by
Parameters
Description This function enables runtime pm at the end of the driver load sequence. Note that this function does currently not enable runtime pm for the subordinate display power domains. That is done by intel_power_domains_enable().
Parameters
Description This function can be used get GT’s forcewake domain references. Normal register access will handle the forcewake domains automatically. However if some sequence requires the GT to not power down a particular forcewake domains this function should be called at the beginning of the sequence. And subsequently the reference should be dropped by symmetric call to intel_unforce_forcewake_put(). Usually caller wants all the domains to be kept awake so the fw_domains would be then FORCEWAKE_ALL.
Parameters
Description This function is a wrapper around
Parameters
Description This function complements
Parameters
Description See
Parameters
Description This function drops the device-level forcewakes for specified
domains obtained by
Parameters
Parameters
Description See
Parameters
Description This routine waits until the target register reg contains the expected value after applying the mask, i.e. it waits until (intel_uncore_read_fw(uncore, reg) & mask) == value Otherwise, the wait will timeout after slow_timeout_ms milliseconds. For atomic context slow_timeout_ms must be zero and fast_timeout_us must be not larger than 20,0000 microseconds. Note that this routine assumes the caller holds forcewake asserted, it is not suitable for very long waits. See intel_wait_for_register() if you wish to wait without holding forcewake for the duration (i.e. you expect the wait to be slow). Return 0 if the register matches the desired condition, or -ETIMEDOUT.
Parameters
Description This routine waits until the target register reg contains the expected value after applying the mask, i.e. it waits until (intel_uncore_read(uncore, reg) & mask) == value Otherwise, the wait will timeout after timeout_ms milliseconds. Return 0 if the register matches the desired condition, or -ETIMEDOUT.
Parameters
Description Returns a set of forcewake domains required to be taken with for example intel_uncore_forcewake_get for the specified register to be accessible in the specified mode (read, write or read/write) with raw mmio accessors. NOTE On Gen6 and Gen7 write forcewake domain (FORCEWAKE_RENDER) requires the callers to do FIFO management on their own or risk losing writes. Interrupt Handling¶These functions provide the basic support for enabling and disabling the interrupt handling support. There’s a lot more functionality in i915_irq.c and related files, but that will be described in separate chapters.
Parameters
Description This function initializes all the irq support including work items, timers and all the vtables. It does not setup the interrupt itself though.
Parameters
Description This function is used to disable interrupts at runtime, both in the runtime pm and the system suspend/resume code.
Parameters
Description This function is used to enable interrupts at runtime, both in the runtime pm and the system suspend/resume code. Intel GVT-g Guest Support(vGPU)¶Intel GVT-g is a graphics virtualization technology which shares the GPU among multiple virtual machines on a time-sharing basis. Each virtual machine is presented a virtual GPU (vGPU), which has equivalent features as the underlying physical GPU (pGPU), so i915 driver can run seamlessly in a virtual machine. This file provides vGPU specific optimizations when running in a virtual machine, to reduce the complexity of vGPU emulation and to improve the overall performance. A primary function introduced here is so-called “address space ballooning” technique. Intel GVT-g partitions global graphics memory among multiple VMs, so each VM can directly access a portion of the memory without hypervisor’s intervention, e.g. filling textures or queuing commands. However with the partitioning an unmodified i915 driver would assume a smaller graphics memory starting from address ZERO, then requires vGPU emulation module to translate the graphics address between ‘guest view’ and ‘host view’, for all registers and command opcodes which contain a graphics memory address. To reduce the complexity, Intel GVT-g introduces “address space ballooning”, by telling the exact partitioning knowledge to each guest i915 driver, which then reserves and prevents non-allocated portions from allocation. Thus vGPU emulation module only needs to scan and validate graphics addresses without complexity of address translation.
Parameters
Description This function is called at the initialization stage, to detect whether running on a vGPU.
Parameters
Description This function is called to deallocate the ballooned-out graphic memory, when driver is unloaded or when ballooning fails.
Parameters
Description This function is called at the initialization stage, to balloon out the graphic address space allocated to other vGPUs, by marking these spaces as reserved. The ballooning related knowledge(starting address and size of the mappable/unmappable graphic memory) is described in the vgt_if structure in a reserved mmio range. To give an example, the drawing below depicts one typical scenario after ballooning. Here the vGPU1 has 2 pieces of graphic address spaces ballooned out each for the mappable and the non-mappable part. From the vGPU1 point of view, the total size is the same as the physical one, with the start address of its graphic space being zero. Yet there are some portions ballooned out( the shadow part, which are marked as reserved by drm allocator). From the host point of view, the graphic address space is partitioned by multiple vGPUs in different VMs. vGPU1 view Host view
0 ------> +-----------+ +-----------+
^ |###########| | vGPU3 |
| |###########| +-----------+
| |###########| | vGPU2 |
| +-----------+ +-----------+
mappable GM | available | ==> | vGPU1 |
| +-----------+ +-----------+
| |###########| | |
v |###########| | Host |
+=======+===========+ +===========+
^ |###########| | vGPU3 |
| |###########| +-----------+
| |###########| | vGPU2 |
| +-----------+ +-----------+
unmappable GM | available | ==> | vGPU1 |
| +-----------+ +-----------+
| |###########| | |
| |###########| | Host |
v |###########| | |
total GM size ------> +-----------+ +-----------+
Return zero on success, non-zero if configuration invalid or ballooning failed Intel GVT-g Host Support(vGPU device model)¶Intel GVT-g is a graphics virtualization technology which shares the GPU among multiple virtual machines on a time-sharing basis. Each virtual machine is presented a virtual GPU (vGPU), which has equivalent features as the underlying physical GPU (pGPU), so i915 driver can run seamlessly in a virtual machine. To virtualize GPU resources GVT-g driver depends on hypervisor technology e.g KVM/VFIO/mdev, Xen, etc. to provide resource access trapping capability and be virtualized within GVT-g device module. More architectural design doc is available on https://01.org/group/2230/documentation-list.
Parameters
Description This function is called at the i915 options sanitize stage.
Parameters
Description This function is called at the initialization stage to create a GVT device. Return Zero on success, negative error code if failed.
Parameters
Description This function is called at the i915 driver unloading stage, to shutdown GVT components and release the related resources.
Parameters
Description This function is called at the i915 driver resume stage to restore required HW status for GVT so that vGPU can continue running after resumed. Workarounds¶This file is intended as a central place to implement most 1 of the required workarounds for hardware to work as originally intended. They fall in five basic categories depending on how/when they are applied:
Layout¶Keep things in this file ordered by WA type, as per the above (context, GT, display, register whitelist, batchbuffer). Then, inside each type, keep the following order:
Display Hardware Handling¶This section covers everything related to the display hardware including the mode setting infrastructure, plane, sprite and cursor handling and display, output probing and related topics. Mode Setting Infrastructure¶The i915 driver is thus far the only DRM driver which doesn’t use the common DRM helper code to implement mode setting sequences. Thus it has its own tailor-made infrastructure for executing a display configuration change. Frontbuffer Tracking¶Many features require us to track changes to the currently active frontbuffer, especially rendering targeted at the frontbuffer. To be able to do so we track frontbuffers using a bitmask for all possible
frontbuffer slots through On a high level there are two types of powersaving features. The first one work like a special cache (FBC and PSR) and are interested when they should stop caching and when to restart caching. This is done by placing callbacks into the invalidate and the flush functions: At invalidate the caching must be stopped and at flush time it can be restarted. And maybe they need to know when the frontbuffer changes (e.g. when the hw doesn’t initiate an invalidate and flush on its own) which can be achieved with placing callbacks into the flip functions. The other type of display power saving feature only cares about busyness (e.g. DRRS). In that case all three (invalidate, flush and flip) indicate busyness. There is no direct way to detect idleness. Instead an idle timer work delayed work should be started from the flush and flip functions and cancelled as soon as busyness is detected.
Parameters
Description This function gets called every time rendering on the given object starts and frontbuffer caching (fbc, low refresh rate for DRRS, panel self refresh) must be invalidated. For ORIGIN_CS any subsequent invalidation will be delayed until the rendering completes or a flip on this frontbuffer plane is scheduled.
Parameters
Description This function gets called every time rendering on the given object has completed and frontbuffer caching can be started again.
Parameters
Description This function gets called every time rendering on the given planes has completed and frontbuffer caching can be started again. Flushes will get delayed if they’re blocked by some outstanding asynchronous rendering. Can be called without any locks held.
Parameters
Description This function gets called after scheduling a flip on obj. The actual frontbuffer flushing will be delayed until completion is signalled with intel_frontbuffer_flip_complete. If an invalidate happens in between this flush will be cancelled. Can be called without any locks held.
Parameters
Description This function gets called after the flip has been latched and will complete on the next vblank. It will execute the flush if it hasn’t been cancelled yet. Can be called without any locks held.
Parameters
Description This function gets called after scheduling a flip on obj. This is for synchronous plane updates which will happen on the next vblank and which will not get delayed by pending gpu rendering. Can be called without any locks held.
Parameters
Description This updates the frontbuffer tracking bits frontbuffer_bits by clearing them from old and setting them in new. Both old and new can be NULL. Display FIFO Underrun Reporting¶The i915 driver checks for display fifo underruns using the interrupt signals provided by the hardware. This is enabled by default and fairly useful to debug display issues, especially watermark settings. If an underrun is detected this is logged into dmesg. To avoid flooding logs and occupying the cpu underrun interrupts are disabled after the first occurrence until the next modeset on a given pipe. Note that underrun detection on gmch platforms is a bit more ugly since there is no interrupt (despite that the signalling bit is in the PIPESTAT pipe interrupt register). Also on some other platforms underrun interrupts are shared, which means that if we detect an underrun we need to disable underrun reporting on all pipes. The code also supports underrun detection on the PCH transcoder.
Parameters
Description This function sets the fifo underrun state for pipe. It is used in the modeset code to avoid false positives since on many platforms underruns are expected when disabling or enabling the pipe. Notice that on some platforms disabling underrun reports for one pipe disables for all due to shared interrupts. Actual reporting is still per-pipe though. Returns the previous state of underrun reporting.
Parameters
Description This function makes us disable or enable PCH fifo underruns for a specific PCH transcoder. Notice that on some PCHs (e.g. CPT/PPT), disabling FIFO underrun reporting for one transcoder may also disable all the other PCH error interruts for the other transcoders, due to the fact that there’s just one interrupt mask/enable bit for all the transcoders. Returns the previous state of underrun reporting.
Parameters
Description This handles a CPU fifo underrun interrupt, generating an underrun warning into dmesg if underrun reporting is enabled and then disables the underrun interrupt to avoid an irq storm.
Parameters
Description This handles a PCH fifo underrun interrupt, generating an underrun warning into dmesg if underrun reporting is enabled and then disables the underrun interrupt to avoid an irq storm.
Parameters
Description Check for CPU fifo underruns immediately. Useful on IVB/HSW where the shared error interrupt may have been disabled, and so CPU fifo underruns won’t necessarily raise an interrupt, and on GMCH platforms where underruns never raise an interrupt.
Parameters
Description Check for PCH fifo underruns immediately. Useful on CPT/PPT where the shared error interrupt may have been disabled, and so PCH fifo underruns won’t necessarily raise an interrupt. Plane Configuration¶This section covers plane configuration and composition with the primary plane, sprites, cursors and overlays. This includes the infrastructure to do atomic vsync’ed updates of all this state and also tightly coupled topics like watermark setup and computation, framebuffer compression and panel self refresh. Atomic Plane Helpers¶The functions here are used by the atomic plane helper functions to implement legacy plane updates (i.e., drm_plane->update_plane() and drm_plane->disable_plane()). This allows plane updates to use the atomic state infrastructure and perform plane updates as separate prepare/check/commit/cleanup steps.
Parameters
Description Allocates and returns a copy of the plane state (both common and Intel-specific) for the specified plane. Return The newly allocated plane state, or NULL on failure.
Parameters
Description Destroys the plane state (both common and Intel-specific) for the specified plane. Asynchronous Page Flip¶Asynchronous page flip is the implementation for the DRM_MODE_PAGE_FLIP_ASYNC flag. Currently async flip is only supported via the drmModePageFlip IOCTL. Correspondingly, support is currently added for primary plane only. Async flip can only change the plane surface address, so anything else changing is rejected from the intel_atomic_check_async() function. Once this check is cleared, flip done interrupt is enabled using the intel_crtc_enable_flip_done() function. As soon as the surface address register is written, flip done interrupt is generated and the requested events are sent to the usersapce in the interrupt handler itself. The timestamp and sequence sent during the flip done event correspond to the last vblank and have no relation to the actual time when the flip done event was sent. Output Probing¶This section covers output probing and related infrastructure like the hotplug interrupt storm detection and mitigation code. Note that the i915 driver still uses most of the common DRM helper code for output probing, so those sections fully apply. Hotplug¶Simply put, hotplug occurs when a display is connected to or disconnected from the system. However, there may be adapters and docking stations and Display Port short pulses and MST devices involved, complicating matters. Hotplug in i915 is handled in many different levels of abstraction. The platform dependent interrupt handling code in i915_irq.c enables, disables, and does preliminary handling of the interrupts. The interrupt handlers gather the hotplug detect (HPD) information from relevant registers into a platform independent mask of hotplug pins that have fired. The platform independent interrupt handler The Display Port work function i915_digport_work_func() calls into intel_dp_hpd_pulse() via hooks, which handles DP short pulses and DP MST long pulses, with failures and non-MST long pulses triggering regular hotplug processing on the connector. The regular hotplug work function i915_hotplug_work_func() calls connector
detect hooks, and, if connector status changes, triggers sending of hotplug
uevent to userspace via Finally, the userspace is responsible for triggering a modeset upon receiving the hotplug uevent, disabling or enabling the crtc as needed. The hotplug interrupt storm detection and mitigation code keeps track of the number of interrupts per hotplug pin per a period of time, and if the number of interrupts exceeds a certain threshold, the interrupt is disabled for a while before being re-enabled. The intention is to mitigate issues raising from broken hardware triggering massive amounts of interrupts and grinding the system to a halt. Current implementation expects that hotplug interrupt storm will not be seen when display port sink is connected, hence on platforms whose DP callback is handled by i915_digport_work_func reenabling of hpd is not performed (it was never expected to be disabled in the first place ;) ) this is specific to DP sinks handled by this routine and any other display such as HDMI or DVI enabled on the same port will have proper logic since it will use i915_hotplug_work_func where this logic is handled.
Parameters
Description It is only valid and used by digital port encoder. Return pin that is associatade with port.
Parameters
Description Gather stats about HPD IRQs from the specified pin, and detect IRQ storms. Only the pin specific stats and state are changed, the caller is responsible for further action. The number of IRQs that are allowed within HPD_STORM_DETECT_PERIOD is stored in dev_priv->hotplug.hpd_storm_threshold which defaults to HPD_STORM_DEFAULT_THRESHOLD. Long IRQs count as +10 to this threshold, and short IRQs count as +1. If this threshold is exceeded, it’s considered an IRQ storm and the IRQ state is set to HPD_MARK_DISABLED. By default, most systems will only count long IRQs towards
The HPD threshold can be controlled through i915_hpd_storm_ctl in debugfs, and should only be adjusted for automated hotplug testing. Return true if an IRQ storm was detected on pin.
Parameters
Description Trigger an HPD interrupt event for the given port, emulating a short pulse generated by the sink, and schedule the dig port work to handle it.
Parameters
Description This is the main hotplug irq handler for all platforms. The platform specific irq handlers call the platform specific hotplug irq handlers, which read and decode the appropriate registers into bitmasks about hpd pins that have triggered (pin_mask), and which of those pins may be long pulses (long_mask). The long_mask is ignored if the port corresponding to the pin is not a digital port. Here, we do hotplug irq storm detection and mitigation, and pass further processing to appropriate bottom halves.
Parameters
Description This function enables the hotplug support. It requires that interrupts have already been enabled with intel_irq_init_hw(). From this point on hotplug and poll request can run concurrently to other code, so locking rules must be obeyed. This is a separate step from interrupt enabling to simplify the locking rules in the driver load and resume code. Also see:
Parameters
Description This function enables polling for all connectors which support HPD. Under certain conditions HPD may not be functional. On most Intel GPUs, this happens when we enter runtime suspend. On Valleyview and Cherryview systems, this also happens when we shut off all of the powerwells. Since this function can get called in contexts where we’re already holding dev->mode_config.mutex, we do the actual hotplug enabling in a seperate worker. Also see:
Parameters
Description This function disables polling for all connectors which support HPD. Under certain conditions HPD may not be functional. On most Intel GPUs, this happens when we enter runtime suspend. On Valleyview and Cherryview systems, this also happens when we shut off all of the powerwells. Since this function can get called in contexts where we’re already holding dev->mode_config.mutex, we do the actual hotplug enabling in a seperate worker. Also used during driver init to initialize connector->polled appropriately for all connectors. Also see: High Definition Audio¶The graphics and audio drivers together support High Definition Audio over HDMI and Display Port. The audio programming sequences are divided into audio codec and controller enable and disable sequences. The graphics driver handles the audio codec sequences, while the audio driver handles the audio controller sequences. The disable sequences must be performed before disabling the transcoder or port. The enable sequences may only be performed after enabling the transcoder and port, and after completed link training. Therefore the audio enable/disable sequences are part of the modeset sequence. The codec and controller sequences could be done either parallel or serial, but generally the ELDV/PD change in the codec sequence indicates to the audio driver that the controller sequence should start. Indeed, most of the co-operation between the graphics and audio drivers is handled via audio related registers. (The notable exception is the power management, not covered here.) The struct
Parameters
Description The enable sequences may only be performed after enabling the transcoder and port, and after completed link training.
Parameters
Description The disable sequences must be performed before disabling the transcoder or port.
Parameters
Parameters
Description This will register with the component framework a child component which
will bind dynamically to the snd_hda_intel driver’s corresponding master
component when the latter is registered. During binding the child
initializes an instance of We ignore any error during registration and continue with reduced functionality (i.e. without HDMI audio).
Parameters
Description Deregisters the audio component, breaking any existing binding to the corresponding snd_hda_intel driver’s master component.
Parameters
Parameters
Definition struct i915_audio_component {
struct drm_audio_component base;
int aud_sample_rate[MAX_PORTS];
};
Members
Intel HDMI LPE Audio Support¶Motivation: Atom platforms (e.g. valleyview and cherryTrail) integrates a DMA-based interface as an alternative to the traditional HDaudio path. While this mode is unrelated to the LPE aka SST audio engine, the documentation refers to this mode as LPE so we keep this notation for the sake of consistency. The interface is handled by a separate standalone driver maintained in the ALSA subsystem for simplicity. To minimize the interaction between the two subsystems, a bridge is setup between the hdmi-lpe-audio and i915: 1. Create a platform device to share MMIO/IRQ resources 2. Make the platform device child of i915 device for runtime PM. 3. Create IRQ chip to forward the LPE audio irqs. the hdmi-lpe-audio driver probes the lpe audio device and creates a new sound card Threats: Due to the restriction in Linux platform device model, user need manually uninstall the hdmi-lpe-audio driver before uninstalling i915 module, otherwise we might run into use-after-free issues after i915 removes the platform device: even though hdmi-lpe-audio driver is released, the modules is still in “installed” status. Implementation: The MMIO/REG platform resources are created according to the registers specification. When forwarding LPE audio irqs, the flow control handler selection depends on the platform, for example on valleyview handle_simple_irq is enough.
Parameters
Description the LPE Audio irq is forwarded to the irq handler registered by LPE audio driver.
Parameters
Return 0 if successful. non-zero if detection or llocation/initialization fails
Parameters
Description release all the resources for LPE audio <-> i915 bridge.
Parameters
Description Notify lpe audio driver of eld change. Panel Self Refresh PSR (PSR/SRD)¶Since Haswell Display controller supports Panel Self-Refresh on display panels witch have a remote frame buffer (RFB) implemented according to PSR spec in eDP1.3. PSR feature allows the display to go to lower standby states when system is idle but display is on as it eliminates display refresh request to DDR memory completely as long as the frame buffer for that display is unchanged. Panel Self Refresh must be supported by both Hardware (source) and Panel (sink). PSR saves power by caching the framebuffer in the panel RFB, which allows us to power down the link and memory controller. For DSI panels the same idea is called “manual mode”. The implementation uses the hardware-based PSR support which automatically
enters/exits self-refresh mode. The hardware takes care of sending the
required DP aux message and could even retrain the link (that part isn’t
enabled yet though). The hardware also keeps track of any frontbuffer
changes to know when to exit self-refresh mode again. Unfortunately that
part doesn’t work too well, hence why the i915 PSR support uses the
software frontbuffer tracking to make sure it doesn’t miss a screen
update. For this integration DC3CO (DC3 clock off) On top of PSR2, GEN12 adds a intermediate power savings state that turns clock off automatically during PSR2 idle state. The smaller overhead of DC3co entry/exit vs. the overhead of PSR2 deep sleep entry/exit allows the HW to enter a low-power state even when page flipping periodically (for instance a 30fps video playback scenario). Every time a flips occurs PSR2 will get out of deep sleep state(if it was), so DC3CO is enabled and tgl_dc3co_disable_work is schedule to run after 6 frames, if no other flip occurs and the function above is executed, DC3CO is disabled and PSR2 is configured to enter deep sleep, resetting again in case of another flip. Front buffer modifications do not trigger DC3CO activation on purpose as it would bring a lot of complexity and most of the moderns systems will only use page flips.
Parameters
Description This function can only be called after the pipe is fully trained and enabled.
Parameters
Description This function needs to be called before disabling pipe.
Parameters
Description This function need to be called after enabling psr.
Parameters
Description This function need to be called after pausing psr.
Parameters
Description This functions will update PSR states, disabling, enabling or switching PSR
version when executing fastsets. For full modeset,
Parameters
Return 0 on success or -ETIMEOUT if PSR status does not idle.
Parameters
Description This function is expected to be called from pipe_update_start() where it is not expected to race with PSR enable or disable.
Parameters
Description Since the hardware frontbuffer tracking has gaps we need to integrate with the software frontbuffer tracking. This function gets called every time frontbuffer rendering starts and a buffer gets dirtied. PSR must be disabled if the frontbuffer mask contains a buffer relevant to PSR. Dirty frontbuffers relevant to PSR are tracked in busy_frontbuffer_bits.”
Parameters
Description Since the hardware frontbuffer tracking has gaps we need to integrate with the software frontbuffer tracking. This function gets called every time frontbuffer rendering has completed and flushed out to memory. PSR can be enabled again if no other frontbuffer relevant to PSR is dirty. Dirty frontbuffers relevant to PSR are tracked in busy_frontbuffer_bits.
Parameters
Description This function is called after the initializing connector. (the initializing of connector treats the handling of connector capabilities) And it initializes basic PSR stuff for each DP Encoder. Frame Buffer Compression (FBC)¶FBC tries to save memory bandwidth (and so power consumption) by compressing the amount of memory used by the display. It is total transparent to user space and completely handled in the kernel. The benefits of FBC are mostly visible with solid backgrounds and variation-less patterns. It comes from keeping the memory footprint small and having fewer memory pages opened and accessed for refreshing the display. i915 is responsible to reserve stolen memory for FBC and configure its offset on proper registers. The hardware takes care of all compress/decompress. However there are many known cases where we have to forcibly disable it to allow proper screen updates.
Parameters
Description This function is used to verify the current state of FBC. FIXME: This should be tracked in the plane config eventually instead of queried at runtime for most callers.
Parameters
Description This is the low level function that actually disables FBC. Callers should grab the FBC lock.
Parameters
Description This function looks at the proposed state for CRTCs and planes, then chooses which pipe is going to have FBC by setting intel_crtc_state->enable_fbc to true. Later, intel_fbc_enable is going to look for state->enable_fbc and then maybe enable FBC for the chosen CRTC. If it does, it will set dev_priv->fbc.crtc.
Parameters
Description This function checks if the given CRTC was chosen for FBC, then enables it if possible. Notice that it doesn’t activate FBC. It is valid to call intel_fbc_enable multiple times for the same pipe without an intel_fbc_disable in the middle, as long as it is deactivated.
Parameters
Description This function disables FBC if it’s associated with the provided CRTC.
Parameters
Description This function disables FBC regardless of which CRTC is associated with it.
Parameters
Description Without FBC, most underruns are harmless and don’t really cause too many problems, except for an annoying message on dmesg. With FBC, underruns can become black screens or even worse, especially when paired with bad watermarks. So in order for us to be on the safe side, completely disable FBC in case we ever detect a FIFO underrun on any pipe. An underrun on any pipe already suggests that watermarks may be bad, so try to be as safe as possible. This function is called from the IRQ handler.
Parameters
Description This function might be called during PM init process. Display Refresh Rate Switching (DRRS)¶Display Refresh Rate Switching (DRRS) is a power conservation feature which enables swtching between low and high refresh rates, dynamically, based on the usage scenario. This feature is applicable for internal panels. Indication that the panel supports DRRS is given by the panel EDID, which would list multiple refresh rates for one resolution. DRRS is of 2 types - static and seamless. Static DRRS involves changing refresh rate (RR) by doing a full modeset (may appear as a blink on screen) and is used in dock-undock scenario. Seamless DRRS involves changing RR without any visual effect to the user and can be used during normal system usage. This is done by programming certain registers. Support for static/seamless DRRS may be indicated in the VBT based on inputs from the panel spec. DRRS saves power by switching to low RR based on usage scenarios. The implementation is based on frontbuffer tracking implementation. When there is a disturbance on the screen triggered by user activity or a periodic system activity, DRRS is disabled (RR is changed to high RR). When there is no movement on screen, after a timeout of 1 second, a switch to low RR is made. For integration with frontbuffer tracking code, DRRS can be further extended to support other internal panels and also the scenario of video playback wherein RR is set based on the rate requested by userspace.
Parameters
Description This function gets called when refresh rate (RR) has to be changed from one frequency to another. Switches can be between high and low RR supported by the panel or to any other RR based on media playback (in this case, RR value needs to be passed from user space). The caller of this function needs to take a lock on dev_priv->drrs.
Parameters
Description Initializes frontbuffer_bits and drrs.dp
Parameters
Parameters
Description This function gets called everytime rendering on the given planes start. Hence DRRS needs to be Upclocked, i.e. (LOW_RR -> HIGH_RR). Dirty frontbuffers relevant to DRRS are tracked in busy_frontbuffer_bits.
Parameters
Description This function gets called every time rendering on the given planes has completed or flip on a crtc is completed. So DRRS should be upclocked (LOW_RR -> HIGH_RR). And also Idleness detection should be started again, if no other planes are dirty. Dirty frontbuffers relevant to DRRS are tracked in busy_frontbuffer_bits.
Parameters
Description This function is called only once at driver load to initialize basic DRRS stuff. Return Downclock mode if panel supports it, else return NULL. DRRS support is determined by the presence of downclock mode (apart from VBT setting). DPIO¶VLV, CHV and BXT have slightly peculiar display PHYs for driving DP/HDMI ports. DPIO is the name given to such a display PHY. These PHYs don’t follow the standard programming model using direct MMIO registers, and instead their registers must be accessed trough IOSF sideband. VLV has one such PHY for driving ports B and C, and CHV adds another PHY for driving port D. Each PHY responds to specific IOSF-SB port. Each display PHY is made up of one or two channels. Each channel houses a common lane part which contains the PLL and other common logic. CH0 common lane also contains the IOSF-SB logic for the Common Register Interface (CRI) ie. the DPIO registers. CRI clock must be running when any DPIO registers are accessed. In addition to having their own registers, the PHYs are also controlled through some dedicated signals from the display controller. These include PLL reference clock enable, PLL enable, and CRI clock selection, for example. Eeach channel also has two splines (also called data lanes), and each spline is made up of one Physical Access Coding Sub-Layer (PCS) block and two TX lanes. So each channel has two PCS blocks and four TX lanes. The TX lanes are used as DP lanes or TMDS data/clock pairs depending on the output type. Additionally the PHY also contains an AUX lane with AUX blocks for each channel. This is used for DP AUX communication, but this fact isn’t really relevant for the driver since AUX is controlled from the display controller side. No DPIO registers need to be accessed during AUX communication, Generally on VLV/CHV the common lane corresponds to the pipe and the spline (PCS/TX) corresponds to the port. For dual channel PHY (VLV/CHV):
This is especially important when we cross the streams ie. drive port B with pipe B, or port C with pipe A. For single channel PHY (CHV):
On BXT the entire PHY channel corresponds to the port. That means the PLL is also now associated with the port rather than the pipe, and so the clock needs to be routed to the appropriate transcoder. Port A PLL is directly connected to transcoder EDP and port B/C PLLs can be routed to any transcoder A/B/C. Note: DDI0 is digital port B, DD1 is digital port C, and DDI2 is digital port D (CHV) or port A (BXT). Dual channel PHY (VLV/CHV/BXT) --------------------------------- | CH0 | CH1 | | CMN/PLL/REF | CMN/PLL/REF | |---------------|---------------| Display PHY | PCS01 | PCS23 | PCS01 | PCS23 | |-------|-------|-------|-------| |TX0|TX1|TX2|TX3|TX0|TX1|TX2|TX3| --------------------------------- | DDI0 | DDI1 | DP/HDMI ports --------------------------------- Single channel PHY (CHV/BXT) ----------------- | CH0 | | CMN/PLL/REF | |---------------| Display PHY | PCS01 | PCS23 | |-------|-------| |TX0|TX1|TX2|TX3| ----------------- | DDI2 | DP/HDMI port ----------------- DMC Firmware Support¶From gen9 onwards we have newly added DMC (Display microcontroller) in display engine to save and restore the state of display engine when it enter into low-power state and comes back to normal.
Parameters
Description DMC firmware is read from a .bin file and kept in internal memory one time. Everytime display comes back from low power state this function is called to copy the firmware from internal memory to registers.
Parameters
Description This function is called at the time of loading the display driver to read firmware from a .bin file and copied into a internal memory.
Parameters
Description Prepare the DMC firmware before entering system suspend. This includes flushing pending work items and releasing any resources acquired during init.
Parameters
Description Reinitialize the DMC firmware during system resume, reacquiring any
resources released in
Parameters
Description Firmmware unloading includes freeing the internal memory and reset the firmware loading status. Video BIOS Table (VBT)¶The Video BIOS Table, or VBT, provides platform and board specific configuration information to the driver that is not discoverable or available through other means. The configuration is mostly related to display hardware. The VBT is available via the ACPI OpRegion or, on older systems, in the PCI ROM. The VBT consists of a VBT Header (defined as The driver parses the VBT during load. The relevant information is stored in driver private data for ease of use, and the actual VBT is not read after that.
Parameters
Description Returns true on valid VBT.
Parameters
Description Parse and initialize settings from the Video BIOS Tables (VBT). If the VBT was not found in ACPI OpRegion, try to find it in PCI ROM first. Also initialize some defaults if the VBT is not present at all.
Parameters
Parameters
Description Return true if TV is present. If no child devices were parsed from VBT, assume TV is present.
Parameters
Description Return true if LVDS is present. If no child devices were parsed from VBT, assume LVDS is present.
Parameters
Description Return true if the device in
Parameters
Description Return true if the device in
Parameters
Description Return true if DSI is present, and return the port in
Parameters
Description Return true if HPD should be inverted for
Parameters
Description Return true if LSPCON is present on this port
Parameters
Description Return true if port requires lane reversal
Definition struct vbt_header {
u8 signature[20];
u16 version;
u16 header_size;
u16 vbt_size;
u8 vbt_checksum;
u8 reserved0;
u32 bdb_offset;
u32 aim_offset[4];
};
Members
Definition struct bdb_header {
u8 signature[16];
u16 version;
u16 header_size;
u16 bdb_size;
};
Members
Display clocks¶The display engine uses several different clocks to do its work. There are two main clocks involved that aren’t directly related to the actual pixel clock or any symbol/bit clock of the actual output port. These are the core display clock (CDCLK) and RAWCLK. CDCLK clocks most of the display pipe logic, and thus its frequency must be high enough to support the rate at which pixels are flowing through the pipes. Downscaling must also be accounted as that increases the effective pixel rate. On several platforms the CDCLK frequency can be changed dynamically to minimize power consumption for a given display configuration. Typically changes to the CDCLK frequency require all the display pipes to be shut down while the frequency is being changed. On SKL+ the DMC will toggle the CDCLK off/on during DC5/6 entry/exit. DMC will not change the active CDCLK frequency however, so that part will still be performed by the driver directly. RAWCLK is a fixed frequency clock, often used by various auxiliary blocks such as AUX CH or backlight PWM. Hence the only thing we really need to know about RAWCLK is its frequency so that various dividers can be programmed correctly.
Parameters
Description Initialize CDCLK. This consists mainly of initializing dev_priv->cdclk.hw and sanitizing the state of the hardware if needed. This is generally done only during the display core initialization sequence, after which the DMC will take care of turning CDCLK off/on as needed.
Parameters
Description Uninitialize CDCLK. This is done only during the display core uninitialization sequence.
Parameters
Return True if changing between the two CDCLK configurations requires all pipes to be off, false if not.
Parameters
Return True if changing between the two CDCLK configurations can be done with just a cd2x divider update, false if not.
Parameters
Return True if the CDCLK configurations don’t match, false if they do.
Parameters
Description Program the hardware based on the passed in CDCLK state, if necessary.
Parameters
Description Program the hardware before updating the HW plane state based on the new CDCLK state, if necessary.
Parameters
Description Program the hardware after updating the HW plane state based on the new CDCLK state, if necessary.
Parameters
Description Determine the maximum CDCLK frequency the platform supports, and also derive the maximum dot clock frequency the maximum CDCLK frequency allows.
Parameters
Description Determine the current CDCLK frequency.
Parameters
Description Determine the current RAWCLK frequency. RAWCLK is a fixed frequency clock so this needs to done only once.
Parameters
Display PLLs¶Display PLLs used for driving outputs vary by platform. While some have per-pipe or per-encoder dedicated PLLs, others allow the use of any PLL from a pool. In the latter scenario, it is possible that multiple pipes share a PLL if their configurations match. This file provides an abstraction over display PLLs. The function
Parameters
Return A pointer to the DPLL with id
Parameters
Return The id of pll
Parameters
Description This calls the PLL’s prepare hook if it has one and if the PLL is not already enabled. The prepare hook is platform specific.
Parameters
Description Enable the shared DPLL used by crtc.
Parameters
Description Disable the shared DPLL used by crtc.
Parameters
Description This is the dpll version of For consistency with atomic helpers this function does a complete swap, i.e. it also puts the current state into state, even though there is no need for that at this moment.
Parameters
Description Select the given port_dpll_id instance from the DPLLs reserved for the CRTC.
Parameters
Description Initialize shared DPLLs for dev.
Parameters
Description This function reserves all required DPLLs for the given CRTC and encoder combination in the current atomic commit state and the new crtc atomic state. The new configuration in the atomic commit state is made effective by
calling The reserved DPLLs should be released by calling
Return True if all required DPLLs were successfully reserved.
Parameters
Description This function releases all DPLLs reserved by The new configuration in the atomic commit state is made effective by
calling
Parameters
Description Update the active DPLL for the given crtc/encoder in crtc’s atomic state,
from the port DPLLs reserved previously by
Parameters
Description Return the output frequency corresponding to pll’s passed in pll_state.
Parameters
Description Read out pll’s hardware state into hw_state.
Parameters
Description Write the relevant values in hw_state to dmesg using drm_dbg_kms.
Constants
Description Enumeration of possible IDs for a DPLL. Real shared dpll ids must be >= 0.
Definition struct intel_shared_dpll_state {
u8 pipe_mask;
struct intel_dpll_hw_state hw_state;
};
Members
Description This structure holds an atomic state for the DPLL, that can represent
either its current state (in struct See also
Definition struct intel_shared_dpll_funcs {
void (*prepare)(struct drm_i915_private *dev_priv, struct intel_shared_dpll *pll);
void (*enable)(struct drm_i915_private *dev_priv, struct intel_shared_dpll *pll);
void (*disable)(struct drm_i915_private *dev_priv, struct intel_shared_dpll *pll);
bool (*get_hw_state)(struct drm_i915_private *dev_priv,struct intel_shared_dpll *pll, struct intel_dpll_hw_state *hw_state);
int (*get_freq)(struct drm_i915_private *i915,const struct intel_shared_dpll *pll, const struct intel_dpll_hw_state *pll_state);
};
Members
Definition struct dpll_info {
const char *name;
const struct intel_shared_dpll_funcs *funcs;
enum intel_dpll_id id;
#define INTEL_DPLL_ALWAYS_ON (1 << 0);
u32 flags;
};
Members
Definition struct intel_shared_dpll {
struct intel_shared_dpll_state state;
u8 active_mask;
bool on;
const struct dpll_info *info;
intel_wakeref_t wakeref;
};
Members
Display State Buffer¶A DSB (Display State Buffer) is a queue of MMIO instructions in the memory which can be offloaded to DSB HW in Display Controller. DSB HW is a DMA engine that can be programmed to download the DSB from memory. It allows driver to batch submit display HW programming. This helps to reduce loading time and CPU activity, thereby making the context switch faster. DSB Support added from Gen12 Intel graphics based platform. DSB’s can access only the pipe, plane, and transcoder Data Island Packet registers. DSB HW can support only register writes (both indexed and direct MMIO writes). There are no registers reads possible with DSB HW engine.
Parameters
Description This function is used for writing register-value pair in command buffer of DSB for auto-increment register. During command buffer overflow, a warning is thrown and rest all erroneous condition register programming is done through mmio write.
Parameters
Description This function is used for writing register-value pair in command buffer of DSB. During command buffer overflow, a warning is thrown and rest all erroneous condition register programming is done through mmio write.
Parameters
Description This function is used to do actual write to hardware using DSB. On errors, fall back to MMIO. Also this function help to reset the context.
Parameters
Description This function prepare the command buffer which is used to store dsb instructions with data.
Parameters
Description This function cleanup the DSB context by unpinning and releasing the VMA object associated with it. Memory Management and Command Submission¶This sections covers all things related to the GEM implementation in the i915 driver. Intel GPU Basics¶An Intel GPU has multiple engines. There are several engine types.
The Intel GPU family is a family of integrated GPU’s using Unified Memory Access. For having the GPU “do work”, user space will feed the GPU batch buffers via one of the ioctls DRM_IOCTL_I915_GEM_EXECBUFFER2 or DRM_IOCTL_I915_GEM_EXECBUFFER2_WR. Most such batchbuffers will instruct the GPU to perform work (for example rendering) and that work needs memory from which to read and memory to which to write. All memory is encapsulated within GEM buffer objects (usually created with the ioctl DRM_IOCTL_I915_GEM_CREATE). An ioctl providing a batchbuffer for the GPU to create will also list all GEM buffer objects that the batchbuffer reads and/or writes. For implementation details of memory management see GEM BO Management Implementation Details. The i915 driver allows user space to create a context via the ioctl DRM_IOCTL_I915_GEM_CONTEXT_CREATE which is identified by a 32-bit integer. Such a context should be viewed by user-space as -loosely- analogous to the idea of a CPU process of an operating system. The i915 driver guarantees that commands issued to a fixed context are to be executed so that writes of a previously issued command are seen by reads of following commands. Actions issued between different contexts (even if from the same file descriptor) are NOT given that guarantee and the only way to synchronize across contexts (even from the same file descriptor) is through the use of fences. At least as far back as Gen4, also have that a context carries with it a GPU HW context; the HW context is essentially (most of atleast) the state of a GPU. In addition to the ordering guarantees, the kernel will restore GPU state via HW context when commands are issued to a context, this saves user space the need to restore (most of atleast) the GPU state at the start of each batchbuffer. The non-deprecated ioctls to submit batchbuffer work can pass that ID (in the lower bits of drm_i915_gem_execbuffer2::rsvd1) to identify what context to use with the command. The GPU has its own memory management and address space. The kernel driver maintains the memory translation table for the GPU. For older GPUs (i.e. those before Gen8), there is a single global such translation table, a global Graphics Translation Table (GTT). For newer generation GPUs each context has its own translation table, called Per-Process Graphics Translation Table (PPGTT). Of important note, is that although PPGTT is named per-process it is actually per context. When user space submits a batchbuffer, the kernel walks the list of GEM buffer objects used by the batchbuffer and guarantees that not only is the memory of each such GEM buffer object resident but it is also present in the (PP)GTT. If the GEM buffer object is not yet placed in the (PP)GTT, then it is given an address. Two consequences of this are: the kernel needs to edit the batchbuffer submitted to write the correct value of the GPU address when a GEM BO is assigned a GPU address and the kernel might evict a different GEM BO from the (PP)GTT to make address room for another GEM BO. Consequently, the ioctls submitting a batchbuffer for execution also include a list of all locations within buffers that refer to GPU-addresses so that the kernel can edit the buffer correctly. This process is dubbed relocation. Locking Guidelines¶Note This is a description of how the locking should be after refactoring is done. Does not necessarily reflect what the locking looks like while WIP.
GEM BO Management Implementation Details¶A VMA represents a GEM BO that is bound into an address space. Therefore, a VMA’s presence cannot be guaranteed before binding, or after unbinding the object into/from the address space. To make things as simple as possible (ie. no refcounting), a VMA’s lifetime will always be <= an objects lifetime. So object refcounting should cover us. Buffer Object Eviction¶This section documents the interface functions for evicting buffer objects to make space available in the virtual gpu address spaces. Note that this is mostly orthogonal to shrinking buffer objects caches, which has the goal to make main memory (shared with the gpu through the unified memory architecture) available.
Parameters
Description This function will try to evict vmas until a free space satisfying the requirements is found. Callers must check first whether any such hole exists already before calling this function. This function is used by the object/vma binding code. Since this function is only used to free up virtual address space it only ignores pinned vmas, and not object where the backing storage itself is pinned. Hence obj->pages_pin_count does not protect against eviction. To clarify: This is for freeing up virtual address space, not for freeing memory in e.g. the shrinker.
Parameters
Description This function will try to evict vmas that overlap the target node. To clarify: This is for freeing up virtual address space, not for freeing memory in e.g. the shrinker.
Parameters
Description This function evicts all vmas from a vm. This is used by the execbuf code as a last-ditch effort to defragment the address space. To clarify: This is for freeing up virtual address space, not for freeing memory in e.g. the shrinker. Buffer Object Memory Shrinking¶This section documents the interface function for shrinking memory usage of buffer object caches. Shrinking is used to make main memory available. Note that this is mostly orthogonal to evicting buffer objects, which has the goal to make space in gpu virtual address spaces.
Parameters
Description This function is the main interface to the shrinker. It will try to release up to target pages of main memory backing storage from buffer objects. Selection of the specific caches can be done with flags. This is e.g. useful when purgeable objects should be removed from caches preferentially. Note that it’s not guaranteed that released amount is actually available as
free system memory - the pages might still be in-used to due to other reasons
(like cpu mmaps) or the mm core has reused them before we could grab them.
Therefore code that needs to explicitly shrink buffer objects caches (e.g. to
avoid deadlocks in memory reclaim) must fall back to Also note that any kind of pinning (both per-vma address space pins and backing storage pins at the buffer object level) result in the shrinker code having to skip the object. Return The number of pages of backing storage actually released.
Parameters
Description This is a simple wraper around This should only be used in code to intentionally quiescent the gpu or as a last-ditch effort when memory seems to have run out. Return The number of pages of backing storage actually released. Batchbuffer Parsing¶Motivation: Certain OpenGL features (e.g. transform feedback, performance monitoring) require userspace code to submit batches containing commands such as MI_LOAD_REGISTER_IMM to access various registers. Unfortunately, some generations of the hardware will noop these commands in “unsecure” batches (which includes all userspace batches submitted via i915) even though the commands may be safe and represent the intended programming model of the device. The software command parser is similar in operation to the command parsing done in hardware for unsecure batches. However, the software parser allows some operations that would be noop’d by hardware, if the parser determines the operation is safe, and submits the batch as “secure” to prevent hardware parsing. Threats: At a high level, the hardware (and software) checks attempt to prevent granting userspace undue privileges. There are three categories of privilege. First, commands which are explicitly defined as privileged or which should only be used by the kernel driver. The parser rejects such commands Second, commands which access registers. To support correct/enhanced userspace functionality, particularly certain OpenGL extensions, the parser provides a whitelist of registers which userspace may safely access Third, commands which access privileged memory (i.e. GGTT, HWS page, etc). The parser always rejects such commands. The majority of the problematic commands fall in the MI_* range, with only a few specific commands on each engine (e.g. PIPE_CONTROL and MI_FLUSH_DW). Implementation: Each engine maintains tables of commands and registers which the parser uses in scanning batch buffers submitted to that engine. Since the set of commands that the parser must check for is significantly smaller than the number of commands supported, the parser tables contain only those commands required by the parser. This generally works because command opcode ranges have standard command length encodings. So for commands that the parser does not need to check, it can easily skip them. This is implemented via a per-engine length decoding vfunc. Unfortunately, there are a number of commands that do not follow the standard length encoding for their opcode range, primarily amongst the MI_* commands. To handle this, the parser provides a way to define explicit “skip” entries in the per-engine command tables. Other command table entries map fairly directly to high level categories mentioned above: rejected, register whitelist. The parser implements a number of checks, including the privileged memory checks, via a general bitmasking mechanism.
Parameters
Description Optionally initializes fields related to batch buffer command parsing in the struct intel_engine_cs based on whether the platform requires software command parsing.
Parameters
Description Releases any resources related to command parsing that may have been initialized for the specified engine.
Parameters
Description Parses the specified batch buffer looking for privilege violations as described in the overview. Return non-zero if the parser finds violations or otherwise fails; -EACCES if the batch appears legal but should use hardware parsing
Parameters
Description The cmd parser maintains a simple increasing integer version number suitable for passing to userspace clients to determine what operations are permitted. Return the current version number of the cmd parser User Batchbuffer Execution¶
Definition struct i915_gem_engines {
union {
struct list_head link;
struct rcu_head rcu;
};
struct i915_sw_fence fence;
struct i915_gem_context *ctx;
unsigned int num_engines;
struct intel_context *engines[];
};
Members
Definition struct i915_gem_engines_iter {
unsigned int idx;
const struct i915_gem_engines *engines;
};
Members
Constants
Definition struct i915_gem_proto_engine {
enum i915_gem_engine_type type;
struct intel_engine_cs *engine;
unsigned int num_siblings;
struct intel_engine_cs **siblings;
struct intel_sseu sseu;
};
Members
Description This struct describes an engine that a context may contain. Engines have three types:
Definition struct i915_gem_proto_context {
struct i915_address_space *vm;
unsigned long user_flags;
struct i915_sched_attr sched;
int num_user_engines;
struct i915_gem_proto_engine *user_engines;
struct intel_sseu legacy_rcs_sseu;
bool single_timeline;
};
Members
Description The The context uAPI allows for two methods of setting context parameters: SET_CONTEXT_PARAM and CONTEXT_CREATE_EXT_SETPARAM. The former is allowed to be called at any time while the later happens as part of GEM_CONTEXT_CREATE. When these were initially added, Currently, everything settable via one is settable via the other. While some params are fairly simple and setting them on a live context is harmless such the context priority, others are far trickier such as the VM or the set of engines. To avoid some truly nasty race conditions, we don’t allow setting the VM or the set of engines on live contexts. The way we dealt with this without breaking older userspace that sets the VM or engine set via SET_CONTEXT_PARAM is to delay the creation of the actual context until after the client is done configuring it with SET_CONTEXT_PARAM. From the perspective of the client, it has the same u32 context ID the whole time. From the perspective of i915, however, it’s an i915_gem_proto_context right up until the point where we attempt to do something which the proto-context can’t handle at which point the real context gets created. This is accomplished via a little xarray dance. When GEM_CONTEXT_CREATE is called, we create a proto-context, reserve a slot in context_xa but leave it NULL, the proto-context in the corresponding slot in proto_context_xa. Then, whenever we go to look up a context, we first check context_xa. If it’s there, we return the i915_gem_context and we’re done. If it’s not, we look in proto_context_xa and, if we find it there, we create the actual context and kill the proto-context. At the time we made this change (April, 2021), we did a fairly complete audit of existing userspace to ensure this wouldn’t break anything:
In order for this dance to work properly, any modification to an i915_gem_proto_context that is exposed to the client via drm_i915_file_private::proto_context_xa must be guarded by drm_i915_file_private::proto_context_lock. The exception is when a proto-context has not yet been exposed such as when handling CONTEXT_CREATE_SET_PARAM during GEM_CONTEXT_CREATE.
Definition struct i915_gem_context {
struct drm_i915_private *i915;
struct drm_i915_file_private *file_priv;
struct i915_gem_engines __rcu *engines;
struct mutex engines_mutex;
struct drm_syncobj *syncobj;
struct i915_address_space __rcu *vm;
struct pid *pid;
struct list_head link;
struct kref ref;
struct rcu_head rcu;
unsigned long user_flags;
#define UCONTEXT_NO_ERROR_CAPTURE 1;
#define UCONTEXT_BANNABLE 2;
#define UCONTEXT_RECOVERABLE 3;
#define UCONTEXT_PERSISTENCE 4;
unsigned long flags;
#define CONTEXT_CLOSED 0;
#define CONTEXT_USER_ENGINES 1;
struct mutex mutex;
struct i915_sched_attr sched;
atomic_t guilty_count;
atomic_t active_count;
unsigned long hang_timestamp[2];
#define CONTEXT_FAST_HANG_JIFFIES (120 * HZ) ;
u8 remap_slice;
struct radix_tree_root handles_vma;
struct mutex lut_mutex;
char name[TASK_COMM_LEN + 8];
struct {
spinlock_t lock;
struct list_head engines;
} stale;
};
Members
Description The Userspace submits commands to be executed on the GPU as an instruction stream within a GEM object we call a batchbuffer. This instructions may refer to other GEM objects containing auxiliary state such as kernels, samplers, render targets and even secondary batchbuffers. Userspace does not know where in the GPU memory these objects reside and so before the batchbuffer is passed to the GPU for execution, those addresses in the batchbuffer and auxiliary objects are updated. This is known as relocation, or patching. To try and avoid having to relocate each object on the next execution, userspace is told the location of those objects in this pass, but this remains just a hint as the kernel may choose a new location for any object in the future. At the level of talking to the hardware, submitting a batchbuffer for the GPU to execute is to add content to a buffer from which the HW command streamer is reading.
Processing an execbuf ioctl is conceptually split up into a few phases.
Reserving resources for the execbuf is the most complicated phase. We neither want to have to migrate the object in the address space, nor do we want to have to update any relocations pointing to this object. Ideally, we want to leave the object where it is and for all the existing relocations to match. If the object is given a new address, or if userspace thinks the object is elsewhere, we have to parse all the relocation entries and update the addresses. Userspace can set the I915_EXEC_NORELOC flag to hint that all the target addresses in all of its objects match the value in the relocation entries and that they all match the presumed offsets given by the list of execbuffer objects. Using this knowledge, we know that if we haven’t moved any buffers, all the relocation entries are valid and we can skip the update. (If userspace is wrong, the likely outcome is an impromptu GPU hang.) The requirement for using I915_EXEC_NO_RELOC are:
The reservation is done is multiple phases. First we try and keep any object already bound in its current location - so as long as meets the constraints imposed by the new execbuffer. Any object left unbound after the first pass is then fitted into any available idle space. If an object does not fit, all objects are removed from the reservation and the process rerun after sorting the objects into a priority order (more difficult to fit objects are tried first). Failing that, the entire VM is cleared and we try to fit the execbuf once last time before concluding that it simply will not fit. A small complication to all of this is that we allow userspace not only to specify an alignment and a size for the object in the address space, but we also allow userspace to specify the exact offset. This objects are simpler to place (the location is known a priori) all we have to do is make sure the space is available. Once all the objects are in place, patching up the buried pointers to point to the final locations is a fairly simple job of walking over the relocation entry arrays, looking up the right address and rewriting the value into the object. Simple! … The relocation entries are stored in user memory and so to access them we have to copy them into a local buffer. That copy has to avoid taking any pagefaults as they may lead back to a GEM object requiring the struct_mutex (i.e. recursive deadlock). So once again we split the relocation into multiple passes. First we try to do everything within an atomic context (avoid the pagefaults) which requires that we never wait. If we detect that we may wait, or if we need to fault, then we have to fallback to a slower path. The slowpath has to drop the mutex. (Can you hear alarm bells yet?) Dropping the mutex means that we lose all the state we have built up so far for the execbuf and we must reset any global data. However, we do leave the objects pinned in their final locations - which is a potential issue for concurrent execbufs. Once we have left the mutex, we can allocate and copy all the relocation entries into a large array at our leisure, reacquire the mutex, reclaim all the objects and other state and then proceed to update any incorrect addresses with the objects. As we process the relocation entries, we maintain a record of whether the object is being written to. Using NORELOC, we expect userspace to provide this information instead. We also check whether we can skip the relocation by comparing the expected value inside the relocation entry with the target’s final address. If they differ, we have to map the current object and rewrite the 4 or 8 byte pointer within. Serialising an execbuf is quite simple according to the rules of the GEM ABI. Execution within each context is ordered by the order of submission. Writes to any GEM object are in order of submission and are exclusive. Reads from a GEM object are unordered with respect to other reads, but ordered by writes. A write submitted after a read cannot occur before the read, and similarly any read submitted after a write cannot occur before the write. Writes are ordered between engines such that only one write occurs at any time (completing any reads beforehand) - using semaphores where available and CPU serialisation otherwise. Other GEM access obey the same rules, any write (either via mmaps using set-domain, or via pwrite) must flush all GPU reads before starting, and any read (either using set-domain or pread) must flush all GPU writes before starting. (Note we only employ a barrier before, we currently rely on userspace not concurrently starting a new execution whilst reading or writing to an object. This may be an advantage or not depending on how much you trust userspace not to shoot themselves in the foot.) Serialisation may just result in the request being inserted into a DAG awaiting its turn, but most simple is to wait on the CPU until all dependencies are resolved. After all of that, is just a matter of closing the request and handing it to the hardware (well, leaving it in a queue to be executed). However, we also offer the ability for batchbuffers to be run with elevated privileges so that they access otherwise hidden registers. (Used to adjust L3 cache etc.) Before any batch is given extra privileges we first must check that it contains no nefarious instructions, we check that each instruction is from our whitelist and all registers are also from an allowed list. We first copy the user’s batchbuffer to a shadow (so that the user doesn’t have access to it, either by the CPU or GPU as we scan it) and then parse each instruction. If everything is ok, we set a flag telling the hardware to run the batchbuffer in trusted mode, otherwise the ioctl is rejected. Scheduling¶
Definition struct i915_sched_engine {
struct kref ref;
spinlock_t lock;
struct list_head requests;
struct list_head hold;
struct tasklet_struct tasklet;
struct i915_priolist default_priolist;
int queue_priority_hint;
struct rb_root_cached queue;
bool no_priolist;
void *private_data;
void (*destroy)(struct kref *kref);
bool (*disabled)(struct i915_sched_engine *sched_engine);
void (*kick_backend)(const struct i915_request *rq, int prio);
void (*bump_inflight_request_prio)(struct i915_request *rq, int prio);
void (*retire_inflight_request_prio)(struct i915_request *rq);
void (*schedule)(struct i915_request *request, const struct i915_sched_attr *attr);
};
Members
Description A schedule engine represents a submission queue with different priority bands. It contains all the common state (relative to the backend) to queue, track, and submit a request. This object at the moment is quite i915 specific but will transition into a container for the drm_gpu_scheduler plus a few other variables once the i915 is integrated with the DRM scheduler. Logical Rings, Logical Ring Contexts and Execlists¶Motivation: GEN8 brings an expansion of the HW contexts: “Logical Ring Contexts”. These expanded contexts enable a number of new abilities, especially “Execlists” (also implemented in this file). One of the main differences with the legacy HW contexts is that logical ring contexts incorporate many more things to the context’s state, like PDPs or ringbuffer control registers: The reason why PDPs are included in the context is straightforward: as PPGTTs (per-process GTTs) are actually per-context, having the PDPs contained there mean you don’t need to do a ppgtt->switch_mm yourself, instead, the GPU will do it for you on the context switch. But, what about the ringbuffer control registers (head, tail, etc..)? shouldn’t we just need a set of those per engine command streamer? This is where the name “Logical Rings” starts to make sense: by virtualizing the rings, the engine cs shifts to a new “ring buffer” with every context switch. When you want to submit a workload to the GPU you: A) choose your context, B) find its appropriate virtualized ring, C) write commands to it and then, finally, D) tell the GPU to switch to that context. Instead of the legacy MI_SET_CONTEXT, the way you tell the GPU to switch to a contexts is via a context execution list, ergo “Execlists”. LRC implementation: Regarding the creation of contexts, we have:
Now that ringbuffers belong per-context (and not per-engine, like before) and that contexts are uniquely tied to a given engine (and not reusable, like before) we need:
The global default context starts its life with these new objects fully allocated and populated. The local default context for each opened fd is more complex, because we don’t know at creation time which engine is going to use them. To handle this, we have implemented a deferred creation of LR contexts: The local context starts its life as a hollow or blank holder, that only gets populated for a given engine once we receive an execbuffer. If later on we receive another execbuffer ioctl for the same context but a different engine, we allocate/populate a new ringbuffer and context backing object and so on. Finally, regarding local contexts created using the ioctl call: as they are only allowed with the render ring, we can allocate & populate them right away (no need to defer anything, at least for now). Execlists implementation: Execlists are the new method by which, on gen8+ hardware, workloads are submitted for execution (as opposed to the legacy, ringbuffer-based, method). This method works as follows: When a request is committed, its commands (the BB start and any leading or trailing commands, like the seqno breadcrumbs) are placed in the ringbuffer for the appropriate context. The tail pointer in the hardware context is not updated at this time, but instead, kept by the driver in the ringbuffer structure. A structure representing this request is added to a request queue for the appropriate engine: this structure contains a copy of the context’s tail after the request was written to the ring buffer and a pointer to the context itself. If the engine’s request queue was empty before the request was added, the queue is processed immediately. Otherwise the queue will be processed during a context switch interrupt. In any case, elements on the queue will get sent (in pairs) to the GPU’s ExecLists Submit Port (ELSP, for short) with a globally unique 20-bits submission ID. When execution of a request completes, the GPU updates the context status buffer with a context complete event and generates a context switch interrupt. During the interrupt handling, the driver examines the events in the buffer: for each context complete event, if the announced ID matches that on the head of the request queue, then that request is retired and removed from the queue. After processing, if any requests were retired and the queue is not empty then a new execution list can be submitted. The two requests at the front of the queue are next to be submitted but since a context may not occur twice in an execution list, if subsequent requests have the same ID as the first then the two requests must be combined. This is done simply by discarding requests at the head of the queue until either only one requests is left (in which case we use a NULL second context) or the first two requests have unique IDs. By always executing the first two requests in the queue the driver ensures that the GPU is kept as busy as possible. In the case where a single context completes but a second context is still executing, the request for this second context will be at the head of the queue when we remove the first one. This request will then be resubmitted along with a new request for a different context, which will cause the hardware to continue executing the second request and queue the new request (the GPU detects the condition of a context getting preempted with the same context and optimizes the context switch flow by not doing preemption, but just sampling the new tail pointer). Global GTT views¶Background and previous state Historically objects could exists (be bound) in global GTT space only as singular instances with a view representing all of the object’s backing pages in a linear fashion. This view will be called a normal view. To support multiple views of the same object, where the number of mapped pages is not equal to the backing store, or where the layout of the pages is not linear, concept of a GGTT view was added. One example of an alternative view is a stereo display driven by a single image. In this case we would have a framebuffer looking like this (2x2 pages):
Above would represent a normal GGTT view as normally mapped for GPU or CPU rendering. In contrast, fed to the display engine would be an alternative view which could look something like this:
In this example both the size and layout of pages in the alternative view is different from the normal view. Implementation and usage GGTT views are implemented using VMAs and are distinguished via enum i915_ggtt_view_type and struct i915_ggtt_view. A new flavour of core GEM functions which work with GGTT bound objects were added with the _ggtt_ infix, and sometimes with _view postfix to avoid renaming in large amounts of code. They take the struct i915_ggtt_view parameter encapsulating all metadata required to implement a view. As a helper for callers which are only interested in the normal view, globally const i915_ggtt_view_normal singleton instance exists. All old core GEM API functions, the ones not taking the view parameter, are operating on, or with the normal GGTT view. Code wanting to add or use a new GGTT view needs to:
New views are required to build a scatter-gather table from within the i915_get_vma_pages function. This table is stored in the vma.ggtt_view and exists for the lifetime of an VMA. Core API is designed to have copy semantics which means that passed in struct i915_ggtt_view does not need to be persistent (left around after calling the core API functions).
Parameters
Description
Return 0 on success, -ENOSPC if no suitable hole is found, -EINTR if asked to wait for eviction and interrupted.
Parameters
Description
Return 0 on success, -ENOSPC if no suitable hole is found, -EINTR if asked to wait for eviction and interrupted. GTT Fences and Swizzling¶
Parameters
Description This function force-removes any fence from the given object, which is useful if the kernel wants to do untiled GTT access.
Parameters
Description When mapping objects through the GTT, userspace wants to be able to write to them without having to worry about swizzling if the object is tiled. This function walks the fence regs looking for a free one for obj, stealing one if it can’t find any. It then sets up the reg based on the object’s properties: address, pitch and tiling format. For an untiled surface, this removes any existing fence. 0 on success, negative error code on failure. Return
Parameters
Description This function walks the fence regs looking for a free one and remove it from the fence_list. It is used to reserve fence for vGPU to use.
Parameters
Description This function add a reserved fence register from vGPU to the fence_list.
Parameters
Description Restore the hw fence state to match the software tracking again, to be called after a gpu reset and on resume. Note that on runtime suspend we only cancel the fences, to be reacquired by the user later.
Parameters
Description Detects bit 6 swizzling of address lookup between IGD access and CPU access through main memory.
Parameters
Description This function fixes up the swizzling in case any page frame number for this
object has changed in bit 17 since that state has been saved with
This is called when pinning backing storage again, since the kernel is free to move unpinned backing storage around (either by directly moving pages or by swapping them out and back in again).
Parameters
Description This function saves the bit 17 of each page frame number so that swizzling
can be fixed up later on with Global GTT Fence Handling¶Important to avoid confusions: “fences” in the i915 driver are not execution fences used to track command completion but hardware detiler objects which wrap a given range of the global GTT. Each platform has only a fairly limited set of these objects. Fences are used to detile GTT memory mappings. They’re also connected to the hardware frontbuffer render tracking and hence interact with frontbuffer compression. Furthermore on older platforms fences are required for tiled objects used by the display engine. They can also be used by the render engine - they’re required for blitter commands and are optional for render commands. But on gen4+ both display (with the exception of fbc) and rendering have their own tiling state bits and don’t need fences. Also note that fences only support X and Y tiling and hence can’t be used for the fancier new tiling formats like W, Ys and Yf. Finally note that because fences are such a restricted resource they’re dynamically associated with objects. Furthermore fence state is committed to the hardware lazily to avoid unnecessary stalls on gen2/3. Therefore code must explicitly call i915_gem_object_get_fence() to synchronize fencing status for cpu access. Also note that some code wants an unfenced view, for those cases the fence can be removed forcefully with i915_gem_object_put_fence(). Internally these functions will synchronize with userspace access by removing CPU ptes into GTT mmaps (not the GTT ptes themselves) as needed. Hardware Tiling and Swizzling Details¶The idea behind tiling is to increase cache hit rates by rearranging pixel data so that a group of pixel accesses are in the same cacheline. Performance improvement from doing this on the back/depth buffer are on the order of 30%. Intel architectures make this somewhat more complicated, though, by adjustments made to addressing of data when the memory is in interleaved mode (matched pairs of DIMMS) to improve memory bandwidth. For interleaved memory, the CPU sends every sequential 64 bytes to an alternate memory channel so it can get the bandwidth from both. The GPU also rearranges its accesses for increased bandwidth to interleaved memory, and it matches what the CPU does for non-tiled. However, when tiled it does it a little differently, since one walks addresses not just in the X direction but also Y. So, along with alternating channels when bit 6 of the address flips, it also alternates when other bits flip – Bits 9 (every 512 bytes, an X tile scanline) and 10 (every two X tile scanlines) are common to both the 915 and 965-class hardware. The CPU also sometimes XORs in higher bits as well, to improve bandwidth doing strided access like we do so frequently in graphics. This is called “Channel XOR Randomization” in the MCH documentation. The result is that the CPU is XORing in either bit 11 or bit 17 to bit 6 of its address decode. All of this bit 6 XORing has an effect on our memory management, as we need to make sure that the 3d driver can correctly address object contents. If we don’t have interleaved memory, all tiling is safe and no swizzling is required. When bit 17 is XORed in, we simply refuse to tile at all. Bit 17 is not just a page offset, so as we page an object out and back in, individual pages in it will have different bit 17 addresses, resulting in each 64 bytes being swapped with its neighbor! Otherwise, if interleaved, we have to tell the 3d driver what the address swizzling it needs to do is, since it’s writing with the CPU to the pages (bit 6 and potentially bit 11 XORed in), and the GPU is reading from the pages (bit 6, 9, and 10 XORed in), resulting in a cumulative bit swizzling required by the CPU of XORing in bit 6, 9, 10, and potentially 11, in order to match what the GPU expects. Object Tiling IOCTLs¶
Parameters
Description Return the required global GTT size for a fence (view of a tiled object), taking into account potential fence register mapping.
Parameters
Description Return the required global GTT alignment for a fence (a view of a tiled object), taking into account potential fence register mapping.
Parameters
Description Sets the tiling mode of an object, returning the required swizzling of bit 6 of addresses in the object. Called by the user via ioctl. Return Zero on success, negative errno on failure.
Parameters
Description Returns the current tiling mode and required bit 6 swizzling for the object. Called by the user via ioctl. Return Zero on success, negative errno on failure.
In principle GEM doesn’t care at all about the internal data layout of an object, and hence it also doesn’t care about tiling or swizzling. There’s two exceptions:
Since neither of this applies for new tiling layouts on modern platforms like W, Ys and Yf tiling GEM only allows object tiling to be set to X or Y tiled. Anything else can be handled in userspace entirely without the kernel’s invovlement. Microcontrollers¶Starting from gen9, three microcontrollers are available on the HW: the graphics microcontroller (GuC), the HEVC/H.265 microcontroller (HuC) and the display microcontroller (DMC). The driver is responsible for loading the firmwares on the microcontrollers; the GuC and HuC firmwares are transferred to WOPCM using the DMA engine, while the DMC firmware is written through MMIO. WOPCM¶WOPCM Layout¶The layout of the WOPCM will be fixed after writing to GuC WOPCM size and offset registers whose values are calculated and determined by HuC/GuC firmware size and set of hardware requirements/restrictions as shown below: +=========> +====================+ <== WOPCM Top ^ | HW contexts RSVD | | +===> +====================+ <== GuC WOPCM Top | ^ | | | | | | | | | | | GuC | | | WOPCM | | | Size +--------------------+ WOPCM | | GuC FW RSVD | | | +--------------------+ | | | GuC Stack RSVD | | | +------------------- + | v | GuC WOPCM RSVD | | +===> +====================+ <== GuC WOPCM base | | WOPCM RSVD | | +------------------- + <== HuC Firmware Top v | HuC FW | +=========> +====================+ <== WOPCM Base GuC accessible WOPCM starts at GuC WOPCM base and ends at GuC WOPCM top. The top part of the WOPCM is reserved for hardware contexts (e.g. RC6 context). GuC¶The GuC is a microcontroller inside the GT HW, introduced in gen9. The GuC is designed to offload some of the functionality usually performed by the host driver; currently the main operations it can take care of are:
The enable_guc module parameter can be used to select which of those operations to enable within GuC. Note that not all the operations are supported on all gen9+ platforms. Enabling the GuC is not mandatory and therefore the firmware is only loaded if at least one of the operations is selected. However, not loading the GuC might result in the loss of some features that do require the GuC (currently just the HuC, but more are expected to land in the future). GuC Firmware Layout¶The GuC/HuC firmware layout looks like this: +======================================================================+
| Firmware blob |
+===============+===============+============+============+============+
| CSS header | uCode | RSA key | modulus | exponent |
+===============+===============+============+============+============+
<-header size-> <---header size continued ----------->
<--- size ----------------------------------------------------------->
<-key size->
<-mod size->
<-exp size->
The firmware may or may not have modulus key and exponent data. The header, uCode and RSA signature are must-have components that will be used by driver. Length of each components, which is all in dwords, can be found in header. In the case that modulus and exponent are not present in fw, a.k.a truncated image, the length value still appears in header. Driver will do some basic fw size validation based on the following rules:
GuC Memory Management¶GuC can’t allocate any memory for its own usage, so all the allocations must be handled by the host driver. GuC accesses the memory via the GGTT, with the exception of the top and bottom parts of the 4GB address space, which are instead re-mapped by the GuC HW to memory location of the FW itself (WOPCM) or other parts of the HW. The driver must take care not to place objects that the GuC is going to access in these reserved ranges. The layout of the GuC address space is shown below: +===========> +====================+ <== FFFF_FFFF ^ | Reserved | | +====================+ <== GUC_GGTT_TOP | | | | | DRAM | GuC | | Address +===> +====================+ <== GuC ggtt_pin_bias Space ^ | | | | | | | GuC | GuC | | WOPCM | WOPCM | | Size | | | | | | v v | | +=======+===> +====================+ <== 0000_0000 The lower part of GuC Address Space [0, ggtt_pin_bias) is mapped to GuC WOPCM while upper part of GuC Address Space [ggtt_pin_bias, GUC_GGTT_TOP) is mapped to DRAM. The value of the GuC ggtt_pin_bias is the GuC WOPCM size.
Parameters
Description This is a wrapper to create an object for use with the GuC. In order to use it inside the GuC, an object needs to be pinned lifetime, so we allocate both some backing storage and a range inside the Global GTT. We must pin it in the GGTT somewhere other than than [0, GUC ggtt_pin_bias) because that range is reserved inside GuC. Return A i915_vma if successful, otherwise an ERR_PTR. GuC-specific firmware loader¶
Parameters
Description Called from intel_uc_init_hw() during driver load, resume from sleep and after a GPU reset. The firmware image should have already been fetched into memory, so only check that fetch succeeded, and then transfer the image to the h/w. Return non-zero code on error GuC-based command submission¶IMPORTANT NOTE: GuC submission is currently not supported in i915. The GuC firmware is moving to an updated submission interface and we plan to turn submission back on when that lands. The below documentation (and related code) matches the old submission model and will be updated as part of the upgrade to the new flow. GuC stage descriptor: During initialization, the driver allocates a static pool of 1024 such descriptors, and shares them with the GuC. Currently, we only use one descriptor. This stage descriptor lets the GuC know about the workqueue and process descriptor. Theoretically, it also lets the GuC know about our HW contexts (context ID, etc…), but we actually employ a kind of submission where the GuC uses the LRCA sent via the work item instead. This is called a “proxy” submission. The Scratch registers: There are 16 MMIO-based registers start from 0xC180. The kernel driver writes a value to the action register (SOFT_SCRATCH_0) along with any data. It then triggers an interrupt on the GuC via another register write (0xC4C8). Firmware writes a success/fail code back to the action register after processes the request. The kernel driver polls waiting for this update and then proceeds. Work Items: There are several types of work items that the host may place into a workqueue, each with its own requirements and limitations. Currently only WQ_TYPE_INORDER is needed to support legacy submission via GuC, which represents in-order queue. The kernel driver packs ring tail pointer and an ELSP context descriptor dword into Work Item. See guc_add_request() GuC ABI¶HXG Message All messages exchanged with GuC are defined using 32 bit dwords. First dword is treated as a message header. Remaining dwords are optional.
HXG Request The HXG Request message should be used to initiate synchronous activity for which confirmation or return data is expected. The recipient of this message shall use HXG Response, HXG Failure or HXG Retry message as a definite reply, and may use HXG Busy message as a intermediate reply. Format of DATA0 and all DATAn fields depends on the ACTION code.
HXG Event The HXG Event message should be used to initiate asynchronous activity that does not involves immediate confirmation nor data. Format of DATA0 and all DATAn fields depends on the ACTION code.
HXG Busy The HXG Busy message may be used to acknowledge reception of the HXG Request message if the recipient expects that it processing will be longer than default timeout. The COUNTER field may be used as a progress indicator.
HXG Retry The HXG Retry message should be used by recipient to indicate that the HXG Request message was dropped and it should be resent again. The REASON field may be used to provide additional information.
HXG Failure The HXG Failure message shall be used as a reply to the HXG Request message that could not be processed due to an error.
HXG Response The HXG Response message shall be used as a reply to the HXG Request message that was successfully processed without an error.
GuC MMIO based communication The MMIO based communication between Host and GuC relies on special hardware registers which format could be defined by the software (so called scratch registers). Each MMIO based message, both Host to GuC (H2G) and GuC to Host (G2H) messages, which maximum length depends on number of available scratch registers, is directly written into those scratch registers. For Gen9+, there are 16 software scratch registers 0xC180-0xC1B8, but no H2G command takes more than 4 parameters and the GuC firmware itself uses an 4-element array to store the H2G message. For Gen11+, there are additional 4 registers 0x190240-0x19024C, which are, regardless on lower count, preferred over legacy ones. The MMIO based communication is mainly used during driver initialization phase to setup the CTB based communication that will be used afterwards. MMIO HXG Message Format of the MMIO messages follows definitions of HXG Message.
CT Buffer Circular buffer used to send CTB Message CTB Descriptor
CTB Message
CTB HXG Message
CTB based communication The CTB (command transport buffer) communication between Host and GuC is based on u32 data stream written to the shared buffer. One buffer can be used to transmit data only in one direction (one-directional channel). Current status of the each buffer is stored in the buffer descriptor. Buffer descriptor holds tail and head fields that represents active data stream. The tail field is updated by the data producer (sender), and head field is updated by the data consumer (receiver): +------------+ | DESCRIPTOR | +=================+============+========+ +============+ | | MESSAGE(s) | | | address |--------->+=================+============+========+ +------------+ | head | ^-----head--------^ +------------+ | tail | ^---------tail-----------------^ +------------+ | size | ^---------------size--------------------^ +------------+ Each message in data stream starts with the single u32 treated as a header, followed by optional set of u32 data that makes message specific payload: +------------+---------+---------+---------+
| MESSAGE |
+------------+---------+---------+---------+
| msg[0] | [1] | ... | [n-1] |
+------------+---------+---------+---------+
| MESSAGE | MESSAGE PAYLOAD |
+ HEADER +---------+---------+---------+
| | 0 | ... | n |
+======+=====+=========+=========+=========+
| 31:16| code| | | |
+------+-----+ | | |
| 15:5|flags| | | |
+------+-----+ | | |
| 4:0| len| | | |
+------+-----+---------+---------+---------+
^-------------len-------------^
The message header consists of:
HOST2GUC_REGISTER_CTB This message is used as part of the CTB based communication setup. This message must be sent as MMIO HXG Message.
HOST2GUC_DEREGISTER_CTB This message is used as part of the CTB based communication teardown. This message must be sent as MMIO HXG Message.
HuC¶The HuC is a dedicated microcontroller for usage in media HEVC (High Efficiency Video Coding) operations. Userspace can directly use the firmware capabilities by adding HuC specific commands to batch buffers. The kernel driver is only responsible for loading the HuC firmware and triggering its security authentication, which is performed by the GuC. For The GuC to correctly perform the authentication, the HuC binary must be loaded before the GuC one. Loading the HuC is optional; however, not using the HuC might negatively impact power usage and/or performance of media workloads, depending on the use-cases. See https://github.com/intel/media-driver for the latest details on HuC functionality.
Parameters
Description Called after HuC and GuC firmware loading during intel_uc_init_hw(). This function invokes the GuC action to authenticate the HuC firmware, passing the offset of the RSA signature to intel_guc_auth_huc(). It then waits for up to 50ms for firmware verification ACK. HuC Memory Management¶Similarly to the GuC, the HuC can’t do any memory allocations on its own, with the difference being that the allocations for HuC usage are handled by the userspace driver instead of the kernel one. The HuC accesses the memory via the PPGTT belonging to the context loaded on the VCS executing the HuC-specific commands. HuC Firmware Layout¶The HuC FW layout is the same as the GuC one, see GuC Firmware Layout DMC¶Tracing¶This sections covers all things related to the tracepoints implemented in the i915 driver. i915_ppgtt_create and i915_ppgtt_release¶With full ppgtt enabled each process using drm will allocate at least one translation table. With these traces it is possible to keep track of the allocation and of the lifetime of the tables; this can be used during testing/debug to verify that we are not leaking ppgtts. These traces identify the ppgtt through the vm pointer, which is also printed by the i915_vma_bind and i915_vma_unbind tracepoints. i915_context_create and i915_context_free¶These tracepoints are used to track creation and deletion of contexts. If full ppgtt is enabled, they also print the address of the vm assigned to the context. Perf¶Overview¶Gen graphics supports a large number of performance counters that can help driver and application developers understand and optimize their use of the GPU. This i915 perf interface enables userspace to configure and open a file descriptor representing a stream of GPU metrics which can then be read() as a stream of sample records. The interface is particularly suited to exposing buffered metrics that are captured by DMA from the GPU, unsynchronized with and unrelated to the CPU. Streams representing a single context are accessible to applications with a corresponding drm file descriptor, such that OpenGL can use the interface without special privileges. Access to system-wide metrics requires root privileges by default, unless changed via the dev.i915.perf_event_paranoid sysctl option. Comparison with Core Perf¶The interface was initially inspired by the core Perf infrastructure but some notable differences are: i915 perf file descriptors represent a “stream” instead of an “event”; where a perf event primarily corresponds to a single 64bit value, while a stream might sample sets of tightly-coupled counters, depending on the configuration. For example the Gen OA unit isn’t designed to support orthogonal configurations of individual counters; it’s configured for a set of related counters. Samples for an i915 perf stream capturing OA metrics will include a set of counter values packed in a compact HW specific format. The OA unit supports a number of different packing formats which can be selected by the user opening the stream. Perf has support for grouping events, but each event in the group is configured, validated and authenticated individually with separate system calls. i915 perf stream configurations are provided as an array of u64 (key,value) pairs, instead of a fixed struct with multiple miscellaneous config members, interleaved with event-type specific members. i915 perf doesn’t support exposing metrics via an mmap’d circular buffer. The supported metrics are being written to memory by the GPU unsynchronized with the CPU, using HW specific packing formats for counter sets. Sometimes the constraints on HW configuration require reports to be filtered before it would be acceptable to expose them to unprivileged applications - to hide the metrics of other processes/contexts. For these use cases a read() based interface is a good fit, and provides an opportunity to filter data as it gets copied from the GPU mapped buffers to userspace buffers. Issues hit with first prototype based on Core Perf¶The first prototype of this driver was based on the core perf infrastructure, and while we did make that mostly work, with some changes to perf, we found we were breaking or working around too many assumptions baked into perf’s currently cpu centric design. In the end we didn’t see a clear benefit to making perf’s implementation and interface more complex by changing design assumptions while we knew we still wouldn’t be able to use any existing perf based userspace tools. Also considering the Gen specific nature of the Observability hardware and how userspace will sometimes need to combine i915 perf OA metrics with side-band OA data captured via MI_REPORT_PERF_COUNT commands; we’re expecting the interface to be used by a platform specific userspace such as OpenGL or tools. This is to say; we aren’t inherently missing out on having a standard vendor/architecture agnostic interface by not using perf. For posterity, in case we might re-visit trying to adapt core perf to be better suited to exposing i915 metrics these were the main pain points we hit:
i915 Driver Entry Points¶This section covers the entrypoints exported outside of i915_perf.c to integrate with drm/i915 and to handle the DRM_I915_PERF_OPEN ioctl.
Parameters
Description Initializes i915-perf state without exposing anything to userspace. Note i915-perf initialization is split into an ‘init’ and ‘register’
phase with the
Parameters
Parameters
Description In particular OA metric sets are advertised under a sysfs metrics/ directory allowing userspace to enumerate valid IDs that can be used to open an i915-perf stream.
Parameters
Description i915-perf state cleanup is split up into an ‘unregister’ and
‘deinit’ phase where the interface is first hidden from
userspace by
Parameters
Description Validates the stream open parameters given by userspace including flags and an array of u64 key, value pair properties. Very little is assumed up front about the nature of the stream being opened (for instance we don’t assume it’s for periodic OA unit metrics). An i915-perf stream is expected to be a suitable interface for other forms of buffered data written by the GPU besides periodic OA metrics. Note we copy the properties from userspace outside of the i915 perf mutex to avoid an awkward lockdep with mmap_lock. Most of the implementation details are handled by
Return A newly opened i915 Perf stream file descriptor or negative error code on failure.
Parameters
Description Cleans up any resources associated with an open i915 perf stream file. NB: close() can’t really fail from the userspace point of view. Return zero on success or a negative error code.
Parameters
Description Validates the submitted OA register to be saved into a new OA config that can then be used for programming the OA unit and its NOA network. Return A new allocated config number to be used with the perf open ioctl or a negative error code on failure.
Parameters
Description Configs can be removed while being used, the will stop appearing in sysfs and their content will be freed when the stream using the config is closed. Return 0 on success or a negative error code on failure. i915 Perf Stream¶This section covers the stream-semantics-agnostic structures and functions for representing an i915 perf stream FD and associated file operations.
Definition struct i915_perf_stream {
struct i915_perf *perf;
struct intel_uncore *uncore;
struct intel_engine_cs *engine;
u32 sample_flags;
int sample_size;
struct i915_gem_context *ctx;
bool enabled;
bool hold_preemption;
const struct i915_perf_stream_ops *ops;
struct i915_oa_config *oa_config;
struct llist_head oa_config_bos;
struct intel_context *pinned_ctx;
u32 specific_ctx_id;
u32 specific_ctx_id_mask;
struct hrtimer poll_check_timer;
wait_queue_head_t poll_wq;
bool pollin;
bool periodic;
int period_exponent;
struct {
struct i915_vma *vma;
u8 *vaddr;
u32 last_ctx_id;
int format;
int format_size;
int size_exponent;
spinlock_t ptr_lock;
u32 aging_tail;
u64 aging_timestamp;
u32 head;
u32 tail;
} oa_buffer;
struct i915_vma *noa_wait;
u64 poll_oa_period;
};
Members
Definition struct i915_perf_stream_ops {
void (*enable)(struct i915_perf_stream *stream);
void (*disable)(struct i915_perf_stream *stream);
void (*poll_wait)(struct i915_perf_stream *stream,struct file *file, poll_table *wait);
int (*wait_unlocked)(struct i915_perf_stream *stream);
int (*read)(struct i915_perf_stream *stream,char __user *buf,size_t count, size_t *offset);
void (*destroy)(struct i915_perf_stream *stream);
};
Members
Parameters
Description Note this function only validates properties in isolation it doesn’t validate that the combination of properties makes sense or that all properties necessary for a particular kind of stream have been set. Note that there currently aren’t any ordering requirements for properties so we shouldn’t validate or assume anything about ordering here. This doesn’t rule out defining new properties with ordering requirements in the future.
Parameters
Description See i915_perf_ioctl_open() for interface details. Implements further stream config validation and stream initialization on
behalf of In the case where userspace is interested in OA unit metrics then further
config validation and stream initialization details will be handled by
Note at this point the props have only been validated in isolation and it’s still necessary to validate that the combination of properties makes sense. Return zero on success or a negative error code.
Parameters
Description Frees all resources associated with the given i915 perf stream, disabling any associated data capture in the process. Note The
Parameters
Description The entry point for handling a read() on a stream file descriptor from
userspace. Most of the work is left to the i915_perf_read_locked() and
We can also consistently treat trying to read from a disabled stream as an IO error so implementations can assume the stream is enabled while reading. Return The number of bytes copied or a negative error code on failure.
Parameters
Description Implementation deferred to Return zero on success or a negative error code. Returns -EINVAL for an unknown ioctl request.
Parameters
Description [Re]enables the associated capture of data for this stream. If a stream was previously enabled then there’s currently no intention to provide userspace any guarantee about the preservation of previously buffered data.
Parameters
Description Disables the associated capture of data for this stream. The intention is that disabling an re-enabling a stream will ideally be cheaper than destroying and re-opening a stream with the same configuration, though there are no formal guarantees about what state or buffered data must be retained between disabling and re-enabling a stream. Note while a stream is disabled it’s considered an error for userspace to attempt to read from the stream (-EIO).
Parameters
Description For handling userspace polling on an i915 perf stream, this ensures poll_wait() gets called with a wait queue that will be woken for new stream data. Note Implementation deferred to Return any poll events that are ready without sleeping
Parameters
Description For handling userspace polling on an i915 perf stream, this calls through to
Note The Return any poll events that are ready without sleeping i915 Perf Observation Architecture Stream¶
Definition struct i915_oa_ops {
bool (*is_valid_b_counter_reg)(struct i915_perf *perf, u32 addr);
bool (*is_valid_mux_reg)(struct i915_perf *perf, u32 addr);
bool (*is_valid_flex_reg)(struct i915_perf *perf, u32 addr);
int (*enable_metric_set)(struct i915_perf_stream *stream, struct i915_active *active);
void (*disable_metric_set)(struct i915_perf_stream *stream);
void (*oa_enable)(struct i915_perf_stream *stream);
void (*oa_disable)(struct i915_perf_stream *stream);
int (*read)(struct i915_perf_stream *stream,char __user *buf,size_t count, size_t *offset);
u32 (*oa_hw_tail_read)(struct i915_perf_stream *stream);
};
Members
Parameters
Description While At this point it has been determined that userspace wants a stream of OA metrics, but still we need to further validate the combined properties are OK. If the configuration makes sense then we can allocate memory for a circular OA buffer and apply the requested metric set configuration. Return zero on success or a negative error code.
Parameters
Description Updates offset according to the number of bytes successfully copied into the userspace buffer. Return zero on success or a negative error code
Parameters
Description [Re]enables hardware periodic sampling according to the period configured when opening the stream. This also starts a hrtimer that will periodically check for data in the circular OA buffer for notifying userspace (e.g. during a read() or poll()).
Parameters
Description Stops the OA unit from periodically writing counter reports into the circular OA buffer. This also stops the hrtimer that periodically checks for data in the circular OA buffer, for notifying userspace.
Parameters
Description Called when userspace tries to read() from a blocking stream FD opened for OA metrics. It waits until the hrtimer callback finds a non-empty OA buffer and wakes us. Note it’s acceptable to have this return with some false positives since any subsequent read handling will return -EAGAIN if there isn’t really data ready for userspace yet. Return zero on success or a negative error code
Parameters
Description For handling userspace polling on an i915 perf stream opened for OA metrics, this starts a poll_wait with the wait queue that our hrtimer callback wakes when it sees data ready to read in the circular OA buffer. Other i915 Perf Internals¶This section simply includes all other currently documented i915 perf internals, in no particular order, but may include some more minor utilities or platform specific details than found in the more high-level sections.
Definition struct perf_open_properties {
u32 sample_flags;
u64 single_context:1;
u64 hold_preemption:1;
u64 ctx_handle;
int metrics_set;
int oa_format;
bool oa_periodic;
int oa_period_exponent;
struct intel_engine_cs *engine;
bool has_sseu;
struct intel_sseu sseu;
u64 poll_oa_period;
};
Members
Description As
Parameters
Description This is either called via fops (for blocking reads in user ctx) or the poll check hrtimer (atomic ctx) to check the OA buffer tail pointer and check if there is data available for userspace to read. This function is central to providing a workaround for the OA unit tail pointer having a race with respect to what data is visible to the CPU. It is responsible for reading tail pointers from the hardware and giving the pointers time to ‘age’ before they are made available for reading. (See description of OA_TAIL_MARGIN_NSEC above for further details.) Besides returning true when there is data available to read() this function also updates the tail, aging_tail and aging_timestamp in the oa_buffer object. Note It’s safe to read OA config state here unlocked, assuming that this is only called while the stream is enabled, while the global OA configuration can’t be modified. Return
Parameters
Description Writes a status record (such as DRM_I915_PERF_RECORD_OA_REPORT_LOST) into the userspace read() buffer. The buf offset will only be updated on success. Return 0 on success, negative error code on failure.
Parameters
Description The contents of a sample are configured through DRM_I915_PERF_PROP_SAMPLE_* properties when opening a stream, tracked as stream->sample_flags. This function copies the requested components of a single sample to the given read() buf. The buf offset will only be updated on success. Return 0 on success, negative error code on failure.
Parameters
Description Notably any error condition resulting in a short read (- Note reports are consumed from the head, and appended to the tail, so the tail chases the head?… If you think that’s mad and back-to-front you’re not alone, but this follows the Gen PRM naming convention. Return 0 on success, negative error code on failure.
Parameters
Description Checks OA unit status registers and if necessary appends corresponding status records for userspace (such as for a buffer full condition) and then initiate appending any buffered OA reports. Updates offset according to the number of bytes successfully copied into the userspace buffer. NB: some data may be successfully copied to the userspace buffer even if an error is returned, and this is reflected in the updated offset. Return zero on success or a negative error code
Parameters
Description Notably any error condition resulting in a short read (- Note reports are consumed from the head, and appended to the tail, so the tail chases the head?… If you think that’s mad and back-to-front you’re not alone, but this follows the Gen PRM naming convention. Return 0 on success, negative error code on failure.
Parameters
Description Checks Gen 7 specific OA unit status registers and if necessary appends corresponding status records for userspace (such as for a buffer full condition) and then initiate appending any buffered OA reports. Updates offset according to the number of bytes successfully copied into the userspace buffer. Return zero on success or a negative error code
Parameters
Description Determine the render context hw id, and ensure it remains fixed for the lifetime of the stream. This ensures that we don’t have to worry about updating the context ID in OACONTROL on the fly. Return zero on success or a negative error code
Parameters
Description In case anything needed doing to ensure the context HW ID would remain valid for the lifetime of the stream, then that can be undone here.
Parameters
Note The Return zero on success or a negative error code. Returns -EINVAL for an unknown ioctl request.
Parameters
Description This version number is used by userspace to detect available features. Style¶The drm/i915 driver codebase has some style rules in addition to (and, in some cases, deviating from) the kernel coding style. Register macro definition style¶The style guide for Follow the style described here for new macros, and while changing existing macros. Do not mass change existing definitions just to update the style. File Layout¶Keep helper macros near the top. For example, _PIPE() and friends. Prefix macros that generally should not be used outside of this file with underscore ‘_’. For example, _PIPE() and friends, single instances of registers that are defined solely for the use by function-like macros. Avoid using the underscore prefixed macros outside of this file. There are exceptions, but keep them to a minimum. There are two basic types of register definitions: Single registers and register groups. Register groups are registers which have two or more instances, for example one per pipe, port, transcoder, etc. Register groups should be defined using function-like macros. For single registers, define the register offset first, followed by register contents. For register groups, define the register instance offsets first, prefixed with underscore, followed by a function-like macro choosing the right instance based on the parameter, followed by register contents. Define the register contents (i.e. bit and bit field macros) from most
significant to least significant bit. Indent the register content macros
using two extra spaces between Define bit fields using Define bits using Group the register and its contents together without blank lines, separate from other registers and their contents with one blank line. Indent macro values from macro names using TABs. Align values vertically. Use braces in macro values as needed to avoid unintended precedence after macro substitution. Use spaces in macro values according to kernel coding style. Use lower case in hexadecimal values. Naming¶Try to name registers according to the specs. If the register name changes in the specs from platform to another, stick to the original name. Try to re-use existing register macro definitions. Only add new macros for new register offsets, or when the register contents have changed enough to warrant a full redefinition. When a register macro changes for a new platform, prefix the new macro using
the platform acronym or generation. For example, When a bit (field) macro changes or gets added for a new platform, while
retaining the existing register macro, add a platform acronym or generation
suffix to the name. For example, Examples¶(Note that the values in the example are indented using spaces instead of TABs to avoid misalignment in generated documentation. Use TABs in the definitions.): #define _FOO_A 0xf000 #define _FOO_B 0xf001 #define FOO(pipe) _MMIO_PIPE(pipe, _FOO_A, _FOO_B) #define FOO_ENABLE REG_BIT(31) #define FOO_MODE_MASK REG_GENMASK(19, 16) #define FOO_MODE_BAR REG_FIELD_PREP(FOO_MODE_MASK, 0) #define FOO_MODE_BAZ REG_FIELD_PREP(FOO_MODE_MASK, 1) #define FOO_MODE_QUX_SNB REG_FIELD_PREP(FOO_MODE_MASK, 2) #define BAR _MMIO(0xb000) #define GEN8_BAR _MMIO(0xb888) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| You are visitor 01402605 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |