Four displays, eight physical windows

The starting problem was a specific configuration on an RTX 5090: four displays worked in Windows and worked on Linux with overlay layers disabled, but the Linux driver rejected the same modes with overlay layers enabled. That comparison narrowed the investigation to how the driver handled overlay resources. Reproduction and target modes.

The configuration combined two high-refresh displays with a 4K secondary display and the HYTE case panel:

These modes required six hardware tiles. Reserving two layers for every tile consumed twelve physical windows on a GPU with eight—even where usage validation had already marked the additional layers unusable.

Allocation was only part of the fix

The implementation allocates physical windows according to validated layer usage. It also keeps hardware ownership consistent with the new assignments, while preserving advertised overlay planes and software mappings.

Changing the allocation alone was not sufficient. The investigation included earlier changes that produced black screens and Xid 56 errors and were rolled back. The working combination handled physical assignments, ownership, and modeset sequencing together. Patch and investigation account.

The production diff makes three related changes:

• Allocate only usable layers. Excluded layers do not keep or acquire physical windows. Allocation still considers changed layer usage when the number of tiles is unchanged.
• Synchronize ownership safely. Assignment and ownership changes take the head-shutdown path before reprogramming. The final owner is written even when a cached value already appears correct, because initialization may have queued a different binding.
• Make the sequencing decision after preparation. The decision to separate window flips from the core modeset update runs after PreUpdate, so it includes assignment changes discovered there.

A separate hardening change retries a failed head after reclaiming resources, instead of allowing an incomplete allocation to appear successful. It also avoids an undefined shift when constructing a 32-window mask. That boundary defect was separate from the eight-window failure. Production diff.

What was validated

The published hardware checks used Ubuntu, NVIDIA 610.43.02, kernel 7.0.0-30-generic, and overlays enabled. The submission itself is based on 610.57.04.

The tests do not establish simultaneous active overlays at the maximum modes where validation makes those layers unavailable. Broader hardware coverage, extended use, and suspend/resume were outside this validation. A recurring platform startup issue also prevented a claim of warm-reboot reliability. Validation details and separate regression harness.

A separate display-wake failure

A later investigation concerned a Samsung display that remained blank after wake even though it was connected and enabled. The driver retained an attached group expecting display-stream compression (DSC), but a successful read reported the sink’s DSC enable bit off.

The focused change restores DSC when a capable DisplayPort sink returns to an attached single-stream group. It uses the existing setter to restore both sink decompression and driver tracking. This is a separate fix from physical-window allocation. NVIDIA PR #1349.

Read the display-wake investigation →

Source and current state

Both upstream submissions were open and unmerged when checked on September 15, 2026:

• #1338 — Physical-window allocation, ownership, and sequencing
• #1349 — DSC restoration on SST reinsertion
• DSC hardware validation and regression tests

The allocation PR documents a joint investigation with GPT-6 Astra. The source, reproduction, and validation boundaries are published so the work can be inspected directly.