zaclanzon.dev

Testing the display path without a reboot

While I debugged NVIDIA display-driver faults on Linux, each experiment cost a reboot. The only check was to look at the monitor. A software decoder lets me check a display stream compression (DSC) bitstream without the display.

dscdecode decodes VESA DSC 1.1 and 1.2 streams at constant bit rate. It is C11 with libc only, built as a library and a command-line tool. I wrote it from the specification, with no VESA model code.

Agreement with the reference model

The VESA C model (version 1.67) is a black box in these tests. For each test image, the model encodes the image. The model and dscdecode then decode the same bitstream, and the outputs are compared bit for bit.

Set Streams Bit-exact
DSC 1.1 RGB 4:4:4, 8 bpc, 17 VESA evaluation images 612 612
DSC 1.1 RGB 4:4:4, 10 and 12 bpc 1,500 1,500
DSC 1.2 RGB 4:4:4, 8 to 16 bpc 2,730 2,730
DSC 1.2 YCbCr 4:4:4, 4:2:2 and 4:2:0, 8 to 16 bpc 4,368 4,368
Hand-derived fixtures 24 24
Discriminators 36 36
Total 9,270 9,270

One release binary ran every row. dscdecode has not been through VESA compliance testing.

Where the specification is ambiguous

The specification says the reference model takes precedence where the two differ (DSC 1.2b §1.4.3). Where the text supports two readings, each reading is a runtime switch. For each open question I built a discriminator: a bitstream that decodes differently under each reading. I committed its predicted outputs before the model decoded it. The default reading is the one the model’s output matched.

Agent test runs

At San Francisco Tech Week’s Model Training Night, I tried to label coding-agent traces by their test results. The traces were six recorded coding-agent sessions on dscdecode. The plan was to label each test-harness call PASS or FAIL by its exit code.

Harness calls Count
In the six sessions 91
Usable: the harness was the last command on its line 1
Dropped: inside a loop, a pipe, or before ;, || or & 90

In 90 of 91 calls, the exit code did not report the test result. I built the labels another way. I generated mutants of dscdecode and graded each one by running the reference-model oracle in 27 configurations. A mutant fails if any configuration exits 1:

Split Mutants Pass Fail
Train 216 61 155
Test, 7 functions not in train 49 10 39
Sanity, behavior-preserving 133 133 0
Total 398 204 194

Source