ci: run the GPU tests on lavapipe on Linux and fail if they skip (#7975)

The Linux runner has no GPU, so the GPU tests ran on Mesa llvmpipe over
GL, which can't render Rgba32Float: 57 GPU checks (50 GPU parity, 3
canvas, 2 adjustment preview parity, 2 effect-map recovery) returned
early and passed without comparing anything. A new step after Test
installs mesa-vulkan-drivers and runs the nextest `gpu` group on
lavapipe (Vulkan), reusing Test's build, and fails unless the parity
tests report the Vulkan backend and no test printed a skip.

No code change: photocraft's tests take whatever adapter wgpu returns,
software adapters included. Follows the CI half of lightcraft#1148.

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

.config/nextest.toml

@@ -11,7 +11,7 @@ nextest-version = "0.9.150"
 #
 # GPU tests: each test is its own process here, so the in-process `gpu_lock()` mutexes no longer
 # serialize them. Too many concurrent wgpu devices time out (`PollError: ... Wait timed out`), so
-# this group caps them at 4.
+# this group caps them at 4. scripts/ci-gpu-lavapipe.py reads this group's filters directly.
 [test-groups]
 gpu = { max-threads = 4 }
 

.github/workflows/ci.yml

@@ -151,7 +151,7 @@ jobs:
       # egui/winit/rfd.
       packages: >-
         libxkbcommon-dev libwayland-dev libx11-dev libxrandr-dev libxi-dev libgl1-mesa-dev
-        libgtk-3-dev
+        libgtk-3-dev mesa-vulkan-drivers libvulkan1
       # The tests build with the optional craft-fonts input (crates/text/build.rs), so the
       # Japanese font tests that skip without it run. Same commit as release.yml.
       craft-fonts-ref: b076417b5880bd4154bd22f00695b1d28c6bea83
@@ -164,7 +164,7 @@ jobs:
       timeout-minutes: 100
       # fmt, clippy, layers, scorecard, tests, panic_hunt, doctests and the wasm32 check, with
       # `--locked --features heif`, then the missing-translations report (xtask/src/main.rs).
-      command: cargo xtask ci
+      command: python3 scripts/test-ci-gpu-lavapipe.py && cargo xtask ci && python3 scripts/ci-gpu-lavapipe.py
 
   # Windows tests run as two legs in parallel: the unit tests and the integration tests. As one job
   # they took ~28 min on the Windows runner; the legs take ~13 and ~15. Splitting the integration

docs/development.md

@@ -113,6 +113,8 @@ the opted-in test; an unset variable leaves the ordinary behavior tests availabl
 After review, remove only the two generated PNGs and their owned temporary directory. The same
 Rust test and variable work on Windows with the native shell's environment-variable syntax.
 
+**GPU tests without a GPU.** Linux CI uses the shared `craft-common-actions` gate with Mesa's Vulkan driver installed. After the regular gate, `python3 scripts/ci-gpu-lavapipe.py` reruns the configured `gpu` group with `WGPU_BACKEND=vulkan` and the lavapipe ICD explicitly selected. It requires Vulkan parity evidence and rejects authored skips, except the documented software-adapter timing exceptions. It reads the filter from `.config/nextest.toml`, so no second filter needs maintaining. To run locally on Linux, install `mesa-vulkan-drivers libvulkan1` and run the same script. Windows uses WARP; this check does not certify Windows skip coverage.
+
 ## Fonts (craft-fonts)
 
 Font assets shared by the Crafting Apps live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this repo: don't commit font files here (Inter and JetBrains Mono in `assets/fonts/` are the only exceptions; new fonts go to craft-fonts).

scripts/ci-gpu-lavapipe.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+"""Run the configured GPU group on lavapipe without silently losing comparisons."""
+import glob
+import os
+from pathlib import Path
+import re
+import subprocess
+import sys
+import tempfile
+import tomllib
+
+
+def gpu_filter(config):
+    filters = [row['filter'] for row in config['profile']['default']['overrides']
+               if row.get('test-group') == 'gpu']
+    if not filters:
+        raise ValueError('nextest GPU group has no filters')
+    return ' | '.join(f'({value})' for value in filters)
+
+
+def validate_log(lines):
+    vulkan = False
+    rejected = []
+    for line in lines:
+        vulkan |= 'GPU parity backend: Vulkan' in line
+        # Authored skips, not nextest's summary of ignored/filtered-out tests.
+        if re.search(r'\bskipping\b|\bno (?:GPU )?adapter\b', line):
+            rejected.append(line.rstrip())
+        if 'SKIP ' in line and not (
+            ('SKIP gpu timing:' in line and 'is a software adapter' in line)
+            or 'SKIP the comparison: this GPU finishes' in line
+            or 'SKIP the sum: this refresh still fit one command buffer' in line
+        ):
+            rejected.append(line.rstrip())
+    if not vulkan:
+        rejected.append('GPU parity tests did not report the Vulkan backend')
+    return rejected
+
+
+def main():
+    root = Path(__file__).resolve().parents[1]
+    with (root / '.config/nextest.toml').open('rb') as source:
+        selected = gpu_filter(tomllib.load(source))
+    icds = sorted(glob.glob('/usr/share/vulkan/icd.d/lvp_icd*.json'))
+    if not icds:
+        raise RuntimeError('Install mesa-vulkan-drivers and libvulkan1 before this check')
+    env = dict(os.environ, WGPU_BACKEND='vulkan', VK_ICD_FILENAMES=os.pathsep.join(icds))
+    command = ['cargo', 'nextest', 'run', '--profile', 'ci', '--workspace', '--locked',
+               '--features', 'heif', '--lib', '--bins', '--tests', '-E', selected,
+               '--success-output', 'immediate', '--color', 'never']
+    # Stream output and keep it on disk, rather than buffering every successful test in memory.
+    with tempfile.TemporaryFile(mode='w+t') as log:
+        with subprocess.Popen(command, cwd=root, env=env, stdout=subprocess.PIPE,
+                              stderr=subprocess.STDOUT, text=True) as process:
+            for line in process.stdout:
+                sys.stdout.write(line)
+                log.write(line)
+            result = process.wait()
+        log.seek(0)
+        errors = validate_log(log)
+    for error in errors:
+        print(f'::error::{error}', file=sys.stderr)
+    return result if result else int(bool(errors))
+
+
+if __name__ == '__main__':
+    sys.exit(main())

scripts/test-ci-gpu-lavapipe.py

@@ -0,0 +1,44 @@
+import importlib.util
+from pathlib import Path
+import unittest
+
+spec = importlib.util.spec_from_file_location('gpu_check', Path(__file__).with_name('ci-gpu-lavapipe.py'))
+check = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(check)
+
+
+class LogChecks(unittest.TestCase):
+    def test_requires_vulkan_evidence(self):
+        self.assertTrue(check.validate_log(['GPU parity backend: Gl']))
+        self.assertTrue(check.validate_log([]))
+
+    def test_success_and_nextest_summary(self):
+        self.assertFalse(check.validate_log(['GPU parity backend: Vulkan (llvmpipe, Cpu)',
+                                            'Summary 100 tests run: 100 passed, 3 skipped']))
+
+    def test_rejects_missing_comparisons(self):
+        for message in ['skipping GPU parity tests: no device', 'skipping: unsupported format',
+                        'no GPU adapter', 'SKIP gpu timing: no adapter',
+                        'SKIP gpu timing: no device with timestamp queries', 'SKIP unknown test']:
+            with self.subTest(message=message):
+                self.assertTrue(check.validate_log(['GPU parity backend: Vulkan', message]))
+
+    def test_allows_only_known_timing_exceptions(self):
+        self.assertFalse(check.validate_log(['GPU parity backend: Vulkan',
+            'SKIP gpu timing: llvmpipe is a software adapter, which cannot show a GPU/CPU split',
+            'SKIP the comparison: this GPU finishes a 6 MP composite in 1 ms against 2 ms of encoding',
+            'SKIP the sum: this refresh still fit one command buffer']))
+
+    def test_selects_all_gpu_filters_from_config(self):
+        config = {'profile': {'default': {'overrides': [
+            {'filter': 'package(a)', 'test-group': 'gpu'},
+            {'filter': 'package(b)', 'test-group': 'cpu'},
+            {'filter': 'test(c)', 'test-group': 'gpu'}]}}}
+        self.assertEqual(check.gpu_filter(config), '(package(a)) | (test(c))')
+        config['profile']['default']['overrides'] = []
+        with self.assertRaises(ValueError):
+            check.gpu_filter(config)
+
+
+if __name__ == '__main__':
+    unittest.main()