mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-07 12:13:22 +00:00
Viewer exposure on the full float range of 32 bpc frames (#2937)
* Viewer exposure on the full float range of 32 bpc frames (#2797) 32 bpc viewer frames on the desktop are rendered without the HDR step (skip_hdr_step, #2916) and kept as half floats in the RAM preview. A display pass on egui-wgpu's device turns each frame into one RGBA8 texture with mips per viewer, in this order: decode, exposure in linear light, Compand / Tone Map, encode, display colour, Show Channel, rounding. Before, exposure was an 8-bit lookup applied after tone mapping, clamping and the display conversion, so -2 EV could not bring values above 1.0 back. - engine: viewer::DisplayPass describes the pass as data; its pixel() is the CPU fallback (no device) and the shader's reference. It replaces exposure_lut / display_transform. FrameEncoding records a frame's curve and the HDR step it still needs. 32 bpc snapshots are taken without the HDR step; snapshot_rgba8 applies it. - ui-egui: FrameImage::CpuF16 (half crate, F16C at run time), byte accounting at 8 bytes a pixel in the cache and the prefetch estimates; display_gpu.rs + display_gpu.wgsl; main viewer, snapshot and passive viewers draw through the pass; the 8-bit image other panels read is made on demand with the HDR step; the Info panel shows float values. - Half-float frames are not written to the disk cache yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Viewer display pass: f16 disk cache entries, window-device gating, review fixes - Disk cache: half-float frame entries (format byte in the entry header, byte planes + LZ4); 8-bit entries keep format 0 and stay valid. 32 bpc viewer frames are written and read again, and the timeline's disk bar sees them. A damaged half-float entry is deleted. - The display pass runs on the window's egui-wgpu device, with or without the GPU compositor (still building, failed, GL fallback). Its pipelines are made inside error scopes; a backend that rejects them gets the CPU pass instead of a blank viewer. - A new LUT of the same size (another ICC profile, Preserve RGB, another display) runs the pass again. - f32 to f16: one pass per L1-sized block, the NaN / infinity fix-up on the bits, no fill of the buffer (4K: 4.7 ms, as the 8-bit conversion). - Mips of odd-sized levels include the last column / row; the snapshot's half floats and texture go with the snapshot; no placeholder texture ids; u64 size checks. - Tests: LUT profiles and negative values against the shader, Bake reruns, the passive viewer's colour, f16 disk round trips; exact comparisons allow 1 LSB on software rasterizers. Docs: the display pass and the f16 disk entries in architecture.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Viewer f16 disk-cache test: temp dir removed on failure too, wait for the queued disk write Review points: the test's directory was only removed by its last statement (now tempfile::TempDir), and the frame is released before its disk write is queued, so flush() could return early (now waits for the entry). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Viewer: oversized half-float frames use the CPU display pass, so exposure/HDR come before the averaging down Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Viewer: fmt and a debug assert for the half-float upload path Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Disk cache: persist_missing also writes half-float frames Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Passive viewer: oversized half-float frames take the CPU display pass instead of leaving a stale texture Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valentin Klein
co-authored by
Claude Sonnet 5.5
Claude Sonnet 5.5
parent
8948a7cdc9
commit
fdcc715829
Generated
+3
@@ -1970,6 +1970,7 @@ dependencies = [
|
||||
name = "effectcraft-ui-egui"
|
||||
version = "0.21.0"
|
||||
dependencies = [
|
||||
"bytemuck",
|
||||
"chrono",
|
||||
"craft-quit",
|
||||
"craft-ui",
|
||||
@@ -1992,6 +1993,7 @@ dependencies = [
|
||||
"egui_extras",
|
||||
"egui_kittest",
|
||||
"exr",
|
||||
"half",
|
||||
"image",
|
||||
"kurbo",
|
||||
"log",
|
||||
@@ -2000,6 +2002,7 @@ dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sys-locale",
|
||||
"tempfile",
|
||||
"toml",
|
||||
"web-time",
|
||||
]
|
||||
|
||||
@@ -112,6 +112,7 @@ filmcraft-geom = { git = "https://github.com/storytold/filmcraft", rev = "d5ead4
|
||||
wgpu = { version = "30.0.1", features = ["fragile-send-sync-non-atomic-wasm"] }
|
||||
bytemuck = "1"
|
||||
pollster = "1"
|
||||
half = "2.7"
|
||||
|
||||
# ui
|
||||
# Shared frontend components; keep this dependency above the engine boundary.
|
||||
|
||||
@@ -159,9 +159,14 @@ fn take_snapshot(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?;
|
||||
let t = s.time_of(cid);
|
||||
let scale = f_p(p, "scale").unwrap_or(1.0).clamp(0.05, 1.0);
|
||||
let img = s.render(cid, t, RenderOpts { scale, guides: true, view: s.view_camera(cid).filter(|_| comp.has_3d()), ..Default::default() });
|
||||
// 32 bpc: without the HDR step, so viewer exposure reaches the over-range values (the
|
||||
// display pass applies the step, as for viewer frames).
|
||||
let skip_hdr_step = s.project.settings.bit_depth.levels().is_none();
|
||||
let opts = RenderOpts { scale, guides: true, view: s.view_camera(cid).filter(|_| comp.has_3d()), skip_hdr_step, ..Default::default() };
|
||||
let img = s.render(cid, t, opts);
|
||||
let (w, h) = (img.width, img.height);
|
||||
s.snapshot = Some(Snapshot { comp: cid, time: t, scale, image: std::sync::Arc::new(img) });
|
||||
let encoding = crate::viewer::FrameEncoding::of(&s.project.settings, skip_hdr_step);
|
||||
s.snapshot = Some(Snapshot { comp: cid, time: t, scale, image: std::sync::Arc::new(img), encoding });
|
||||
s.events.push(crate::Event::ProjectChanged { revision: s.revision });
|
||||
Ok(json!({"width": w, "height": h, "time": t.seconds()}))
|
||||
}
|
||||
@@ -298,15 +303,17 @@ fn new_path(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
Ok(json!({"layer": lid.0, "group": group, "path": path_uid}))
|
||||
}
|
||||
|
||||
/// Snapshot image as straight 8-bit RGBA rows (for frontends / `--out`).
|
||||
/// Snapshot image as straight 8-bit RGBA rows (for frontends / `--out`), with the HDR step a
|
||||
/// 32 bpc snapshot was taken without.
|
||||
pub fn snapshot_rgba8(s: &Session) -> Option<(u32, u32, Vec<u8>)> {
|
||||
let snap = s.snapshot.as_ref()?;
|
||||
let img = &snap.image;
|
||||
let mut out = Vec::with_capacity(img.data.len() * 4);
|
||||
for p in &img.data {
|
||||
let a = p[3].clamp(0.0, 1.0);
|
||||
let un = |c: f32| if a > 0.0 { (c / a).clamp(0.0, 1.0) } else { 0.0 };
|
||||
out.extend([un(p[0]), un(p[1]), un(p[2]), a].map(|v| (v * 255.0 + 0.5) as u8));
|
||||
let un = |c: f32| if a > 0.0 { c / a } else { 0.0 };
|
||||
let c = snap.encoding.finish([un(p[0]), un(p[1]), un(p[2])]).map(|v| v.clamp(0.0, 1.0));
|
||||
out.extend([c[0], c[1], c[2], a].map(|v| (v * 255.0 + 0.5) as u8));
|
||||
}
|
||||
Some((img.width, img.height, out))
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ mod color_view;
|
||||
mod crypto_pick;
|
||||
mod delete_keys;
|
||||
mod disk_cache;
|
||||
mod display_pass;
|
||||
mod effects;
|
||||
mod essential;
|
||||
mod essential_more;
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
//! The viewer's display pass (`viewer::DisplayPass`): exposure on the float range of 32 bpc
|
||||
//! frames, the HDR step after exposure, the display colour after both, and the frame encodings.
|
||||
|
||||
use effectcraft_color::{ColorSpace, linear_to_srgb};
|
||||
use effectcraft_project::HdrMode;
|
||||
use effectcraft_render::RenderOpts;
|
||||
use effectcraft_render::color::{compand, tone_map};
|
||||
use effectcraft_time::Tick;
|
||||
use serde_json::json;
|
||||
|
||||
use crate::Session;
|
||||
use crate::viewer::{Channel, DisplayColor, DisplayPass, FrameEncoding, PassCurve, SimProfile, Simulation};
|
||||
|
||||
const SRGB: FrameEncoding = FrameEncoding { curve: PassCurve::Srgb, hdr: HdrMode::Clip };
|
||||
|
||||
fn rgb(encoding: FrameEncoding, stops: f32) -> DisplayPass {
|
||||
DisplayPass::new(Channel::Rgb, false, stops, encoding, None)
|
||||
}
|
||||
|
||||
/// An opaque grey of linear light `l`, sRGB-encoded as 32 bpc frames store it.
|
||||
fn grey(l: f32) -> [f32; 4] {
|
||||
let e = PassCurve::Srgb.encode(l);
|
||||
[e, e, e, 1.0]
|
||||
}
|
||||
|
||||
fn q(v: f32) -> u8 {
|
||||
(v.clamp(0.0, 1.0) * 255.0 + 0.5).floor() as u8
|
||||
}
|
||||
|
||||
/// The issue's case: over-range values come back with negative exposure (4.0 at -2 EV is white)
|
||||
/// and 0.5 at +1 EV is white; values of the 8-bit range keep today's bytes.
|
||||
#[test]
|
||||
fn exposure_reaches_over_range_values() {
|
||||
let table: [(f32, [u8; 5]); 6] = [
|
||||
(0.25, [137, 99, 71, 34, 188]),
|
||||
(0.5, [188, 137, 99, 49, 255]),
|
||||
(1.0, [255, 188, 137, 71, 255]),
|
||||
(2.0, [255, 255, 188, 99, 255]),
|
||||
(4.0, [255, 255, 255, 137, 255]),
|
||||
(16.0, [255, 255, 255, 255, 255]),
|
||||
];
|
||||
for (l, want) in table {
|
||||
for (stops, w) in [0.0, -1.0, -2.0, -4.0, 1.0].into_iter().zip(want) {
|
||||
assert_eq!(rgb(SRGB, stops).pixel(grey(l))[0], w, "linear {l} at {stops} EV");
|
||||
}
|
||||
}
|
||||
assert_eq!(rgb(SRGB, -2.0).pixel(grey(4.0)), [255, 255, 255, 255]);
|
||||
assert_eq!(rgb(SRGB, 1.0).pixel(grey(0.5)), [255, 255, 255, 255]);
|
||||
// Premultiplied: alpha stays, colour scales with it.
|
||||
let e = PassCurve::Srgb.encode(4.0);
|
||||
let half = rgb(SRGB, -2.0).pixel([e * 0.5, e * 0.5, e * 0.5, 0.5]);
|
||||
assert!(half[3] == 128 && half[..3].iter().all(|c| (127..=128).contains(c)), "{half:?}");
|
||||
}
|
||||
|
||||
/// Compand / Tone Map run after exposure: a value under the knee that exposure pushes over it
|
||||
/// is companded (not clipped), and highlights darkened by exposure are not companded first.
|
||||
#[test]
|
||||
fn the_hdr_step_runs_after_exposure() {
|
||||
let compand_enc = FrameEncoding { curve: PassCurve::Srgb, hdr: HdrMode::Compand };
|
||||
// 0.6 is below the 0.8 knee: unchanged at 0 EV, companded to about 0.97 at +1 EV.
|
||||
assert_eq!(rgb(compand_enc, 0.0).pixel(grey(0.6))[0], q(linear_to_srgb(0.6)));
|
||||
let up = rgb(compand_enc, 1.0).pixel(grey(0.6))[0];
|
||||
assert_eq!(up, q(linear_to_srgb(compand(1.2))));
|
||||
assert!(up < 255, "companded after exposure, not clipped: {up}");
|
||||
// 3.0 at -2 EV is 0.75, under the knee: compand first would have made it 0.25.
|
||||
let down = rgb(compand_enc, -2.0).pixel(grey(3.0))[0];
|
||||
assert_eq!(down, q(linear_to_srgb(0.75)));
|
||||
assert!(down > q(linear_to_srgb(compand(3.0) * 0.25)) + 50, "{down}");
|
||||
// Tone Map (luminance): the same order.
|
||||
let tm = FrameEncoding { curve: PassCurve::Srgb, hdr: HdrMode::ToneMap };
|
||||
for (l, stops) in [(0.6f32, 1.0f32), (3.0, -2.0), (16.0, -1.0), (0.1, 0.0)] {
|
||||
let k = 2f32.powf(stops);
|
||||
let want = q(linear_to_srgb(tone_map([l * k; 3])[0]));
|
||||
assert_eq!(rgb(tm, stops).pixel(grey(l))[0], want, "{l} at {stops}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The display colour converts the exposed output numbers: exposure first, then the monitor
|
||||
/// conversion (before: the conversion of the 8-bit frame, then exposure).
|
||||
#[test]
|
||||
fn display_colour_follows_exposure() {
|
||||
let dc =
|
||||
DisplayColor::new(Some(ColorSpace::Srgb), false, ColorSpace::Srgb, true, ColorSpace::DisplayP3, Simulation::default(), &Default::default()).unwrap();
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, 1.0, SRGB, Some(&dc));
|
||||
let red = [PassCurve::Srgb.encode(0.4), 0.0, 0.0, 1.0];
|
||||
let exposed = [linear_to_srgb(0.8), 0.0, 0.0];
|
||||
let want = dc.apply_straight(exposed).map(q);
|
||||
assert_eq!(pass.pixel(red)[..3], want);
|
||||
// The conversion is not applied to a single channel's view.
|
||||
assert!(DisplayPass::new(Channel::Red, false, 0.0, SRGB, Some(&dc)).color.is_none());
|
||||
}
|
||||
|
||||
/// The pass's display colour is `DisplayColor::apply_straight`: the same conversions in the same
|
||||
/// order (a few ULPs apart where the pass uses `f32` for PQ / HLG).
|
||||
#[test]
|
||||
fn pass_colour_matches_display_color() {
|
||||
let custom = Default::default();
|
||||
let mut cases = vec![];
|
||||
for display in [ColorSpace::DisplayP3, ColorSpace::Rec709, ColorSpace::Rec2020, ColorSpace::Srgb] {
|
||||
for profile in SimProfile::ALL {
|
||||
for preserve_rgb in [false, true] {
|
||||
let sim = Simulation { profile, preserve_rgb };
|
||||
cases.extend(DisplayColor::new(Some(ColorSpace::Srgb), false, ColorSpace::Srgb, true, display, sim, &custom));
|
||||
}
|
||||
}
|
||||
cases.extend(DisplayColor::new(Some(ColorSpace::Rec2020), true, ColorSpace::Srgb, false, display, Simulation::default(), &custom));
|
||||
}
|
||||
// A LUT-based profile (an uneven, channel-mixing grid).
|
||||
cases.push(DisplayColor::from_lut(std::sync::Arc::new(test_lut())));
|
||||
assert!(cases.len() > 40);
|
||||
for dc in &cases {
|
||||
let pass = dc.pass();
|
||||
for i in 0..=20 {
|
||||
for c in [[i as f32 / 20.0, 0.3, 0.9], [0.05, i as f32 / 20.0, 0.5], [1.0, 1.0, i as f32 / 20.0]] {
|
||||
let (a, b) = (pass.apply(c), dc.apply_straight(c));
|
||||
for k in 0..3 {
|
||||
// The simulation's 8-bit step can land either side of a tie.
|
||||
assert!((a[k] - b[k]).abs() < 1e-4 || ((a[k] - b[k]).abs() < 1.5 / 255.0 && c != [1.0; 3]), "{dc:?} {c:?}: {a:?} vs {b:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A 17³ LUT that mixes channels and bends the curve (as an ICC profile's device does).
|
||||
pub(crate) fn test_lut() -> effectcraft_color::icc::Lut3d {
|
||||
effectcraft_color::icc::Lut3d::bake(17, |c| [c[0].powf(1.3) * 0.9 + c[1] * 0.1, c[1] * 0.8 + c[2] * 0.2, (c[2] * 0.7 + c[0] * 0.3).sqrt()])
|
||||
}
|
||||
|
||||
/// Frames keep their encoding: the output space's curve (the Working Gamma's when unmanaged) and
|
||||
/// the HDR step only when it was skipped and applies (managed, SDR output).
|
||||
#[test]
|
||||
fn frame_encodings() {
|
||||
let mut s = Session::default();
|
||||
s.execute("comp.new", json!({"width": 8, "height": 8, "duration": 1})).unwrap();
|
||||
assert_eq!(FrameEncoding::of(&s.project.settings, true), SRGB, "unmanaged: sRGB at Working Gamma 2.2, no HDR step");
|
||||
s.execute("file.projectSettings", json!({"bitDepth": 32, "workingSpace": "srgb", "hdr": "compand"})).unwrap();
|
||||
let settings = s.project.settings.clone();
|
||||
assert_eq!(FrameEncoding::of(&settings, true), FrameEncoding { curve: PassCurve::Srgb, hdr: HdrMode::Compand });
|
||||
assert_eq!(FrameEncoding::of(&settings, false), SRGB, "rendered with the step: nothing pending");
|
||||
s.execute("file.projectSettings", json!({"outputSpace": "rec2100pq"})).unwrap();
|
||||
assert_eq!(FrameEncoding::of(&s.project.settings, true), FrameEncoding { curve: PassCurve::Pq, hdr: HdrMode::Clip }, "HDR outputs never tone map");
|
||||
s.execute("file.projectSettings", json!({"outputSpace": "rec709"})).unwrap();
|
||||
assert_eq!(FrameEncoding::of(&s.project.settings, true).curve, PassCurve::Gamma(2.4));
|
||||
}
|
||||
|
||||
/// A 32 bpc Compand project with an over-range layer: `FrameEncoding::finish` on the frame
|
||||
/// rendered without the HDR step gives the frame rendered with it, and Take Snapshot (now taken
|
||||
/// without the step) exports the same bytes as before, within a rounding step.
|
||||
#[test]
|
||||
fn the_skipped_hdr_step_finishes_like_the_render() {
|
||||
let mut s = Session::default();
|
||||
s.execute("comp.new", json!({"width": 16, "height": 8, "duration": 1})).unwrap();
|
||||
s.execute("file.projectSettings", json!({"bitDepth": 32, "workingSpace": "srgb", "hdr": "compand"})).unwrap();
|
||||
let lid = s.execute("layer.newSolid", json!({"color": "#c08040", "width": 16, "height": 8})).unwrap()["layer"].as_u64().unwrap();
|
||||
s.execute("effect.apply", json!({"layers": [lid], "effect": "Exposure"})).unwrap();
|
||||
s.execute("prop.set", json!({"layer": lid, "path": "effects/#1/master/exposure", "value": 1.5})).unwrap();
|
||||
let cid = s.active_comp_id().unwrap();
|
||||
let with = s.render(cid, Tick::ZERO, RenderOpts::default());
|
||||
let without = s.render(cid, Tick::ZERO, RenderOpts { skip_hdr_step: true, ..Default::default() });
|
||||
let enc = FrameEncoding::of(&s.project.settings, true);
|
||||
assert_eq!(enc.hdr, HdrMode::Compand);
|
||||
let (w, o) = (with.data[0], without.data[0]);
|
||||
assert!(o[0] > 1.0 && w[0] < 1.0, "over range without the step, companded with it: {o:?} {w:?}");
|
||||
let f = enc.finish([o[0], o[1], o[2]]);
|
||||
for k in 0..3 {
|
||||
assert!((f[k] - w[k]).abs() < 1e-4, "{f:?} vs {w:?}");
|
||||
}
|
||||
s.execute("view.takeSnapshot", json!({})).unwrap();
|
||||
assert_eq!(s.snapshot.as_ref().unwrap().encoding, enc);
|
||||
let (_, _, rgba) = crate::commands::viewer_cmds::snapshot_rgba8(&s).unwrap();
|
||||
let before = w.map(|v| (v.clamp(0.0, 1.0) * 255.0 + 0.5) as u8);
|
||||
for k in 0..4 {
|
||||
assert!((rgba[k] as i32 - before[k] as i32).abs() <= 1, "{:?} vs {before:?}", &rgba[..4]);
|
||||
}
|
||||
// 8 bpc snapshots are rendered with the step, as before.
|
||||
s.execute("file.projectSettings", json!({"bitDepth": 8})).unwrap();
|
||||
s.execute("view.takeSnapshot", json!({})).unwrap();
|
||||
assert_eq!(s.snapshot.as_ref().unwrap().encoding.hdr, HdrMode::Clip);
|
||||
}
|
||||
|
||||
/// The 8-bit CPU path's tables give what the pass gives pixel by pixel: every value of every
|
||||
/// channel view, opaque and not, with and without exposure.
|
||||
#[test]
|
||||
fn the_8_bit_tables_are_the_pass() {
|
||||
for (channel, colorized) in
|
||||
[(Channel::Rgb, false), (Channel::RgbStraight, false), (Channel::Red, false), (Channel::Green, true), (Channel::Blue, false), (Channel::Alpha, false)]
|
||||
{
|
||||
for stops in [0.0f32, 1.0, -2.5] {
|
||||
let pass = DisplayPass::new(channel, colorized, stops, SRGB, None);
|
||||
let px: Vec<[u8; 4]> = (0..256 * 4)
|
||||
.map(|i| {
|
||||
let v = (i % 256) as u8;
|
||||
let a = [255u8, 255, 128, 3][i / 256];
|
||||
let c = |k: u8| ((v as u32 * k as u32) % 256 * a as u32 / 255) as u8;
|
||||
[c(1), c(7), c(13), a]
|
||||
})
|
||||
.collect();
|
||||
let mut got = px.clone();
|
||||
pass.apply_rgba8(&mut got);
|
||||
for (p, g) in px.iter().zip(&got) {
|
||||
assert_eq!(*g, pass.pixel(p.map(|v| v as f32 / 255.0)), "{channel:?} {colorized} {stops}: {p:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -8,7 +8,7 @@ use effectcraft_time::Tick;
|
||||
use serde_json::json;
|
||||
|
||||
use crate::Session;
|
||||
use crate::viewer::{Channel, PRI_GRID, PRI_GUIDE, PRI_LAYER, PRI_VERTEX, SnapKind, SnapSource, SnapTarget, display_transform, snap};
|
||||
use crate::viewer::{Channel, DisplayPass, FrameEncoding, PRI_GRID, PRI_GUIDE, PRI_LAYER, PRI_VERTEX, PassCurve, SnapKind, SnapSource, SnapTarget, snap};
|
||||
|
||||
fn comp() -> Session {
|
||||
let mut s = Session::default();
|
||||
@@ -225,15 +225,19 @@ fn snapping_options_command_sets_and_toggles() {
|
||||
|
||||
// ---------------------------------------------------------------- channels, exposure, snapshot
|
||||
|
||||
/// The display pass on 8-bit frames (sRGB, no HDR step): Show Channel and exposure.
|
||||
fn display_rgba8(px: [u8; 4], channel: Channel, colorized: bool, stops: f32) -> [u8; 4] {
|
||||
let encoding = FrameEncoding { curve: PassCurve::Srgb, hdr: effectcraft_project::HdrMode::Clip };
|
||||
let mut v = [px];
|
||||
DisplayPass::new(channel, colorized, stops, encoding, None).apply_rgba8(&mut v);
|
||||
v[0]
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn channel_and_exposure_pixel_transforms() {
|
||||
// Premultiplied: straight (200, 100, 50) at 50% alpha.
|
||||
let px = [100u8, 50, 25, 128];
|
||||
let run = |c: Channel, col: bool, stops: f32| {
|
||||
let mut v = [px];
|
||||
display_transform(&mut v, c, col, stops);
|
||||
v[0]
|
||||
};
|
||||
let run = |c: Channel, col: bool, stops: f32| display_rgba8(px, c, col, stops);
|
||||
assert_eq!(run(Channel::Rgb, false, 0.0), px);
|
||||
assert_eq!(run(Channel::Red, false, 0.0), [100, 100, 100, 255]);
|
||||
assert_eq!(run(Channel::Green, true, 0.0), [0, 50, 0, 255]);
|
||||
@@ -242,13 +246,13 @@ fn channel_and_exposure_pixel_transforms() {
|
||||
let st = run(Channel::RgbStraight, false, 0.0);
|
||||
assert!((st[0] as i32 - 199).abs() <= 1 && (st[1] as i32 - 100).abs() <= 1 && st[3] == 255, "{st:?}");
|
||||
// +1 stop doubles linear light: sRGB 50% gray → about 69%; -1 stop darkens; 0 is identity.
|
||||
let mut g = [[128u8, 128, 128, 255]];
|
||||
display_transform(&mut g, Channel::Rgb, false, 1.0);
|
||||
assert!((174..=178).contains(&g[0][0]), "{:?}", g[0]);
|
||||
let mut g = [[128u8, 128, 128, 255]];
|
||||
display_transform(&mut g, Channel::Rgb, false, -1.0);
|
||||
assert!((92..=96).contains(&g[0][0]), "{:?}", g[0]);
|
||||
assert_eq!(crate::viewer::exposure_lut(0.0)[200], 200);
|
||||
let g = display_rgba8([128, 128, 128, 255], Channel::Rgb, false, 1.0);
|
||||
assert!((174..=178).contains(&g[0]), "{g:?}");
|
||||
let g = display_rgba8([128, 128, 128, 255], Channel::Rgb, false, -1.0);
|
||||
assert!((92..=96).contains(&g[0]), "{g:?}");
|
||||
for v in 0..=255u8 {
|
||||
assert_eq!(display_rgba8([v, v, v, 255], Channel::Rgb, false, 0.0), [v, v, v, 255]);
|
||||
}
|
||||
// Exposure applies to a single channel too, and keeps alpha untouched in RGB.
|
||||
assert!(run(Channel::Red, false, 1.0)[0] > 100);
|
||||
assert_eq!(run(Channel::Rgb, false, 2.0)[3], 128);
|
||||
|
||||
+13
-58
@@ -1,5 +1,5 @@
|
||||
//! Composition viewer maths shared by the frontends and tests: snapping (targets, tolerance and
|
||||
//! priority), the Show Channel / Exposure display transforms, and the viewer snapshot.
|
||||
//! priority), the display pass (Show Channel, exposure, display colour), and the viewer snapshot.
|
||||
//!
|
||||
//! Nothing here touches the project; edits go through commands.
|
||||
|
||||
@@ -15,6 +15,9 @@ use effectcraft_render::{EvalCtx, ExprHost};
|
||||
use effectcraft_time::Tick;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
mod display_pass;
|
||||
pub use display_pass::{DisplayPass, FrameEncoding, PassColor, PassConv, PassCurve, SHADER_EXPONENTS, hdr_step, lut_eval};
|
||||
|
||||
// ---------------------------------------------------------------- snapping
|
||||
|
||||
/// What a snap target is: a point (snaps both axes) or an axis-aligned line.
|
||||
@@ -500,63 +503,6 @@ fn linear_to_srgb(v: f32) -> f32 {
|
||||
if v <= 0.003_130_8 { v * 12.92 } else { 1.055 * v.powf(1.0 / 2.4) - 0.055 }
|
||||
}
|
||||
|
||||
/// 8-bit exposure lookup table: `stops` applied in linear light.
|
||||
pub fn exposure_lut(stops: f32) -> [u8; 256] {
|
||||
let k = 2f32.powf(stops);
|
||||
let mut lut = [0u8; 256];
|
||||
for (i, o) in lut.iter_mut().enumerate() {
|
||||
let lin = srgb_to_linear(i as f32 / 255.0) * k;
|
||||
*o = (linear_to_srgb(lin).clamp(0.0, 1.0) * 255.0 + 0.5) as u8;
|
||||
}
|
||||
lut
|
||||
}
|
||||
|
||||
/// Apply the viewer's channel and exposure to premultiplied 8-bit RGBA pixels in place.
|
||||
/// RGB shows the image (exposure only); Red/Green/Blue/Alpha show one channel as an opaque
|
||||
/// grayscale image (or tinted in that channel's colour when `colorized`); RGB Straight shows the
|
||||
/// unpremultiplied colour, opaque.
|
||||
pub fn display_transform(px: &mut [[u8; 4]], channel: Channel, colorized: bool, stops: f32) {
|
||||
let lut = (stops != 0.0).then(|| exposure_lut(stops));
|
||||
let ex = |v: u8| lut.as_ref().map(|l| l[v as usize]).unwrap_or(v);
|
||||
for p in px.iter_mut() {
|
||||
let [r, g, b, a] = *p;
|
||||
*p = match channel {
|
||||
Channel::Rgb => {
|
||||
if lut.is_none() {
|
||||
continue;
|
||||
}
|
||||
// Exposure on straight colour, re-premultiplied.
|
||||
let un = |c: u8| if a == 0 { 0 } else { ((c as u32 * 255 + a as u32 / 2) / a as u32).min(255) as u8 };
|
||||
let pm = |c: u8| ((ex(un(c)) as u32 * a as u32 + 127) / 255) as u8;
|
||||
[pm(r), pm(g), pm(b), a]
|
||||
}
|
||||
Channel::Red | Channel::Green | Channel::Blue => {
|
||||
let (i, c) = match channel {
|
||||
Channel::Red => (0, r),
|
||||
Channel::Green => (1, g),
|
||||
_ => (2, b),
|
||||
};
|
||||
let v = ex(c);
|
||||
if colorized {
|
||||
let mut o = [0, 0, 0, 255];
|
||||
o[i] = v;
|
||||
o
|
||||
} else {
|
||||
[v, v, v, 255]
|
||||
}
|
||||
}
|
||||
Channel::Alpha => {
|
||||
let v = ex(a);
|
||||
[v, v, v, 255]
|
||||
}
|
||||
Channel::RgbStraight => {
|
||||
let un = |c: u8| if a == 0 { 0 } else { ((c as u32 * 255 + a as u32 / 2) / a as u32).min(255) as u8 };
|
||||
[ex(un(r)), ex(un(g)), ex(un(b)), 255]
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- display colour management
|
||||
|
||||
/// View ▸ Simulate Output: the output device a preview imitates.
|
||||
@@ -999,6 +945,12 @@ impl DisplayColor {
|
||||
)
|
||||
}
|
||||
|
||||
/// A conversion that is a baked 3D LUT alone, as a LUT-based My Custom RGB profile makes
|
||||
/// (tests of the display pass).
|
||||
pub fn from_lut(lut: Arc<effectcraft_color::icc::Lut3d>) -> DisplayColor {
|
||||
DisplayColor { sim: None, display: None, lut: Some(lut) }
|
||||
}
|
||||
|
||||
/// Whether the simulation runs a LUT-based ICC profile (baked into a 3D LUT).
|
||||
pub fn is_lut(&self) -> bool {
|
||||
self.lut.is_some()
|
||||
@@ -1097,6 +1049,9 @@ pub struct Snapshot {
|
||||
/// Render scale the image was taken at.
|
||||
pub scale: f64,
|
||||
pub image: Arc<Image>,
|
||||
/// How `image` is encoded: rendered without the HDR step in 32 bpc, so viewer exposure
|
||||
/// reaches its over-range values; the display pass applies the step.
|
||||
pub encoding: FrameEncoding,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -0,0 +1,404 @@
|
||||
//! The viewer's display pass as plain data: what turns a rendered frame into the pixels the
|
||||
//! Composition panel shows. The frontend runs it in a shader. The same steps on the CPU
|
||||
//! ([`DisplayPass::pixel`]) are its fallback without a GPU and the reference the shader is tested
|
||||
//! against, so both use the same `f32` operations in the same order.
|
||||
//!
|
||||
//! Steps, per pixel of a premultiplied frame encoded in the output space:
|
||||
//! 1. Unpremultiply (NaN alpha counts as 0, NaN colour as 0).
|
||||
//! 2. When exposure or an HDR step applies: decode with the frame's curve, multiply by 2^stops
|
||||
//! in linear light, run the pending HDR step (Project Settings ▸ HDR: Compand / Tone Map, for
|
||||
//! frames rendered without it), encode again.
|
||||
//! 3. Clamp to 0..1: the output's numbers.
|
||||
//! 4. The display colour conversion (Use Display Color Management, Simulate Output), on RGB and
|
||||
//! RGB Straight only.
|
||||
//! 5. Show Channel, premultiply (without step 2 or a display conversion: the frame's own
|
||||
//! premultiplied values, clamped to alpha), and round to 8 bits with `floor(v · 255 + 0.5)`.
|
||||
//!
|
||||
//! Exposure therefore acts on the full float range of 32 bpc frames, before tone mapping and
|
||||
//! before the display conversion. With neither exposure nor an HDR step, step 2 is skipped, so
|
||||
//! 8-bit frames pass through unchanged.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use effectcraft_color::icc::Lut3d;
|
||||
use effectcraft_color::space;
|
||||
use effectcraft_project::{HdrMode, ProjectSettings};
|
||||
use effectcraft_render::color::Pipe;
|
||||
|
||||
use super::{Channel, Conv, Curve, DisplayColor};
|
||||
|
||||
/// A transfer curve of the display pass.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub enum PassCurve {
|
||||
Linear,
|
||||
Srgb,
|
||||
/// A pure power curve (BT.1886 2.4, Mac 1.8, custom gammas).
|
||||
Gamma(f32),
|
||||
Pq,
|
||||
Hlg,
|
||||
}
|
||||
|
||||
/// SMPTE ST 2084 and BT.2100 HLG constants (`effectcraft_color::space`), in `f32`.
|
||||
const PQ_M1: f32 = 2610.0 / 16384.0;
|
||||
const PQ_M2: f32 = 2523.0 / 4096.0 * 128.0;
|
||||
const PQ_C1: f32 = 3424.0 / 4096.0;
|
||||
const PQ_C2: f32 = 2413.0 / 4096.0 * 32.0;
|
||||
const PQ_C3: f32 = 2392.0 / 4096.0 * 32.0;
|
||||
/// Linear 1.0 as a share of PQ's 10 000 cd/m² (the 203 cd/m² reference white).
|
||||
const PQ_WHITE: f32 = 203.0 / 10_000.0;
|
||||
const HLG_A: f32 = 0.178_832_77;
|
||||
const HLG_B: f32 = 0.284_668_92;
|
||||
const HLG_C: f32 = 0.559_910_7;
|
||||
const HLG_REFERENCE: f32 = 0.264_962_56;
|
||||
|
||||
/// The exponents the shader is handed (`vals2.yzw`): 1 / 2.4, 1 / PQ m2, 1 / PQ m1.
|
||||
pub const SHADER_EXPONENTS: [f32; 3] = [1.0 / 2.4, 1.0 / PQ_M2, 1.0 / PQ_M1];
|
||||
|
||||
/// `a^g` for `a ≥ 0`, 0 at 0 (WGSL leaves `pow(0, g)` to the implementation).
|
||||
fn power(a: f32, g: f32) -> f32 {
|
||||
if a <= 0.0 { 0.0 } else { a.powf(g) }
|
||||
}
|
||||
|
||||
impl PassCurve {
|
||||
/// The curve a colour space encodes with.
|
||||
pub fn of(curve: space::Curve) -> PassCurve {
|
||||
match curve {
|
||||
space::Curve::Linear => PassCurve::Linear,
|
||||
space::Curve::Srgb => PassCurve::Srgb,
|
||||
space::Curve::Gamma24 => PassCurve::Gamma(2.4),
|
||||
space::Curve::Pq => PassCurve::Pq,
|
||||
space::Curve::Hlg => PassCurve::Hlg,
|
||||
}
|
||||
}
|
||||
|
||||
fn of_viewer(c: Curve) -> PassCurve {
|
||||
match c {
|
||||
Curve::Srgb => PassCurve::Srgb,
|
||||
Curve::Gamma(g) => PassCurve::Gamma(g),
|
||||
Curve::Linear => PassCurve::Linear,
|
||||
Curve::Space(c) => PassCurve::of(c),
|
||||
}
|
||||
}
|
||||
|
||||
/// The shader's curve id.
|
||||
pub fn id(self) -> u32 {
|
||||
match self {
|
||||
PassCurve::Linear => 0,
|
||||
PassCurve::Srgb => 1,
|
||||
PassCurve::Gamma(_) => 2,
|
||||
PassCurve::Pq => 3,
|
||||
PassCurve::Hlg => 4,
|
||||
}
|
||||
}
|
||||
|
||||
/// The shader's parameter to decode with: the gamma (1 for other curves).
|
||||
pub fn decode_param(self) -> f32 {
|
||||
if let PassCurve::Gamma(g) = self { g } else { 1.0 }
|
||||
}
|
||||
|
||||
/// The shader's parameter to encode with: 1 / the gamma, as [`PassCurve::encode`] divides.
|
||||
pub fn encode_param(self) -> f32 {
|
||||
if let PassCurve::Gamma(g) = self { 1.0 / g } else { 1.0 }
|
||||
}
|
||||
|
||||
/// Encoded → linear light, mirrored around zero.
|
||||
pub fn decode(self, v: f32) -> f32 {
|
||||
let a = v.abs();
|
||||
let l = match self {
|
||||
PassCurve::Linear => a,
|
||||
PassCurve::Srgb => {
|
||||
if a <= 0.04045 {
|
||||
a / 12.92
|
||||
} else {
|
||||
power((a + 0.055) / 1.055, 2.4)
|
||||
}
|
||||
}
|
||||
PassCurve::Gamma(g) => power(a, g),
|
||||
PassCurve::Pq => {
|
||||
let p = power(a.min(1.0), 1.0 / PQ_M2);
|
||||
power((p - PQ_C1).max(0.0) / (PQ_C2 - PQ_C3 * p), 1.0 / PQ_M1) / PQ_WHITE
|
||||
}
|
||||
PassCurve::Hlg => {
|
||||
let e = if a <= 0.5 { a * a / 3.0 } else { (((a - HLG_C) / HLG_A).exp() + HLG_B) / 12.0 };
|
||||
e / HLG_REFERENCE
|
||||
}
|
||||
};
|
||||
l.copysign(v)
|
||||
}
|
||||
|
||||
/// Linear light → encoded, mirrored around zero.
|
||||
pub fn encode(self, v: f32) -> f32 {
|
||||
let a = v.abs();
|
||||
let e = match self {
|
||||
PassCurve::Linear => a,
|
||||
PassCurve::Srgb => {
|
||||
if a <= 0.003_130_8 {
|
||||
a * 12.92
|
||||
} else {
|
||||
1.055 * power(a, 1.0 / 2.4) - 0.055
|
||||
}
|
||||
}
|
||||
PassCurve::Gamma(g) => power(a, 1.0 / g),
|
||||
PassCurve::Pq => {
|
||||
let y = power(a * PQ_WHITE, PQ_M1);
|
||||
power((PQ_C1 + PQ_C2 * y) / (1.0 + PQ_C3 * y), PQ_M2)
|
||||
}
|
||||
PassCurve::Hlg => {
|
||||
let e = a * HLG_REFERENCE;
|
||||
if e <= 1.0 / 12.0 { (3.0 * e).sqrt() } else { HLG_A * (12.0 * e - HLG_B).ln() + HLG_C }
|
||||
}
|
||||
};
|
||||
e.copysign(v)
|
||||
}
|
||||
}
|
||||
|
||||
/// A conversion of the display colour: decode, an optional linear-light matrix, encode.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct PassConv {
|
||||
pub from: PassCurve,
|
||||
/// Rows of the linear RGB matrix (`None`: the same primaries).
|
||||
pub matrix: Option<[[f32; 3]; 3]>,
|
||||
pub to: PassCurve,
|
||||
}
|
||||
|
||||
impl PassConv {
|
||||
fn of(c: &Conv) -> PassConv {
|
||||
PassConv { from: PassCurve::of_viewer(c.from), matrix: c.m, to: PassCurve::of_viewer(c.to) }
|
||||
}
|
||||
|
||||
pub fn apply(&self, c: [f32; 3]) -> [f32; 3] {
|
||||
let l = c.map(|v| self.from.decode(v));
|
||||
let l = match &self.matrix {
|
||||
Some(m) => [0, 1, 2].map(|i| m[i][0] * l[0] + m[i][1] * l[1] + m[i][2] * l[2]),
|
||||
None => l,
|
||||
};
|
||||
l.map(|v| self.to.encode(v))
|
||||
}
|
||||
}
|
||||
|
||||
/// The display colour conversion ([`DisplayColor`]) as data for the display pass.
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum PassColor {
|
||||
/// One conversion; with Simulate Output, the conversion to the simulated device, its 8-bit
|
||||
/// numbers (`quantize`) and the conversion from the device to the display.
|
||||
Convs { first: PassConv, quantize: bool, second: Option<PassConv> },
|
||||
/// A LUT-based My Custom RGB profile: the whole simulation baked into a 3D LUT.
|
||||
Lut(Arc<Lut3d>),
|
||||
}
|
||||
|
||||
impl PassColor {
|
||||
/// One straight colour (0..1) to the display's numbers (0..1).
|
||||
pub fn apply(&self, c: [f32; 3]) -> [f32; 3] {
|
||||
let o = match self {
|
||||
PassColor::Lut(l) => lut_eval(l, c),
|
||||
PassColor::Convs { first, quantize, second } => {
|
||||
let mut o = first.apply(c);
|
||||
if *quantize {
|
||||
o = o.map(|v| (v.clamp(0.0, 1.0) * 255.0 + 0.5).floor() / 255.0);
|
||||
}
|
||||
match second {
|
||||
Some(s) => s.apply(o),
|
||||
None => o,
|
||||
}
|
||||
}
|
||||
};
|
||||
o.map(|v| v.clamp(0.0, 1.0))
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Lut3d::eval`], written as the shader samples the LUT (the same products and sums).
|
||||
pub fn lut_eval(l: &Lut3d, c: [f32; 3]) -> [f32; 3] {
|
||||
let n = l.size.max(2);
|
||||
let mut k = [0usize; 3];
|
||||
let mut f = [0f32; 3];
|
||||
for i in 0..3 {
|
||||
let x = c[i].clamp(0.0, 1.0) * (n - 1) as f32;
|
||||
k[i] = (x.floor() as usize).min(n - 2);
|
||||
f[i] = x - k[i] as f32;
|
||||
}
|
||||
let at = |r: usize, g: usize, b: usize| l.data.get((r * n + g) * n + b).copied().unwrap_or_default();
|
||||
let mut out = [0f32; 3];
|
||||
for corner in 0..8 {
|
||||
let (dr, dg, db) = (corner >> 2 & 1, corner >> 1 & 1, corner & 1);
|
||||
let w = (if dr == 1 { f[0] } else { 1.0 - f[0] }) * (if dg == 1 { f[1] } else { 1.0 - f[1] }) * (if db == 1 { f[2] } else { 1.0 - f[2] });
|
||||
let v = at(k[0] + dr, k[1] + dg, k[2] + db);
|
||||
for i in 0..3 {
|
||||
out[i] += w * v[i];
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
impl DisplayColor {
|
||||
/// The conversion as data for the display pass.
|
||||
pub fn pass(&self) -> PassColor {
|
||||
if let Some(l) = &self.lut {
|
||||
return PassColor::Lut(l.clone());
|
||||
}
|
||||
match (&self.sim, &self.display) {
|
||||
(Some((to, show)), _) => PassColor::Convs { first: PassConv::of(to), quantize: true, second: show.as_ref().map(PassConv::of) },
|
||||
(None, Some(d)) => PassColor::Convs { first: PassConv::of(d), quantize: false, second: None },
|
||||
(None, None) => {
|
||||
PassColor::Convs { first: PassConv { from: PassCurve::Linear, matrix: None, to: PassCurve::Linear }, quantize: false, second: None }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// How a rendered frame's pixels are encoded: the curve of its output space, and the HDR step
|
||||
/// (Project Settings ▸ HDR) it was rendered without and the display pass still runs (`Clip`:
|
||||
/// none). Kept with the frame, so a later settings change doesn't reinterpret older frames.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct FrameEncoding {
|
||||
pub curve: PassCurve,
|
||||
pub hdr: HdrMode,
|
||||
}
|
||||
|
||||
impl FrameEncoding {
|
||||
/// A frame of a project with `settings`; `skipped_hdr_step`: rendered with
|
||||
/// `RenderOpts::skip_hdr_step`.
|
||||
pub fn of(settings: &ProjectSettings, skipped_hdr_step: bool) -> FrameEncoding {
|
||||
let pipe = Pipe::of(settings);
|
||||
let curve = pipe.output_linear().decode.map_or(PassCurve::Linear, |s| PassCurve::of(s.curve()));
|
||||
let hdr = if skipped_hdr_step && pipe.output_hdr().is_some() { settings.hdr } else { HdrMode::Clip };
|
||||
FrameEncoding { curve, hdr }
|
||||
}
|
||||
|
||||
/// The frame as it would have been rendered with the HDR step: one straight colour.
|
||||
pub fn finish(&self, c: [f32; 3]) -> [f32; 3] {
|
||||
if self.hdr == HdrMode::Clip {
|
||||
return c;
|
||||
}
|
||||
let l = hdr_step(self.hdr, c.map(|v| self.curve.decode(v)));
|
||||
l.map(|v| self.curve.encode(v))
|
||||
}
|
||||
}
|
||||
|
||||
/// Project Settings ▸ HDR on linear light (`effectcraft_render::color::output_hdr`).
|
||||
pub fn hdr_step(mode: HdrMode, l: [f32; 3]) -> [f32; 3] {
|
||||
match mode {
|
||||
HdrMode::Clip => l,
|
||||
HdrMode::Compand => l.map(effectcraft_render::color::compand),
|
||||
HdrMode::ToneMap => effectcraft_render::color::tone_map(l),
|
||||
}
|
||||
}
|
||||
|
||||
/// The whole display pass of one viewer: Show Channel, exposure, the frame's encoding and the
|
||||
/// display colour conversion.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct DisplayPass {
|
||||
pub channel: Channel,
|
||||
pub colorized: bool,
|
||||
/// 2^stops of the viewer's exposure.
|
||||
pub gain: f32,
|
||||
pub encoding: FrameEncoding,
|
||||
pub color: Option<PassColor>,
|
||||
}
|
||||
|
||||
impl DisplayPass {
|
||||
/// `stops` of exposure on frames encoded as `encoding`.
|
||||
pub fn new(channel: Channel, colorized: bool, stops: f32, encoding: FrameEncoding, color: Option<&DisplayColor>) -> DisplayPass {
|
||||
let gain = if stops == 0.0 || !stops.is_finite() { 1.0 } else { 2f32.powf(stops) };
|
||||
// The display conversion applies to the colour views only (as before).
|
||||
let color = color.filter(|_| matches!(channel, Channel::Rgb | Channel::RgbStraight)).map(DisplayColor::pass);
|
||||
DisplayPass { channel, colorized, gain, encoding, color }
|
||||
}
|
||||
|
||||
/// Step 2 of the module docs runs.
|
||||
pub fn changes_light(&self) -> bool {
|
||||
self.gain != 1.0 || self.encoding.hdr != HdrMode::Clip
|
||||
}
|
||||
|
||||
/// Nothing changes the frame's pixels (beyond clamping and rounding).
|
||||
pub fn is_identity(&self) -> bool {
|
||||
self.channel == Channel::Rgb && !self.changes_light() && self.color.is_none()
|
||||
}
|
||||
|
||||
/// Step 2: exposure and the pending HDR step on straight colour.
|
||||
fn light(&self, c: [f32; 3]) -> [f32; 3] {
|
||||
if !self.changes_light() {
|
||||
return c;
|
||||
}
|
||||
let curve = self.encoding.curve;
|
||||
let l = hdr_step(self.encoding.hdr, c.map(|v| curve.decode(v) * self.gain));
|
||||
l.map(|v| curve.encode(v))
|
||||
}
|
||||
|
||||
/// One premultiplied pixel (encoded in the output space) → premultiplied RGBA8 on screen.
|
||||
pub fn pixel(&self, p: [f32; 4]) -> [u8; 4] {
|
||||
let q = |v: f32| (v.clamp(0.0, 1.0) * 255.0 + 0.5).floor() as u8;
|
||||
let a = if p[3].is_nan() { 0.0 } else { p[3].clamp(0.0, 1.0) };
|
||||
let un = |v: f32| if a > 0.0 && !v.is_nan() { v / a } else { 0.0 };
|
||||
let c = self.light([un(p[0]), un(p[1]), un(p[2])]).map(|v| v.clamp(0.0, 1.0));
|
||||
// A premultiplied channel: without exposure or HDR step straight from the frame (clamped
|
||||
// to alpha, as `display_pixel`), so nothing changes it by a division's rounding.
|
||||
let premul = |i: usize| if self.changes_light() || p[i].is_nan() { c[i] * a } else { p[i].clamp(0.0, a) };
|
||||
match self.channel {
|
||||
Channel::Rgb | Channel::RgbStraight => {
|
||||
let c = match &self.color {
|
||||
Some(dc) => dc.apply(c),
|
||||
None => c,
|
||||
};
|
||||
if self.channel == Channel::RgbStraight {
|
||||
[q(c[0]), q(c[1]), q(c[2]), 255]
|
||||
} else if self.color.is_none() {
|
||||
[q(premul(0)), q(premul(1)), q(premul(2)), q(a)]
|
||||
} else {
|
||||
[q(c[0] * a), q(c[1] * a), q(c[2] * a), q(a)]
|
||||
}
|
||||
}
|
||||
Channel::Red | Channel::Green | Channel::Blue => {
|
||||
let i = match self.channel {
|
||||
Channel::Red => 0,
|
||||
Channel::Green => 1,
|
||||
_ => 2,
|
||||
};
|
||||
let v = q(premul(i));
|
||||
if self.colorized {
|
||||
let mut o = [0, 0, 0, 255];
|
||||
o[i] = v;
|
||||
o
|
||||
} else {
|
||||
[v, v, v, 255]
|
||||
}
|
||||
}
|
||||
Channel::Alpha => {
|
||||
// Exposure lightens or darkens the alpha view as an encoded grey.
|
||||
let v = if self.gain != 1.0 { self.encoding.curve.encode(self.encoding.curve.decode(a) * self.gain) } else { a };
|
||||
let v = q(v);
|
||||
[v, v, v, 255]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The pass on premultiplied 8-bit pixels in place (the CPU path for 8-bit frames). Without
|
||||
/// a display conversion or HDR step each channel of an opaque pixel goes through the same
|
||||
/// curve on its own, so those pixels (and the Alpha view, which reads alpha only) look the
|
||||
/// result up in a table of [`DisplayPass::pixel`] (the browser has no threads to share
|
||||
/// the per-pixel work).
|
||||
pub fn apply_rgba8(&self, px: &mut [[u8; 4]]) {
|
||||
use rayon::prelude::*;
|
||||
if self.is_identity() {
|
||||
return;
|
||||
}
|
||||
let f = |v: u8| v as f32 / 255.0;
|
||||
if self.channel == Channel::Alpha {
|
||||
let t: [[u8; 4]; 256] = std::array::from_fn(|a| self.pixel([0.0, 0.0, 0.0, f(a as u8)]));
|
||||
px.par_iter_mut().for_each(|p| *p = t[p[3] as usize]);
|
||||
return;
|
||||
}
|
||||
let table = (self.color.is_none() && self.encoding.hdr == HdrMode::Clip)
|
||||
.then(|| -> [[u8; 4]; 256] { std::array::from_fn(|v| self.pixel([f(v as u8), f(v as u8), f(v as u8), 1.0])) });
|
||||
px.par_iter_mut().for_each(|p| {
|
||||
*p = match &table {
|
||||
Some(t) if p[3] == 255 => match self.channel {
|
||||
Channel::Rgb | Channel::RgbStraight => [t[p[0] as usize][0], t[p[1] as usize][1], t[p[2] as usize][2], 255],
|
||||
Channel::Red => t[p[0] as usize],
|
||||
Channel::Green => t[p[1] as usize],
|
||||
_ => t[p[2] as usize],
|
||||
},
|
||||
_ => self.pixel(p.map(f)),
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
+164
-15
@@ -6,7 +6,9 @@
|
||||
//! files' size and modification time included) plus frame, scale and view; layer buffers by
|
||||
//! the layer cache's content key salted with the project's footage fingerprint. A changed
|
||||
//! project therefore never reads stale pixels: its keys simply change.
|
||||
//! * Payloads are LZ4-compressed (`lz4_flex`, MIT): frames as 8-bit premultiplied RGBA, layer
|
||||
//! * Payloads are LZ4-compressed (`lz4_flex`, MIT): frames as 8-bit premultiplied RGBA (or, for
|
||||
//! 32 bpc viewer frames, half-float RGBA with the bytes shuffled into planes; the entry
|
||||
//! header's format byte tells them apart, 8-bit entries keep format 0), layer
|
||||
//! buffers as `f32` RGBA with the bytes shuffled into planes (exact, so cached and fresh
|
||||
//! renders are bit-identical), Roto Brush / Object Matte segmentations as
|
||||
//! `FrameSeg::to_bytes` (keyed by their content-keyed chain key and scale, so they are
|
||||
@@ -76,6 +78,19 @@ pub struct Frame8 {
|
||||
pub rgba: Vec<u8>,
|
||||
}
|
||||
|
||||
/// A cached 32 bpc viewer frame: premultiplied RGBA half floats (IEEE binary16 bits), four
|
||||
/// per pixel.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct FrameF16 {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub rgba: Vec<u16>,
|
||||
}
|
||||
|
||||
/// The entry header's format byte of frame entries.
|
||||
const FORMAT_RGBA8: u8 = 0;
|
||||
const FORMAT_F16: u8 = 1;
|
||||
|
||||
/// Counters for tests, the Info panel and `cache.diskStats`.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
|
||||
pub struct DiskStats {
|
||||
@@ -226,7 +241,8 @@ impl LayerPrefetch {
|
||||
|
||||
#[cfg_attr(target_arch = "wasm32", allow(dead_code))]
|
||||
enum Job {
|
||||
Write(Kind, u128, Vec<u8>),
|
||||
/// Kind, key, header format byte, payload.
|
||||
Write(Kind, u128, u8, Vec<u8>),
|
||||
}
|
||||
|
||||
struct Queue {
|
||||
@@ -245,7 +261,7 @@ impl Queue {
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
fn pop(&mut self) -> Option<Job> {
|
||||
if let Some(j) = self.frames.pop_front() {
|
||||
let Job::Write(_, _, p) = &j;
|
||||
let Job::Write(_, _, _, p) = &j;
|
||||
self.frame_bytes = self.frame_bytes.saturating_sub(p.len());
|
||||
return Some(j);
|
||||
}
|
||||
@@ -520,16 +536,21 @@ impl DiskCache {
|
||||
}
|
||||
|
||||
fn read(&self, kind: Kind, key: u128) -> Option<Vec<u8>> {
|
||||
self.read_entry(kind, key).map(|(_, p)| p)
|
||||
}
|
||||
|
||||
/// An entry's payload and its header's format byte.
|
||||
fn read_entry(&self, kind: Kind, key: u128) -> Option<(u8, Vec<u8>)> {
|
||||
if !self.contains(kind, key) {
|
||||
self.misses.fetch_add(1, Ordering::Relaxed);
|
||||
return None;
|
||||
}
|
||||
let path = self.path(kind, key);
|
||||
let data = std::fs::read(&path).ok();
|
||||
let ok = data.as_deref().and_then(|d| verify(d, kind));
|
||||
let ok = data.as_deref().and_then(|d| verify(d, kind).map(|p| (d[5], p)));
|
||||
match ok {
|
||||
Some(payload) => {
|
||||
let payload = payload.to_vec();
|
||||
Some((format, payload)) => {
|
||||
let payload = (format, payload.to_vec());
|
||||
self.hits.fetch_add(1, Ordering::Relaxed);
|
||||
if let Ok(mut g) = self.index.lock() {
|
||||
g.touch(kind, key);
|
||||
@@ -552,6 +573,12 @@ impl DiskCache {
|
||||
}
|
||||
}
|
||||
|
||||
/// A valid file whose payload does not decode: delete it, as a damaged one.
|
||||
fn discard(&self, kind: Kind, key: u128) {
|
||||
let _ = std::fs::remove_file(self.path(kind, key));
|
||||
self.forget(kind, key);
|
||||
}
|
||||
|
||||
fn forget(&self, kind: Kind, key: u128) {
|
||||
if let Ok(mut g) = self.index.lock() {
|
||||
g.remove(kind, key);
|
||||
@@ -560,9 +587,14 @@ impl DiskCache {
|
||||
|
||||
/// Queue a write; `false` when it was dropped (a full queue).
|
||||
fn enqueue(&self, kind: Kind, key: u128, payload: Vec<u8>) -> bool {
|
||||
self.enqueue_format(kind, key, FORMAT_RGBA8, payload)
|
||||
}
|
||||
|
||||
/// [`DiskCache::enqueue`] with the entry's header format byte.
|
||||
fn enqueue_format(&self, kind: Kind, key: u128, format: u8, payload: Vec<u8>) -> bool {
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
{
|
||||
self.write_now(kind, key, &payload);
|
||||
self.write_format(kind, key, format, &payload);
|
||||
true
|
||||
}
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
@@ -580,9 +612,9 @@ impl DiskCache {
|
||||
if kind == Kind::Frame {
|
||||
q.pending_frames.insert(key);
|
||||
q.frame_bytes = q.frame_bytes.saturating_add(payload.len());
|
||||
q.frames.push_back(Job::Write(kind, key, payload));
|
||||
q.frames.push_back(Job::Write(kind, key, format, payload));
|
||||
} else {
|
||||
q.jobs.push_back(Job::Write(kind, key, payload));
|
||||
q.jobs.push_back(Job::Write(kind, key, format, payload));
|
||||
}
|
||||
cv.notify_all();
|
||||
true
|
||||
@@ -591,6 +623,10 @@ impl DiskCache {
|
||||
|
||||
/// Write an entry now (atomic rename), then evict down to the limit.
|
||||
fn write_now(&self, kind: Kind, key: u128, payload: &[u8]) {
|
||||
self.write_format(kind, key, FORMAT_RGBA8, payload);
|
||||
}
|
||||
|
||||
fn write_format(&self, kind: Kind, key: u128, format: u8, payload: &[u8]) {
|
||||
if self.contains(kind, key) {
|
||||
return;
|
||||
}
|
||||
@@ -600,7 +636,7 @@ impl DiskCache {
|
||||
self.write_failed(&e);
|
||||
return;
|
||||
}
|
||||
let data = seal(kind, payload);
|
||||
let data = seal_format(kind, format, payload);
|
||||
// An entry that cannot fit would be evicted straight away.
|
||||
if data.len() as u64 > self.limit_bytes() {
|
||||
return;
|
||||
@@ -646,11 +682,43 @@ impl DiskCache {
|
||||
|
||||
// ------------------------------------------------------------ typed entries
|
||||
|
||||
/// An 8-bit frame (`None` for a half-float entry of that key, which [`DiskCache::get_frame_f16`]
|
||||
/// reads).
|
||||
pub fn get_frame(&self, key: u128) -> Option<Frame8> {
|
||||
let p = self.read(Kind::Frame, key)?;
|
||||
let (format, p) = self.read_entry(Kind::Frame, key)?;
|
||||
if format != FORMAT_RGBA8 {
|
||||
return None;
|
||||
}
|
||||
decode_frame(&p)
|
||||
}
|
||||
|
||||
/// A half-float frame (32 bpc viewer frames). A damaged one is deleted.
|
||||
pub fn get_frame_f16(&self, key: u128) -> Option<FrameF16> {
|
||||
let (format, p) = self.read_entry(Kind::Frame, key)?;
|
||||
if format != FORMAT_F16 {
|
||||
return None;
|
||||
}
|
||||
let f = decode_frame_f16(&p);
|
||||
if f.is_none() {
|
||||
self.discard(Kind::Frame, key);
|
||||
}
|
||||
f
|
||||
}
|
||||
|
||||
/// Queue a half-float frame for writing (`width * height * 4` binary16 values). `true` when
|
||||
/// it is on disk or queued; `false` when refused (wrong size, a full queue).
|
||||
pub fn put_frame_f16(&self, key: u128, width: u32, height: u32, rgba: &[u16]) -> bool {
|
||||
if rgba.len() as u64 != width as u64 * height as u64 * 4 {
|
||||
return false;
|
||||
}
|
||||
self.contains(Kind::Frame, key) || self.enqueue_format(Kind::Frame, key, FORMAT_F16, encode_frame_f16(width, height, rgba))
|
||||
}
|
||||
|
||||
/// Write a half-float frame synchronously (tests).
|
||||
pub fn put_frame_f16_now(&self, key: u128, width: u32, height: u32, rgba: &[u16]) {
|
||||
self.write_format(Kind::Frame, key, FORMAT_F16, &encode_frame_f16(width, height, rgba));
|
||||
}
|
||||
|
||||
/// Queue a frame for writing (8-bit premultiplied RGBA, `width * height * 4` bytes). `true`
|
||||
/// when it is on disk or queued; `false` when refused (wrong size, a full queue).
|
||||
pub fn put_frame(&self, key: u128, width: u32, height: u32, rgba: &[u8]) -> bool {
|
||||
@@ -762,8 +830,8 @@ fn writer(dc: std::sync::Weak<DiskCache>, q: Arc<(Mutex<Queue>, Condvar)>) {
|
||||
}
|
||||
};
|
||||
let Some(cache) = dc.upgrade() else { return };
|
||||
let Job::Write(kind, key, payload) = job;
|
||||
cache.write_now(kind, key, &payload);
|
||||
let Job::Write(kind, key, format, payload) = job;
|
||||
cache.write_format(kind, key, format, &payload);
|
||||
drop(cache);
|
||||
if let Ok(mut g) = m.lock() {
|
||||
g.busy = false;
|
||||
@@ -775,12 +843,17 @@ fn writer(dc: std::sync::Weak<DiskCache>, q: Arc<(Mutex<Queue>, Condvar)>) {
|
||||
}
|
||||
}
|
||||
|
||||
/// An entry file's bytes: header (magic, kind), payload, checksum.
|
||||
/// An entry file's bytes: header (magic, kind, format 0), payload, checksum.
|
||||
pub fn seal(kind: Kind, payload: &[u8]) -> Vec<u8> {
|
||||
seal_format(kind, FORMAT_RGBA8, payload)
|
||||
}
|
||||
|
||||
/// [`seal`] with the header's format byte (frames: 0 = 8-bit, 1 = half float).
|
||||
fn seal_format(kind: Kind, format: u8, payload: &[u8]) -> Vec<u8> {
|
||||
let mut data = Vec::with_capacity(payload.len() + 16);
|
||||
data.extend_from_slice(MAGIC);
|
||||
data.push(kind.tag());
|
||||
data.extend_from_slice(&[0, 0, 0]);
|
||||
data.extend_from_slice(&[format, 0, 0]);
|
||||
data.extend_from_slice(payload);
|
||||
let sum = checksum(&data);
|
||||
data.extend_from_slice(&sum.to_le_bytes());
|
||||
@@ -879,6 +952,46 @@ fn decode_frame(p: &[u8]) -> Option<Frame8> {
|
||||
(rgba.len() == w as usize * h as usize * 4).then_some(Frame8 { width: w, height: h, rgba })
|
||||
}
|
||||
|
||||
/// Half floats with their low and high bytes in planes (per channel), then LZ4: the high bytes
|
||||
/// (sign, exponent) repeat a lot, mixed with the low ones LZ4 finds little.
|
||||
fn encode_frame_f16(w: u32, h: u32, rgba: &[u16]) -> Vec<u8> {
|
||||
use rayon::prelude::*;
|
||||
let n = rgba.len() / 4;
|
||||
let mut planes = vec![0u8; n * 8];
|
||||
// Plane (byte k, channel c) at (k · 4 + c) · n.
|
||||
planes.par_chunks_mut(n.max(1)).enumerate().for_each(|(plane, out)| {
|
||||
let (k, c) = (plane / 4, plane % 4);
|
||||
for (o, px) in out.iter_mut().zip(rgba.as_chunks::<4>().0) {
|
||||
*o = px[c].to_le_bytes()[k];
|
||||
}
|
||||
});
|
||||
let mut out = Vec::with_capacity(planes.len() / 3 + 16);
|
||||
out.extend_from_slice(&w.to_le_bytes());
|
||||
out.extend_from_slice(&h.to_le_bytes());
|
||||
out.extend_from_slice(&lz4_flex::compress_prepend_size(&planes));
|
||||
out
|
||||
}
|
||||
|
||||
fn decode_frame_f16(p: &[u8]) -> Option<FrameF16> {
|
||||
if p.len() < 8 {
|
||||
return None;
|
||||
}
|
||||
let w = u32::from_le_bytes(p[0..4].try_into().ok()?);
|
||||
let h = u32::from_le_bytes(p[4..8].try_into().ok()?);
|
||||
let n = (w as usize).checked_mul(h as usize)?;
|
||||
let planes = lz4_flex::decompress_size_prepended(&p[8..]).ok()?;
|
||||
if planes.len() != n.checked_mul(8)? {
|
||||
return None;
|
||||
}
|
||||
let mut rgba = vec![0u16; n * 4];
|
||||
for (i, px) in rgba.as_chunks_mut::<4>().0.iter_mut().enumerate() {
|
||||
for (c, v) in px.iter_mut().enumerate() {
|
||||
*v = u16::from_le_bytes([planes[c * n + i], planes[(4 + c) * n + i]]);
|
||||
}
|
||||
}
|
||||
Some(FrameF16 { width: w, height: h, rgba })
|
||||
}
|
||||
|
||||
fn encode_layer(b: &Buf) -> Vec<u8> {
|
||||
let (w, h) = (b.img.width, b.img.height);
|
||||
let n = b.img.data.len();
|
||||
@@ -1175,6 +1288,42 @@ mod tests {
|
||||
assert_eq!(parse_entry_name(&format!("{key:032x}.tmp.1")), None);
|
||||
}
|
||||
|
||||
/// Half-float frames (32 bpc viewer frames) round-trip bit for bit, over-range values,
|
||||
/// negatives and odd sizes included; they and 8-bit frames never read as each other; a
|
||||
/// damaged half-float entry is a miss and is deleted; they count toward the size limit.
|
||||
#[test]
|
||||
fn half_float_frames_round_trip() {
|
||||
let dir = tmpdir("f16");
|
||||
let dc = DiskCache::open(&dir, 1 << 30).unwrap();
|
||||
let (w, h) = (7u32, 5u32);
|
||||
// 4.0, 16.0, -0.25, 65504 and a spread of bit patterns.
|
||||
let rgba: Vec<u16> = (0..w * h * 4).map(|i| [0x4400, 0x4c00, 0xb400, 0x7bff][i as usize % 4] ^ ((i * 37) as u16 & 0x03ff)).collect();
|
||||
dc.put_frame_f16_now(1, w, h, &rgba);
|
||||
assert_eq!(dc.get_frame_f16(1), Some(FrameF16 { width: w, height: h, rgba: rgba.clone() }));
|
||||
assert!(dc.get_frame(1).is_none(), "not an 8-bit frame");
|
||||
dc.put_frame_now(2, 2, 2, &[9; 16]);
|
||||
assert!(dc.get_frame_f16(2).is_none() && dc.contains(Kind::Frame, 2), "an 8-bit entry stays valid");
|
||||
assert_eq!(dc.get_frame(2).unwrap().rgba, vec![9; 16]);
|
||||
assert!(dc.stats().bytes > 0);
|
||||
// Reopened, still there (format byte persisted).
|
||||
drop(dc);
|
||||
let dc = DiskCache::open(&dir, 1 << 30).unwrap();
|
||||
assert_eq!(dc.get_frame_f16(1).unwrap().rgba, rgba);
|
||||
// A sealed entry with a payload of the wrong size: deleted on read.
|
||||
let p = dc.path(Kind::Frame, 3);
|
||||
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
|
||||
std::fs::write(&p, seal_format(Kind::Frame, FORMAT_F16, &encode_frame_f16(w, h + 1, &rgba[..])[..])).unwrap();
|
||||
dc.rescan();
|
||||
assert!(dc.contains(Kind::Frame, 3));
|
||||
assert!(dc.get_frame_f16(3).is_none());
|
||||
assert!(!dc.contains(Kind::Frame, 3) && !p.exists());
|
||||
// Wrong lengths are not written.
|
||||
dc.put_frame_f16(4, w, h, &rgba[1..]);
|
||||
dc.flush();
|
||||
assert!(!dc.contains(Kind::Frame, 4));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// Segmentations round-trip by (chain key, scale), survive a reopen (rescan), are bounded by
|
||||
/// the size limit like every entry, and a damaged file is a miss.
|
||||
#[test]
|
||||
|
||||
@@ -33,11 +33,16 @@ rayon = { workspace = true }
|
||||
web-time = { workspace = true }
|
||||
kurbo = { workspace = true }
|
||||
image = { workspace = true }
|
||||
# Half-float viewer frames of 32 bpc projects (F16C is picked at run time).
|
||||
half = { workspace = true }
|
||||
bytemuck = { workspace = true }
|
||||
|
||||
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
|
||||
# Floating panels open in OS windows of their own on desktop; the web keeps them in the page.
|
||||
craft-ui = { workspace = true, features = ["native-windows"] }
|
||||
craft-update = { workspace = true }
|
||||
# Waits for the display pass's error scopes (immediately ready on native wgpu).
|
||||
pollster = { workspace = true }
|
||||
sys-locale = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
@@ -50,6 +55,7 @@ effectcraft-segment = { workspace = true }
|
||||
effectcraft-host = { workspace = true }
|
||||
effectcraft-media = { workspace = true }
|
||||
exr = { workspace = true }
|
||||
tempfile = { workspace = true }
|
||||
|
||||
[build-dependencies]
|
||||
serde_json = { workspace = true }
|
||||
|
||||
@@ -79,6 +79,13 @@ impl EffectcraftApp {
|
||||
viewer_shown: None,
|
||||
crypto_probe_busy: Default::default(),
|
||||
viewer_image: None,
|
||||
viewer_f16: None,
|
||||
window_wgpu: None,
|
||||
display_gpu: None,
|
||||
display_gpu_failed: false,
|
||||
viewer_bake: Default::default(),
|
||||
snapshot_bake: Default::default(),
|
||||
snapshot_f16: None,
|
||||
viewer_readback: Default::default(),
|
||||
wgpu: None,
|
||||
gpu: None,
|
||||
|
||||
@@ -73,14 +73,30 @@ impl EffectcraftApp {
|
||||
renderer.free_texture(id);
|
||||
}
|
||||
for (_, texture) in self.passive_tex.values() {
|
||||
if let panels::viewers::PassiveTexture::Gpu(id, _) = texture {
|
||||
renderer.free_texture(id);
|
||||
match texture {
|
||||
panels::viewers::PassiveTexture::Gpu(id, _) => renderer.free_texture(id),
|
||||
panels::viewers::PassiveTexture::Baked(b) => {
|
||||
if let Some(id) = b.id() {
|
||||
renderer.free_texture(&id);
|
||||
}
|
||||
}
|
||||
panels::viewers::PassiveTexture::Cpu(_) => {}
|
||||
}
|
||||
}
|
||||
drop(renderer);
|
||||
}
|
||||
// The display pass's textures: half-float frames go through the CPU from now on.
|
||||
self.viewer_bake.free(self.window_wgpu.as_ref());
|
||||
self.snapshot_bake.free(self.window_wgpu.as_ref());
|
||||
self.display_gpu = None;
|
||||
self.window_wgpu = None;
|
||||
self.viewer_native = None;
|
||||
self.passive_tex.retain(|_, (_, texture)| matches!(texture, panels::viewers::PassiveTexture::Cpu(_)));
|
||||
self.viewer_shown = self.viewer_tex.as_ref().map(|(texture, key, _)| (texture.id(), *key));
|
||||
self.viewer_shown = match &self.viewer_f16 {
|
||||
// A half-float frame stays on screen through the CPU pass.
|
||||
Some(_) => self.viewer_shown.map(|(_, key)| (None, key)),
|
||||
None => self.viewer_tex.as_ref().map(|(texture, key, _)| (Some(texture.id()), *key)),
|
||||
};
|
||||
self.viewer_image = None;
|
||||
// Old completion closures retain only their detached slot, not the replacement.
|
||||
self.viewer_readback = Default::default();
|
||||
@@ -152,6 +168,9 @@ impl EffectcraftApp {
|
||||
return;
|
||||
}
|
||||
let Some(rs) = frame.wgpu_render_state() else { return };
|
||||
// The display pass needs only this device, not the compositor (which may take tens of
|
||||
// seconds to build, or fail).
|
||||
self.window_wgpu = Some(rs.clone());
|
||||
// Help ▸ System Compatibility Report names the adapter and backend the window got.
|
||||
let info = rs.adapter.get_info();
|
||||
self.session.window_adapter = Some(format!(
|
||||
|
||||
@@ -221,10 +221,7 @@ impl EffectcraftApp {
|
||||
self.playback.start_frame = cur;
|
||||
// `prefetch_playback` requests every uncached frame of offsets 0..window from here.
|
||||
let window = (ahead * 3).min(fit - 1);
|
||||
let need = self.frames.get(&FrameKey { frame: cur, ..series }).map(|img| {
|
||||
let [w, h] = img.size();
|
||||
startup_frames(ahead, window, self.frames.budget(), w.saturating_mul(h).saturating_mul(4))
|
||||
});
|
||||
let need = self.frames.get(&FrameKey { frame: cur, ..series }).map(|img| startup_frames(ahead, window, self.frames.budget(), img.bytes()));
|
||||
if need.is_some() && self.playback.first_cached.is_none() {
|
||||
self.playback.first_cached = Some(now);
|
||||
}
|
||||
@@ -448,10 +445,11 @@ impl EffectcraftApp {
|
||||
((self.frames.budget() as f64 * 0.9 / self.frame_bytes(comp, scale)) as i64).max(1)
|
||||
}
|
||||
|
||||
/// Bytes of one comp-sized viewer frame at `scale` (Extended Viewer frames are larger).
|
||||
/// Bytes of one comp-sized viewer frame at `scale` (Extended Viewer frames are larger; 32 bpc
|
||||
/// frames are half floats, twice the bytes of 8-bit ones).
|
||||
fn frame_bytes(&self, comp: &effectcraft_engine::project::Comp, scale: f64) -> f64 {
|
||||
let side = |n: u32| (n as f64 * scale).ceil().max(1.0);
|
||||
side(comp.width) * side(comp.height) * 4.0
|
||||
side(comp.width) * side(comp.height) * frames::frame_pixel_bytes(&self.session.project.settings) as f64
|
||||
}
|
||||
|
||||
fn frames_parallelism(&self) -> usize {
|
||||
|
||||
@@ -7,11 +7,18 @@ impl EffectcraftApp {
|
||||
}
|
||||
|
||||
/// The viewer's current pixels as 8-bit premultiplied RGBA, reading a GPU frame back the
|
||||
/// first time something asks for it.
|
||||
/// first time something asks for it. A half-float frame (32 bpc) is converted the first time:
|
||||
/// as rendered with Project Settings ▸ HDR, without the viewer's exposure or display colour.
|
||||
///
|
||||
/// The browser can't wait for the GPU: there the first call starts an asynchronous readback
|
||||
/// and returns `None`; a later call (the next frame) has the pixels.
|
||||
pub fn viewer_pixels(&mut self) -> Option<Arc<egui::ColorImage>> {
|
||||
if self.viewer_image.is_none()
|
||||
&& let Some(f) = &self.viewer_f16
|
||||
{
|
||||
let pass = effectcraft_engine::viewer::DisplayPass::new(effectcraft_engine::viewer::Channel::Rgb, false, 0.0, f.encoding, None);
|
||||
self.viewer_image = Some(Arc::new(f.to_color_image(&pass)));
|
||||
}
|
||||
let to_image = |bytes: Vec<u8>, w: u32, h: u32| {
|
||||
let px = bytes.as_chunks::<4>().0.iter().map(|c| egui::Color32::from_rgba_premultiplied(c[0], c[1], c[2], c[3])).collect();
|
||||
Arc::new(egui::ColorImage::new([w as usize, h as usize], px))
|
||||
@@ -44,6 +51,36 @@ impl EffectcraftApp {
|
||||
self.viewer_image.clone()
|
||||
}
|
||||
|
||||
/// The half-float frame on screen (32 bpc projects), for float readouts.
|
||||
pub fn viewer_f16(&self) -> Option<&Arc<frames::F16Frame>> {
|
||||
self.viewer_f16.as_ref()
|
||||
}
|
||||
|
||||
/// The texture the Composition viewer's display pass drew last (half-float frames, or 8-bit
|
||||
/// frames with exposure, a channel view or display colour), when it ran on the GPU.
|
||||
pub fn viewer_display_texture(&self) -> Option<egui::TextureId> {
|
||||
self.viewer_bake.id()
|
||||
}
|
||||
|
||||
/// The display pass's pipelines on egui-wgpu's device, made the first time (desktop; the
|
||||
/// browser's viewer converts on the CPU for now). `None` without a device.
|
||||
pub(crate) fn display_gpu(&mut self) -> Option<Arc<crate::display_gpu::DisplayGpu>> {
|
||||
if cfg!(target_arch = "wasm32") || self.display_gpu_failed {
|
||||
return None;
|
||||
}
|
||||
if self.display_gpu.is_none() {
|
||||
let rs = self.window_wgpu.as_ref()?;
|
||||
match crate::display_gpu::DisplayGpu::try_new(&rs.device) {
|
||||
Ok(g) => self.display_gpu = Some(Arc::new(g)),
|
||||
Err(e) => {
|
||||
log::warn!("the viewer's display pass runs on the CPU: {e}");
|
||||
self.display_gpu_failed = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
self.display_gpu.clone()
|
||||
}
|
||||
|
||||
/// The viewer's Resolution: the active composition's (saved with the project).
|
||||
pub fn viewer_res(&self) -> state::Resolution {
|
||||
self.session.active_comp().map(|c| c.resolution).unwrap_or_default()
|
||||
@@ -173,7 +210,9 @@ impl EffectcraftApp {
|
||||
backend: effectcraft_engine::render::Backend::Auto,
|
||||
nested_switches: self.session.prefs.general.switches_affect_nested_comps,
|
||||
draft_shadows: self.session.prefs.three_d.realtime_shadows,
|
||||
skip_hdr_step: false,
|
||||
// 32 bpc: half-float frames without the HDR step; the display pass applies it after
|
||||
// the viewer's exposure (#2797).
|
||||
skip_hdr_step: frames::f16_frames(&self.session.project.settings),
|
||||
proxy: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -710,7 +710,8 @@ mod tests {
|
||||
options.exposure = 1.0;
|
||||
let exposed = press_sampling_viewer(&mut harness).unwrap();
|
||||
let mut expected = [[32, 64, 96, 255]];
|
||||
effectcraft_engine::viewer::display_transform(&mut expected, effectcraft_engine::viewer::Channel::Rgb, false, 1.0);
|
||||
let encoding = effectcraft_engine::viewer::FrameEncoding::of(&harness.state().session.project.settings, false);
|
||||
effectcraft_engine::viewer::DisplayPass::new(effectcraft_engine::viewer::Channel::Rgb, false, 1.0, encoding, None).apply_rgba8(&mut expected);
|
||||
for (actual, expected) in exposed.iter().zip(expected[0]) {
|
||||
assert!((*actual - expected as f32 / 255.0).abs() < 0.01, "{exposed:?}");
|
||||
}
|
||||
@@ -736,6 +737,94 @@ mod tests {
|
||||
assert_eq!(press_sampling_viewer(&mut harness), None, "wireframe must not sample an old pixel frame");
|
||||
}
|
||||
|
||||
/// A 32 bpc project whose solid is about 4.0 in linear light (white +2 stops), in half-float viewer
|
||||
/// frames; `hdr`: Project Settings ▸ HDR (with an sRGB working space).
|
||||
fn hdr_viewer(hdr: &str) -> egui_kittest::Harness<'static, crate::EffectcraftApp> {
|
||||
let mut s = effectcraft_engine::Session::default();
|
||||
s.execute("comp.new", serde_json::json!({"width": 320, "height": 180, "duration": 2})).unwrap();
|
||||
s.execute("file.projectSettings", serde_json::json!({"bitDepth": 32, "workingSpace": "srgb", "hdr": hdr})).unwrap();
|
||||
let lid = s.execute("layer.newSolid", serde_json::json!({"color": "#ffffff"})).unwrap()["layer"].as_u64().unwrap();
|
||||
s.execute("effect.apply", serde_json::json!({"layers": [lid], "effect": "Exposure"})).unwrap();
|
||||
s.execute("prop.set", serde_json::json!({"layer": lid, "path": "effects/#1/master/exposure", "value": 2.0})).unwrap();
|
||||
let mut harness = egui_kittest::Harness::builder().with_size(vec2(1200.0, 800.0)).build_eframe(|_| crate::EffectcraftApp::new(s));
|
||||
for _ in 0..400 {
|
||||
harness.step();
|
||||
if harness.state().frames.inflight() == 0 && harness.state().viewer_f16.is_some() {
|
||||
harness.run_steps(3);
|
||||
return harness;
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
}
|
||||
panic!("the half-float viewer frame did not arrive");
|
||||
}
|
||||
|
||||
/// The linear light of the frame's first pixel (sRGB-encoded).
|
||||
fn linear_of(harness: &egui_kittest::Harness<'_, crate::EffectcraftApp>) -> f32 {
|
||||
let p = harness.state().viewer_f16.as_ref().unwrap().pixel(0).unwrap();
|
||||
effectcraft_engine::color::srgb_to_linear(p[0] / p[3])
|
||||
}
|
||||
|
||||
fn exposed(harness: &mut egui_kittest::Harness<'_, crate::EffectcraftApp>, stops: f32) -> f32 {
|
||||
harness.state_mut().session.state.viewer.exposure = stops;
|
||||
harness.run_steps(2);
|
||||
press_sampling_viewer(harness).unwrap()[0]
|
||||
}
|
||||
|
||||
/// #2797: in 32 bpc the viewer's exposure reaches values above 1.0. White +2 stops (about 4.0)
|
||||
/// is white at 0 EV and still white at -2 EV (the 8-bit viewer showed 0.25 there), and at -3 EV
|
||||
/// it is an eighth of its linear light.
|
||||
#[test]
|
||||
fn viewer_exposure_reaches_over_range_values_in_32_bpc() {
|
||||
let mut harness = hdr_viewer("clip");
|
||||
let l = linear_of(&harness);
|
||||
assert!(l > 3.9, "the frame keeps the over-range value: {l}");
|
||||
let srgb = |l: f32| effectcraft_engine::color::linear_to_srgb(l);
|
||||
assert!((exposed(&mut harness, 0.0) - 1.0).abs() < 0.01);
|
||||
assert!((exposed(&mut harness, -2.0) - 1.0).abs() < 0.01, "-2 EV");
|
||||
let v = exposed(&mut harness, -3.0);
|
||||
assert!((v - srgb(l / 8.0)).abs() < 0.01 && v < 0.8, "-3 EV: {v}");
|
||||
// The 8-bit image other panels read: as rendered (clipped), without the exposure.
|
||||
let img = harness.state_mut().viewer_pixels().unwrap();
|
||||
assert_eq!(img.pixels[0].to_array(), [255, 255, 255, 255]);
|
||||
// A 32 bpc snapshot keeps the range too.
|
||||
harness.state_mut().session.execute("view.takeSnapshot", serde_json::json!({})).unwrap();
|
||||
harness.state_mut().session.state.viewer.show_snapshot = true;
|
||||
let v = exposed(&mut harness, -3.0);
|
||||
assert!((v - srgb(l / 8.0)).abs() < 0.01, "snapshot at -3 EV: {v}");
|
||||
}
|
||||
|
||||
/// Compand runs after the exposure: at -3 EV the 4.0 white is about 0.5, under the knee, so it
|
||||
/// shows as that (companding first left 0.99, darkened to 0.12). At 0 EV it is companded as
|
||||
/// before.
|
||||
#[test]
|
||||
fn compand_follows_the_viewer_exposure() {
|
||||
let mut harness = hdr_viewer("compand");
|
||||
assert_eq!(harness.state().viewer_f16.as_ref().unwrap().encoding.hdr, effectcraft_engine::project::HdrMode::Compand);
|
||||
let srgb = |l: f32| effectcraft_engine::color::linear_to_srgb(l);
|
||||
let l = linear_of(&harness);
|
||||
let at0 = exposed(&mut harness, 0.0);
|
||||
let want = srgb(effectcraft_engine::render::color::compand(l));
|
||||
assert!((at0 - want).abs() < 0.01, "0 EV: {at0} vs {want}");
|
||||
let v = exposed(&mut harness, -3.0);
|
||||
assert!((v - srgb(l / 8.0)).abs() < 0.01 && v < 0.8, "-3 EV: {v}");
|
||||
// Other panels' 8-bit image is the companded frame, as the render was.
|
||||
let img = harness.state_mut().viewer_pixels().unwrap();
|
||||
assert!((img.pixels[0].r() as f32 / 255.0 - want).abs() < 0.01);
|
||||
// A passive viewer gets the half-float frame through the same pass (no channel or exposure
|
||||
// of its own; on the CPU without a device): the companded frame.
|
||||
let ctx = harness.ctx.clone();
|
||||
let (key, f) = (harness.state().viewer_shown.unwrap().1, harness.state().viewer_f16.clone().unwrap());
|
||||
crate::panels::viewers::set_texture(harness.state_mut(), &ctx, 7, key, crate::frames::FrameImage::CpuF16(f), None, 0);
|
||||
let Some((_, crate::panels::viewers::PassiveTexture::Cpu(tex))) = harness.state().passive_tex.get(&7) else { panic!("no passive texture") };
|
||||
assert_eq!(tex.size(), [320, 180]);
|
||||
let f = harness.state().viewer_f16.clone().unwrap();
|
||||
let crate::frames::FrameImage::Cpu(shown) = crate::panels::viewers::cpu_passive_image(harness.state(), crate::frames::FrameImage::CpuF16(f), None)
|
||||
else {
|
||||
panic!("an 8-bit image")
|
||||
};
|
||||
assert!((shown.pixels[0].r() as f32 / 255.0 - want).abs() < 0.01, "the passive viewer shows the companded frame: {:?}", shown.pixels[0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_new_modal_backdrop_cannot_deliver_a_pending_sample() {
|
||||
let mut harness = sampling_viewer();
|
||||
|
||||
@@ -0,0 +1,492 @@
|
||||
//! The viewer's display pass on egui-wgpu's device (#2797): a rendered frame (half floats for 32
|
||||
//! bpc projects, RGBA8 otherwise) goes through Show Channel, exposure, the pending HDR step and
|
||||
//! the display colour (`effectcraft_engine::viewer::DisplayPass`) into one premultiplied RGBA8
|
||||
//! texture with mips per viewer, which egui draws like the GPU compositor's frames (native
|
||||
//! texture, the Viewer Zoom Quality sampler).
|
||||
//!
|
||||
//! This is not the compositor's `Gpu` (which may be missing): it needs only egui-wgpu's device.
|
||||
//! Without one (no adapter, headless tests, the device retired) the same pass runs on the CPU
|
||||
//! (`DisplayPass::pixel`). The shader (`display_gpu.wgsl`) repeats the CPU steps operation by
|
||||
//! operation; the tests below hold it to them.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use effectcraft_engine::viewer::{Channel, DisplayPass, PassColor, PassConv, PassCurve};
|
||||
use eframe::egui_wgpu;
|
||||
use eframe::wgpu;
|
||||
|
||||
use crate::frames::F16Frame;
|
||||
|
||||
const UNIFORM_BYTES: usize = 12 * 16;
|
||||
|
||||
/// A frame for the display pass.
|
||||
#[derive(Clone)]
|
||||
pub enum Source {
|
||||
F16(Arc<F16Frame>),
|
||||
Rgba8(Arc<egui::ColorImage>),
|
||||
}
|
||||
|
||||
impl Source {
|
||||
fn same(&self, o: &Source) -> bool {
|
||||
match (self, o) {
|
||||
(Source::F16(a), Source::F16(b)) => Arc::ptr_eq(a, b),
|
||||
(Source::Rgba8(a), Source::Rgba8(b)) => Arc::ptr_eq(a, b),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn size(&self) -> [usize; 2] {
|
||||
match self {
|
||||
Source::F16(f) => f.size(),
|
||||
Source::Rgba8(c) => c.size,
|
||||
}
|
||||
}
|
||||
|
||||
/// A half-float frame past the texture limit goes through the CPU pass: the display
|
||||
/// conversion (exposure, HDR) comes before the averaging down, as with the 8-bit mips, and
|
||||
/// the GPU would average the scene-linear values first.
|
||||
pub fn needs_cpu_pass(&self, max_side: usize) -> bool {
|
||||
matches!(self, Source::F16(_)) && crate::frames::fit_factor(self.size(), max_side, 1) > 1
|
||||
}
|
||||
}
|
||||
|
||||
/// The pass's uniform block (`Params` in the shader).
|
||||
pub fn uniform(pass: &DisplayPass) -> [u8; UNIFORM_BYTES] {
|
||||
let channel = match pass.channel {
|
||||
Channel::Rgb => 0u32,
|
||||
Channel::Red => 1,
|
||||
Channel::Green => 2,
|
||||
Channel::Blue => 3,
|
||||
Channel::Alpha => 4,
|
||||
Channel::RgbStraight => 5,
|
||||
};
|
||||
let hdr = match pass.encoding.hdr {
|
||||
effectcraft_engine::project::HdrMode::Clip => 0u32,
|
||||
effectcraft_engine::project::HdrMode::Compand => 1,
|
||||
effectcraft_engine::project::HdrMode::ToneMap => 2,
|
||||
};
|
||||
let curve = pass.encoding.curve;
|
||||
let none = PassConv { from: PassCurve::Linear, matrix: None, to: PassCurve::Linear };
|
||||
let (kind, quantize, first, second, lut_n) = match &pass.color {
|
||||
None => (0u32, false, none, None, 2u32),
|
||||
Some(PassColor::Convs { first, quantize, second }) => (1, *quantize, *first, *second, 2),
|
||||
Some(PassColor::Lut(l)) => (2, false, none, None, l.size.max(2) as u32),
|
||||
};
|
||||
let s = second.unwrap_or(none);
|
||||
let mat = |m: Option<[[f32; 3]; 3]>| {
|
||||
let m = m.unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]);
|
||||
m.map(|r| [r[0], r[1], r[2], 0.0])
|
||||
};
|
||||
let [x24, xm2, xm1] = effectcraft_engine::viewer::SHADER_EXPONENTS;
|
||||
let u = |v: u32| v.to_le_bytes();
|
||||
let f = |v: f32| v.to_le_bytes();
|
||||
let mut b = Vec::with_capacity(UNIFORM_BYTES);
|
||||
for v in [
|
||||
channel,
|
||||
pass.colorized as u32,
|
||||
curve.id(),
|
||||
hdr,
|
||||
kind,
|
||||
quantize as u32,
|
||||
second.is_some() as u32,
|
||||
lut_n,
|
||||
first.from.id(),
|
||||
first.to.id(),
|
||||
s.from.id(),
|
||||
s.to.id(),
|
||||
] {
|
||||
b.extend(u(v));
|
||||
}
|
||||
for v in [
|
||||
pass.gain,
|
||||
curve.decode_param(),
|
||||
first.matrix.is_some() as u32 as f32,
|
||||
s.matrix.is_some() as u32 as f32,
|
||||
first.from.decode_param(),
|
||||
first.to.encode_param(),
|
||||
s.from.decode_param(),
|
||||
s.to.encode_param(),
|
||||
curve.encode_param(),
|
||||
x24,
|
||||
xm2,
|
||||
xm1,
|
||||
] {
|
||||
b.extend(f(v));
|
||||
}
|
||||
for row in mat(first.matrix).into_iter().chain(mat(s.matrix)) {
|
||||
for v in row {
|
||||
b.extend(f(v));
|
||||
}
|
||||
}
|
||||
let mut out = [0u8; UNIFORM_BYTES];
|
||||
out.copy_from_slice(&b);
|
||||
out
|
||||
}
|
||||
|
||||
/// Mip levels of a full chain for a `w` × `h` texture.
|
||||
fn mip_levels(w: u32, h: u32) -> u32 {
|
||||
32 - w.max(h).max(1).leading_zeros()
|
||||
}
|
||||
|
||||
/// The display pass's pipelines on one device (made once, kept with the app).
|
||||
pub struct DisplayGpu {
|
||||
bake: wgpu::RenderPipeline,
|
||||
mip: wgpu::RenderPipeline,
|
||||
bake_layout: wgpu::BindGroupLayout,
|
||||
mip_layout: wgpu::BindGroupLayout,
|
||||
no_lut: wgpu::TextureView,
|
||||
max_side: u32,
|
||||
}
|
||||
|
||||
impl DisplayGpu {
|
||||
/// The pipelines, or why the device would not make them (a shader a backend rejects): the
|
||||
/// viewer then runs the pass on the CPU instead of showing nothing.
|
||||
pub fn try_new(device: &wgpu::Device) -> Result<DisplayGpu, String> {
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
{
|
||||
let internal = device.push_error_scope(wgpu::ErrorFilter::Internal);
|
||||
let validation = device.push_error_scope(wgpu::ErrorFilter::Validation);
|
||||
let made = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| DisplayGpu::new(device)));
|
||||
let validation = pollster::block_on(validation.pop());
|
||||
let internal = pollster::block_on(internal.pop());
|
||||
if let Some(e) = validation.or(internal) {
|
||||
return Err(format!("the display pass's pipelines: {e}"));
|
||||
}
|
||||
made.map_err(|_| "the display pass's pipelines: the backend panicked".to_string())
|
||||
}
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
{
|
||||
Ok(DisplayGpu::new(device))
|
||||
}
|
||||
}
|
||||
|
||||
pub fn new(device: &wgpu::Device) -> DisplayGpu {
|
||||
let module = device.create_shader_module(wgpu::ShaderModuleDescriptor {
|
||||
label: Some("viewer display pass"),
|
||||
source: wgpu::ShaderSource::Wgsl(include_str!("display_gpu.wgsl").into()),
|
||||
});
|
||||
let tex = |binding: u32, dim: wgpu::TextureViewDimension, filterable: bool| wgpu::BindGroupLayoutEntry {
|
||||
binding,
|
||||
visibility: wgpu::ShaderStages::FRAGMENT,
|
||||
ty: wgpu::BindingType::Texture { sample_type: wgpu::TextureSampleType::Float { filterable }, view_dimension: dim, multisampled: false },
|
||||
count: None,
|
||||
};
|
||||
let uniform = wgpu::BindGroupLayoutEntry {
|
||||
binding: 0,
|
||||
visibility: wgpu::ShaderStages::FRAGMENT,
|
||||
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },
|
||||
count: None,
|
||||
};
|
||||
let bake_layout = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
|
||||
label: Some("viewer display pass"),
|
||||
entries: &[uniform, tex(1, wgpu::TextureViewDimension::D2, false), tex(2, wgpu::TextureViewDimension::D3, false)],
|
||||
});
|
||||
let mip_layout = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
|
||||
label: Some("viewer display mips"),
|
||||
entries: &[tex(1, wgpu::TextureViewDimension::D2, false)],
|
||||
});
|
||||
let pipeline = |label: &str, layout: &wgpu::BindGroupLayout, entry: &str| {
|
||||
let pl =
|
||||
device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor { label: Some(label), bind_group_layouts: &[Some(layout)], immediate_size: 0 });
|
||||
device.create_render_pipeline(&wgpu::RenderPipelineDescriptor {
|
||||
label: Some(label),
|
||||
layout: Some(&pl),
|
||||
vertex: wgpu::VertexState { module: &module, entry_point: Some("vs"), buffers: &[], compilation_options: Default::default() },
|
||||
fragment: Some(wgpu::FragmentState {
|
||||
module: &module,
|
||||
entry_point: Some(entry),
|
||||
targets: &[Some(wgpu::ColorTargetState { format: wgpu::TextureFormat::Rgba8Unorm, blend: None, write_mask: wgpu::ColorWrites::ALL })],
|
||||
compilation_options: Default::default(),
|
||||
}),
|
||||
primitive: wgpu::PrimitiveState::default(),
|
||||
depth_stencil: None,
|
||||
multisample: wgpu::MultisampleState::default(),
|
||||
multiview_mask: None,
|
||||
cache: None,
|
||||
})
|
||||
};
|
||||
let no_lut = device.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some("viewer display LUT (none)"),
|
||||
size: wgpu::Extent3d { width: 1, height: 1, depth_or_array_layers: 1 },
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D3,
|
||||
format: wgpu::TextureFormat::Rgba32Float,
|
||||
usage: wgpu::TextureUsages::TEXTURE_BINDING,
|
||||
view_formats: &[],
|
||||
});
|
||||
DisplayGpu {
|
||||
bake: pipeline("viewer display pass", &bake_layout, "fs"),
|
||||
mip: pipeline("viewer display mips", &mip_layout, "mip"),
|
||||
bake_layout,
|
||||
mip_layout,
|
||||
no_lut: no_lut.create_view(&Default::default()),
|
||||
max_side: device.limits().max_texture_dimension_2d,
|
||||
}
|
||||
}
|
||||
|
||||
/// The largest texture side the device takes.
|
||||
pub fn max_side(&self) -> usize {
|
||||
self.max_side as usize
|
||||
}
|
||||
|
||||
/// `src` in a texture (half floats or RGBA8), averaged down by `k` first when `k > 1`
|
||||
/// (8-bit frames past the texture limit, #201; half-float ones never get here, see
|
||||
/// [`Source::needs_cpu_pass`]): `reuse` when it has the size and format (a playing
|
||||
/// preview's frames), else a new one.
|
||||
fn upload(&self, device: &wgpu::Device, queue: &wgpu::Queue, src: &Source, k: usize, reuse: Option<wgpu::Texture>) -> wgpu::Texture {
|
||||
let write = |w: u32, h: u32, format: wgpu::TextureFormat, bytes: &[u8]| {
|
||||
let (w, h) = (w.max(1), h.max(1));
|
||||
let fits = |t: &wgpu::Texture| t.width() == w && t.height() == h && t.format() == format;
|
||||
let tex = reuse.clone().filter(fits).unwrap_or_else(|| {
|
||||
device.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some("viewer frame (display pass input)"),
|
||||
size: wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D2,
|
||||
format,
|
||||
usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,
|
||||
view_formats: &[],
|
||||
})
|
||||
});
|
||||
let px = if format == wgpu::TextureFormat::Rgba16Float { 8 } else { 4 };
|
||||
if bytes.len() as u64 >= w as u64 * h as u64 * px as u64 {
|
||||
queue.write_texture(
|
||||
wgpu::TexelCopyTextureInfo { texture: &tex, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
|
||||
bytes,
|
||||
wgpu::TexelCopyBufferLayout { offset: 0, bytes_per_row: Some(w * px), rows_per_image: Some(h) },
|
||||
wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
|
||||
);
|
||||
}
|
||||
tex
|
||||
};
|
||||
let f16 = wgpu::TextureFormat::Rgba16Float;
|
||||
let rgba8 = wgpu::TextureFormat::Rgba8Unorm;
|
||||
match src {
|
||||
Source::F16(f) => {
|
||||
debug_assert_eq!(k, 1, "half-float frames past the texture limit take the CPU pass");
|
||||
write(f.width, f.height, f16, bytemuck::cast_slice(f.bits()))
|
||||
}
|
||||
Source::Rgba8(c) if k > 1 => {
|
||||
let c = crate::frames::fit_texture_by((**c).clone(), usize::MAX, k);
|
||||
write(c.size[0] as u32, c.size[1] as u32, rgba8, c.as_raw())
|
||||
}
|
||||
Source::Rgba8(c) => write(c.size[0] as u32, c.size[1] as u32, rgba8, c.as_raw()),
|
||||
}
|
||||
}
|
||||
|
||||
fn lut(&self, device: &wgpu::Device, queue: &wgpu::Queue, lut: &effectcraft_engine::color::icc::Lut3d) -> wgpu::Texture {
|
||||
let n = lut.size.max(2) as u32;
|
||||
let tex = device.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some("viewer display LUT"),
|
||||
size: wgpu::Extent3d { width: n, height: n, depth_or_array_layers: n },
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D3,
|
||||
format: wgpu::TextureFormat::Rgba32Float,
|
||||
usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,
|
||||
view_formats: &[],
|
||||
});
|
||||
// data[(r · n + g) · n + b]: x = b, y = g, z = r.
|
||||
let mut texels: Vec<f32> = Vec::with_capacity((n * n * n * 4) as usize);
|
||||
for i in 0..(n * n * n) as usize {
|
||||
let v = lut.data.get(i).copied().unwrap_or_default();
|
||||
texels.extend([v[0], v[1], v[2], 1.0]);
|
||||
}
|
||||
queue.write_texture(
|
||||
wgpu::TexelCopyTextureInfo { texture: &tex, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
|
||||
bytemuck::cast_slice(&texels),
|
||||
wgpu::TexelCopyBufferLayout { offset: 0, bytes_per_row: Some(n * 16), rows_per_image: Some(n) },
|
||||
wgpu::Extent3d { width: n, height: n, depth_or_array_layers: n },
|
||||
);
|
||||
tex
|
||||
}
|
||||
|
||||
/// The display texture for `input`: premultiplied RGBA8, a full mip chain.
|
||||
fn output(device: &wgpu::Device, w: u32, h: u32) -> wgpu::Texture {
|
||||
device.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some("viewer frame (displayed)"),
|
||||
size: wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
|
||||
mip_level_count: mip_levels(w, h),
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D2,
|
||||
format: wgpu::TextureFormat::Rgba8Unorm,
|
||||
usage: wgpu::TextureUsages::RENDER_ATTACHMENT | wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_SRC,
|
||||
view_formats: &[],
|
||||
})
|
||||
}
|
||||
|
||||
/// Run the pass from `input` into level 0 of `out`, then fill its mips.
|
||||
fn run(
|
||||
&self,
|
||||
device: &wgpu::Device,
|
||||
queue: &wgpu::Queue,
|
||||
input: &wgpu::Texture,
|
||||
out: &wgpu::Texture,
|
||||
params: &[u8; UNIFORM_BYTES],
|
||||
lut: Option<&wgpu::Texture>,
|
||||
) {
|
||||
use wgpu::util::DeviceExt as _;
|
||||
let buf = device.create_buffer_init(&wgpu::util::BufferInitDescriptor {
|
||||
label: Some("viewer display pass"),
|
||||
contents: params,
|
||||
usage: wgpu::BufferUsages::UNIFORM,
|
||||
});
|
||||
let input_view = input.create_view(&Default::default());
|
||||
let lut_view = lut.map(|l| l.create_view(&Default::default()));
|
||||
let bake = device.create_bind_group(&wgpu::BindGroupDescriptor {
|
||||
label: Some("viewer display pass"),
|
||||
layout: &self.bake_layout,
|
||||
entries: &[
|
||||
wgpu::BindGroupEntry { binding: 0, resource: buf.as_entire_binding() },
|
||||
wgpu::BindGroupEntry { binding: 1, resource: wgpu::BindingResource::TextureView(&input_view) },
|
||||
wgpu::BindGroupEntry { binding: 2, resource: wgpu::BindingResource::TextureView(lut_view.as_ref().unwrap_or(&self.no_lut)) },
|
||||
],
|
||||
});
|
||||
let level = |l: u32| out.create_view(&wgpu::TextureViewDescriptor { base_mip_level: l, mip_level_count: Some(1), ..Default::default() });
|
||||
let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("viewer display pass") });
|
||||
let pass = |enc: &mut wgpu::CommandEncoder, target: &wgpu::TextureView, pipeline: &wgpu::RenderPipeline, bg: &wgpu::BindGroup| {
|
||||
let mut p = enc.begin_render_pass(&wgpu::RenderPassDescriptor {
|
||||
label: Some("viewer display pass"),
|
||||
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
|
||||
view: target,
|
||||
depth_slice: None,
|
||||
resolve_target: None,
|
||||
ops: wgpu::Operations { load: wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT), store: wgpu::StoreOp::Store },
|
||||
})],
|
||||
..Default::default()
|
||||
});
|
||||
p.set_pipeline(pipeline);
|
||||
p.set_bind_group(0, bg, &[]);
|
||||
p.draw(0..3, 0..1);
|
||||
};
|
||||
pass(&mut enc, &level(0), &self.bake, &bake);
|
||||
for l in 1..out.mip_level_count() {
|
||||
let above = level(l - 1);
|
||||
let bg = device.create_bind_group(&wgpu::BindGroupDescriptor {
|
||||
label: Some("viewer display mips"),
|
||||
layout: &self.mip_layout,
|
||||
entries: &[wgpu::BindGroupEntry { binding: 1, resource: wgpu::BindingResource::TextureView(&above) }],
|
||||
});
|
||||
pass(&mut enc, &level(l), &self.mip, &bg);
|
||||
}
|
||||
queue.submit([enc.finish()]);
|
||||
}
|
||||
}
|
||||
|
||||
/// One viewer's displayed texture (the Composition viewer, a passive viewer, the snapshot): the
|
||||
/// frame it was made from and the pass's uniform, so it runs again only when either changes.
|
||||
#[derive(Default)]
|
||||
pub struct Bake {
|
||||
source: Option<Source>,
|
||||
/// Averaged down by this for the texture limit (1: not).
|
||||
factor: usize,
|
||||
input: Option<wgpu::Texture>,
|
||||
output: Option<wgpu::Texture>,
|
||||
id: Option<egui::TextureId>,
|
||||
params: Option<[u8; UNIFORM_BYTES]>,
|
||||
lut: Option<(Arc<effectcraft_engine::color::icc::Lut3d>, wgpu::Texture)>,
|
||||
smooth: Option<bool>,
|
||||
/// How often the pass ran (tests).
|
||||
runs: u64,
|
||||
}
|
||||
|
||||
impl Bake {
|
||||
/// The egui texture showing `src` through `pass` (sampled with the Viewer Zoom Quality's
|
||||
/// sampler for `smooth`), running the pass again when the frame or the pass changed.
|
||||
pub fn show(&mut self, rs: &egui_wgpu::RenderState, gpu: &DisplayGpu, src: &Source, pass: &DisplayPass, smooth: bool) -> Option<egui::TextureId> {
|
||||
if src.needs_cpu_pass(gpu.max_side()) {
|
||||
return None;
|
||||
}
|
||||
let (device, queue) = (&rs.device, &rs.queue);
|
||||
let params = uniform(pass);
|
||||
let new_source = !self.source.as_ref().is_some_and(|s| s.same(src));
|
||||
if new_source {
|
||||
self.factor = crate::frames::fit_factor(src.size(), gpu.max_side(), 1);
|
||||
self.input = Some(gpu.upload(device, queue, src, self.factor, self.input.take()));
|
||||
self.source = Some(src.clone());
|
||||
}
|
||||
let lut = match &pass.color {
|
||||
Some(PassColor::Lut(l)) => Some(l.clone()),
|
||||
_ => None,
|
||||
};
|
||||
// A new LUT (another profile, Preserve RGB, another display) has the same size, so the
|
||||
// uniform alone doesn't tell: it reruns the pass itself.
|
||||
let new_lut = match (&lut, &self.lut) {
|
||||
(Some(l), Some((have, _))) if Arc::ptr_eq(l, have) => false,
|
||||
(Some(l), _) => {
|
||||
self.lut = Some((l.clone(), gpu.lut(device, queue, l)));
|
||||
true
|
||||
}
|
||||
(None, had) => {
|
||||
let changed = had.is_some();
|
||||
self.lut = None;
|
||||
changed
|
||||
}
|
||||
};
|
||||
let input = self.input.as_ref()?;
|
||||
let (w, h) = (input.width(), input.height());
|
||||
let resized = self.output.as_ref().is_none_or(|o| o.width() != w || o.height() != h);
|
||||
if resized {
|
||||
self.output = Some(DisplayGpu::output(device, w, h));
|
||||
}
|
||||
let output = self.output.as_ref()?;
|
||||
if new_source || resized || new_lut || self.params != Some(params) {
|
||||
gpu.run(device, queue, input, output, ¶ms, self.lut.as_ref().map(|(_, t)| t));
|
||||
self.params = Some(params);
|
||||
self.runs += 1;
|
||||
}
|
||||
let view = output.create_view(&Default::default());
|
||||
let sampler = || crate::panels::viewer::zoom_sampler(smooth);
|
||||
let mut ren = rs.renderer.write();
|
||||
let id = match self.id {
|
||||
Some(id) if !resized && self.smooth == Some(smooth) => id,
|
||||
Some(id) => {
|
||||
ren.update_egui_texture_from_wgpu_texture_with_sampler_options(device, &view, sampler(), id);
|
||||
id
|
||||
}
|
||||
None => ren.register_native_texture_with_sampler_options(device, &view, sampler()),
|
||||
};
|
||||
self.id = Some(id);
|
||||
self.smooth = Some(smooth);
|
||||
Some(id)
|
||||
}
|
||||
|
||||
/// The texture egui draws, once [`Bake::show`] made it.
|
||||
pub fn id(&self) -> Option<egui::TextureId> {
|
||||
self.id
|
||||
}
|
||||
|
||||
/// The frame shown (the one last given to [`Bake::show`]).
|
||||
pub fn source(&self) -> Option<&Source> {
|
||||
self.source.as_ref()
|
||||
}
|
||||
|
||||
/// The texture coordinates of the frame in its texture (see `frames::fitted_uv`).
|
||||
pub fn uv(&self) -> egui::Rect {
|
||||
let size = self.source.as_ref().map_or([1, 1], Source::size);
|
||||
crate::frames::fitted_uv(size, self.factor.max(1))
|
||||
}
|
||||
|
||||
/// Give the texture back to the renderer.
|
||||
pub fn free(&mut self, rs: Option<&egui_wgpu::RenderState>) {
|
||||
if let (Some(id), Some(rs)) = (self.id.take(), rs) {
|
||||
rs.renderer.write().free_texture(&id);
|
||||
}
|
||||
*self = Bake::default();
|
||||
}
|
||||
|
||||
/// How often the pass ran for this slot.
|
||||
pub fn runs(&self) -> u64 {
|
||||
self.runs
|
||||
}
|
||||
|
||||
/// The displayed texture (tests read it back).
|
||||
pub fn texture(&self) -> Option<&wgpu::Texture> {
|
||||
self.output.as_ref()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
@@ -0,0 +1,314 @@
|
||||
// The viewer's display pass (#2797): a rendered frame (premultiplied, encoded in the output
|
||||
// space; Rgba16Float for 32 bpc frames, Rgba8Unorm otherwise) → the premultiplied RGBA8 texture
|
||||
// the viewer draws. Line by line the same operations as `effectcraft_engine::viewer::DisplayPass`
|
||||
// (its `pixel` is this shader's CPU reference and fallback):
|
||||
// unpremultiply → decode, × 2^stops, Compand / Tone Map, encode → clamp → display colour →
|
||||
// Show Channel, premultiply → floor(v · 255 + 0.5).
|
||||
|
||||
struct Params {
|
||||
// channel (0 RGB, 1 R, 2 G, 3 B, 4 A, 5 RGB Straight), colorized, frame curve, HDR step
|
||||
// (0 none, 1 Compand, 2 Tone Map)
|
||||
ids0: vec4<u32>,
|
||||
// display colour (0 none, 1 conversions, 2 LUT), 8-bit device numbers, a second conversion,
|
||||
// LUT size
|
||||
ids1: vec4<u32>,
|
||||
// curves: first conversion's decode and encode, second conversion's decode and encode
|
||||
ids2: vec4<u32>,
|
||||
// gain (2^stops), frame curve's gamma, first / second conversion has a matrix (1.0)
|
||||
vals0: vec4<f32>,
|
||||
// gammas of the ids2 curves (encoding: 1 / gamma)
|
||||
vals1: vec4<f32>,
|
||||
// 1 / the frame curve's gamma, 1 / 2.4, 1 / PQ m2, 1 / PQ m1: divided on the CPU, as the
|
||||
// reference does, so the exponents are the same bits
|
||||
vals2: vec4<f32>,
|
||||
m1: array<vec4<f32>, 3>,
|
||||
m2: array<vec4<f32>, 3>,
|
||||
};
|
||||
|
||||
@group(0) @binding(0) var<uniform> P: Params;
|
||||
@group(0) @binding(1) var frame: texture_2d<f32>;
|
||||
@group(0) @binding(2) var lut: texture_3d<f32>;
|
||||
|
||||
@vertex
|
||||
fn vs(@builtin(vertex_index) i: u32) -> @builtin(position) vec4<f32> {
|
||||
// One triangle covering the target.
|
||||
let x = f32((i << 1u) & 2u);
|
||||
let y = f32(i & 2u);
|
||||
return vec4<f32>(x * 2.0 - 1.0, 1.0 - y * 2.0, 0.0, 1.0);
|
||||
}
|
||||
|
||||
// The curves take the gamma to decode with, or 1 / gamma to encode with.
|
||||
// SMPTE ST 2084 and BT.2100 HLG (effectcraft_color::space), linear 1.0 = reference white.
|
||||
const PQ_M1: f32 = 0.1593017578125;
|
||||
const PQ_M2: f32 = 78.84375;
|
||||
const PQ_C1: f32 = 0.8359375;
|
||||
const PQ_C2: f32 = 18.8515625;
|
||||
const PQ_C3: f32 = 18.6875;
|
||||
const PQ_WHITE: f32 = 0.0203;
|
||||
const HLG_A: f32 = 0.17883277;
|
||||
const HLG_B: f32 = 0.28466892;
|
||||
const HLG_C: f32 = 0.55991073;
|
||||
const HLG_REFERENCE: f32 = 0.26496256;
|
||||
|
||||
fn power(a: f32, g: f32) -> f32 {
|
||||
if (a <= 0.0) {
|
||||
return 0.0;
|
||||
}
|
||||
return pow(a, g);
|
||||
}
|
||||
|
||||
fn copysign(m: f32, v: f32) -> f32 {
|
||||
if (v < 0.0) {
|
||||
return -m;
|
||||
}
|
||||
return m;
|
||||
}
|
||||
|
||||
fn decode(curve: u32, g: f32, v: f32) -> f32 {
|
||||
let a = abs(v);
|
||||
var l = a;
|
||||
switch curve {
|
||||
case 1u: {
|
||||
if (a <= 0.04045) {
|
||||
l = a / 12.92;
|
||||
} else {
|
||||
l = power((a + 0.055) / 1.055, 2.4);
|
||||
}
|
||||
}
|
||||
case 2u: {
|
||||
l = power(a, g);
|
||||
}
|
||||
case 3u: {
|
||||
let p = power(min(a, 1.0), P.vals2.z);
|
||||
l = power(max(p - PQ_C1, 0.0) / (PQ_C2 - PQ_C3 * p), P.vals2.w) / PQ_WHITE;
|
||||
}
|
||||
case 4u: {
|
||||
var e: f32;
|
||||
if (a <= 0.5) {
|
||||
e = a * a / 3.0;
|
||||
} else {
|
||||
e = (exp((a - HLG_C) / HLG_A) + HLG_B) / 12.0;
|
||||
}
|
||||
l = e / HLG_REFERENCE;
|
||||
}
|
||||
default: {}
|
||||
}
|
||||
return copysign(l, v);
|
||||
}
|
||||
|
||||
fn encode(curve: u32, g: f32, v: f32) -> f32 {
|
||||
let a = abs(v);
|
||||
var e = a;
|
||||
switch curve {
|
||||
case 1u: {
|
||||
if (a <= 0.0031308) {
|
||||
e = a * 12.92;
|
||||
} else {
|
||||
e = 1.055 * power(a, P.vals2.y) - 0.055;
|
||||
}
|
||||
}
|
||||
case 2u: {
|
||||
e = power(a, g);
|
||||
}
|
||||
case 3u: {
|
||||
let y = power(a * PQ_WHITE, PQ_M1);
|
||||
e = power((PQ_C1 + PQ_C2 * y) / (1.0 + PQ_C3 * y), PQ_M2);
|
||||
}
|
||||
case 4u: {
|
||||
let s = a * HLG_REFERENCE;
|
||||
if (s <= 1.0 / 12.0) {
|
||||
e = sqrt(3.0 * s);
|
||||
} else {
|
||||
e = HLG_A * log(12.0 * s - HLG_B) + HLG_C;
|
||||
}
|
||||
}
|
||||
default: {}
|
||||
}
|
||||
return copysign(e, v);
|
||||
}
|
||||
|
||||
fn decode3(curve: u32, g: f32, c: vec3<f32>) -> vec3<f32> {
|
||||
return vec3<f32>(decode(curve, g, c.x), decode(curve, g, c.y), decode(curve, g, c.z));
|
||||
}
|
||||
|
||||
fn encode3(curve: u32, g: f32, c: vec3<f32>) -> vec3<f32> {
|
||||
return vec3<f32>(encode(curve, g, c.x), encode(curve, g, c.y), encode(curve, g, c.z));
|
||||
}
|
||||
|
||||
fn mat_mul(m: array<vec4<f32>, 3>, l: vec3<f32>) -> vec3<f32> {
|
||||
return vec3<f32>(
|
||||
m[0].x * l.x + m[0].y * l.y + m[0].z * l.z,
|
||||
m[1].x * l.x + m[1].y * l.y + m[1].z * l.z,
|
||||
m[2].x * l.x + m[2].y * l.y + m[2].z * l.z,
|
||||
);
|
||||
}
|
||||
|
||||
// effectcraft_render::color::compand
|
||||
fn compand(v: f32) -> f32 {
|
||||
let k = 0.8;
|
||||
if (v <= k) {
|
||||
return v;
|
||||
}
|
||||
return k + (1.0 - k) * (1.0 - exp(-(v - k) / (1.0 - k)));
|
||||
}
|
||||
|
||||
// effectcraft_render::color::tone_map (extended Reinhard of Rec. 709 luminance, white 4.0)
|
||||
fn tone_map(c: vec3<f32>) -> vec3<f32> {
|
||||
let w = 4.0;
|
||||
let l = 0.2126 * c.x + 0.7152 * c.y + 0.0722 * c.z;
|
||||
if (l <= 0.0) {
|
||||
return c;
|
||||
}
|
||||
let lm = l * (1.0 + l / (w * w)) / (1.0 + l);
|
||||
let k = lm / l;
|
||||
return c * k;
|
||||
}
|
||||
|
||||
fn lut_at(n: u32, r: u32, g: u32, b: u32) -> vec3<f32> {
|
||||
return textureLoad(lut, vec3<u32>(b, g, r), 0).xyz;
|
||||
}
|
||||
|
||||
// effectcraft_engine::viewer::lut_eval: trilinear in the same order.
|
||||
fn lut_eval(c: vec3<f32>) -> vec3<f32> {
|
||||
let n = P.ids1.w;
|
||||
let x = clamp(c, vec3<f32>(0.0), vec3<f32>(1.0)) * f32(n - 1u);
|
||||
let k = min(vec3<u32>(floor(x)), vec3<u32>(n - 2u));
|
||||
let f = x - vec3<f32>(k);
|
||||
var out = vec3<f32>(0.0);
|
||||
for (var corner = 0u; corner < 8u; corner++) {
|
||||
let d = vec3<u32>((corner >> 2u) & 1u, (corner >> 1u) & 1u, corner & 1u);
|
||||
var wr = 1.0 - f.x;
|
||||
if (d.x == 1u) {
|
||||
wr = f.x;
|
||||
}
|
||||
var wg = 1.0 - f.y;
|
||||
if (d.y == 1u) {
|
||||
wg = f.y;
|
||||
}
|
||||
var wb = 1.0 - f.z;
|
||||
if (d.z == 1u) {
|
||||
wb = f.z;
|
||||
}
|
||||
let w = wr * wg * wb;
|
||||
out = out + w * lut_at(n, k.x + d.x, k.y + d.y, k.z + d.z);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
fn display_colour(c: vec3<f32>) -> vec3<f32> {
|
||||
var o = c;
|
||||
if (P.ids1.x == 2u) {
|
||||
o = lut_eval(c);
|
||||
} else if (P.ids1.x == 1u) {
|
||||
o = decode3(P.ids2.x, P.vals1.x, o);
|
||||
if (P.vals0.z == 1.0) {
|
||||
o = mat_mul(P.m1, o);
|
||||
}
|
||||
o = encode3(P.ids2.y, P.vals1.y, o);
|
||||
if (P.ids1.y == 1u) {
|
||||
o = floor(clamp(o, vec3<f32>(0.0), vec3<f32>(1.0)) * 255.0 + 0.5) / 255.0;
|
||||
}
|
||||
if (P.ids1.z == 1u) {
|
||||
o = decode3(P.ids2.z, P.vals1.z, o);
|
||||
if (P.vals0.w == 1.0) {
|
||||
o = mat_mul(P.m2, o);
|
||||
}
|
||||
o = encode3(P.ids2.w, P.vals1.w, o);
|
||||
}
|
||||
}
|
||||
return clamp(o, vec3<f32>(0.0), vec3<f32>(1.0));
|
||||
}
|
||||
|
||||
fn q(v: f32) -> f32 {
|
||||
return floor(clamp(v, 0.0, 1.0) * 255.0 + 0.5) / 255.0;
|
||||
}
|
||||
|
||||
@fragment
|
||||
fn fs(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
|
||||
let s = textureLoad(frame, vec2<i32>(floor(pos.xy)), 0);
|
||||
// Frames are never NaN here: `frames::F16Frame` and 8-bit frames hold finite values only
|
||||
// (the NaN test is the reference's, kept for safety; WGSL may drop it). Alpha is clamped to
|
||||
// 0..1 first, as `display_pixel` and `DisplayPass::pixel` do (the compositor's alpha never
|
||||
// exceeds 1).
|
||||
var a = s.w;
|
||||
if (!(a == a)) {
|
||||
a = 0.0;
|
||||
}
|
||||
a = clamp(a, 0.0, 1.0);
|
||||
var c = vec3<f32>(0.0);
|
||||
if (a > 0.0) {
|
||||
c = s.xyz / a;
|
||||
}
|
||||
let channel = P.ids0.x;
|
||||
let curve = P.ids0.z;
|
||||
let g = P.vals0.y;
|
||||
let gain = P.vals0.x;
|
||||
let light = gain != 1.0 || P.ids0.w != 0u;
|
||||
// Premultiplied channels without exposure or HDR step: straight from the frame.
|
||||
var pm = clamp(s.xyz, vec3<f32>(0.0), vec3<f32>(a));
|
||||
if (light) {
|
||||
var l = decode3(curve, g, c) * gain;
|
||||
if (P.ids0.w == 1u) {
|
||||
l = vec3<f32>(compand(l.x), compand(l.y), compand(l.z));
|
||||
} else if (P.ids0.w == 2u) {
|
||||
l = tone_map(l);
|
||||
}
|
||||
c = encode3(curve, P.vals2.x, l);
|
||||
}
|
||||
c = clamp(c, vec3<f32>(0.0), vec3<f32>(1.0));
|
||||
if (light) {
|
||||
pm = c * a;
|
||||
}
|
||||
if (channel == 0u || channel == 5u) {
|
||||
if (P.ids1.x != 0u) {
|
||||
c = display_colour(c);
|
||||
pm = c * a;
|
||||
}
|
||||
if (channel == 5u) {
|
||||
return vec4<f32>(q(c.x), q(c.y), q(c.z), 1.0);
|
||||
}
|
||||
return vec4<f32>(q(pm.x), q(pm.y), q(pm.z), q(a));
|
||||
}
|
||||
if (channel == 4u) {
|
||||
var v = a;
|
||||
if (gain != 1.0) {
|
||||
v = encode(curve, P.vals2.x, decode(curve, g, a) * gain);
|
||||
}
|
||||
let e = q(v);
|
||||
return vec4<f32>(e, e, e, 1.0);
|
||||
}
|
||||
let v = q(pm[channel - 1u]);
|
||||
if (P.ids0.y == 1u) {
|
||||
var o = vec4<f32>(0.0, 0.0, 0.0, 1.0);
|
||||
o[channel - 1u] = v;
|
||||
return o;
|
||||
}
|
||||
return vec4<f32>(v, v, v, 1.0);
|
||||
}
|
||||
|
||||
// The next mip level of the displayed texture (`frame` = the level above): each texel the mean
|
||||
// of the premultiplied texels it covers, 2 × 2, and at the right / bottom edge of a level above
|
||||
// of odd size also its last column / row (3 wide or high), so no texel of the frame is dropped
|
||||
// (the level sizes are the GPU's, rounded down). Rounded as the pass rounds.
|
||||
@fragment
|
||||
fn mip(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
|
||||
let above = vec2<i32>(textureDimensions(frame));
|
||||
let size = max(above / 2, vec2<i32>(1));
|
||||
let p = vec2<i32>(floor(pos.xy));
|
||||
let lo = min(p * 2, above - vec2<i32>(1));
|
||||
var hi = min(p * 2 + vec2<i32>(2), above);
|
||||
if (p.x == size.x - 1) {
|
||||
hi.x = above.x;
|
||||
}
|
||||
if (p.y == size.y - 1) {
|
||||
hi.y = above.y;
|
||||
}
|
||||
var s = vec4<f32>(0.0);
|
||||
for (var y = lo.y; y < hi.y; y++) {
|
||||
for (var x = lo.x; x < hi.x; x++) {
|
||||
s = s + textureLoad(frame, vec2<i32>(x, y), 0);
|
||||
}
|
||||
}
|
||||
let n = f32((hi.x - lo.x) * (hi.y - lo.y));
|
||||
return floor(s / n * 255.0 + 0.5) / 255.0;
|
||||
}
|
||||
@@ -0,0 +1,582 @@
|
||||
//! The display pass on a real device (any wgpu adapter; CI runs lavapipe), held to its CPU
|
||||
//! reference `DisplayPass::pixel`. Skips without an adapter.
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use effectcraft_engine::project::HdrMode;
|
||||
use effectcraft_engine::render::Image;
|
||||
use effectcraft_engine::viewer::{Channel, DisplayColor, DisplayPass, FrameEncoding, PassCurve, SimProfile, Simulation};
|
||||
use eframe::egui_wgpu;
|
||||
use eframe::wgpu;
|
||||
|
||||
use super::{DisplayGpu, Source};
|
||||
use crate::frames::F16Frame;
|
||||
|
||||
/// One device at a time (tests run in parallel).
|
||||
static DEVICE: Mutex<()> = Mutex::new(());
|
||||
|
||||
struct Dev {
|
||||
instance: wgpu::Instance,
|
||||
adapter: wgpu::Adapter,
|
||||
device: wgpu::Device,
|
||||
queue: wgpu::Queue,
|
||||
/// A software rasterizer (lavapipe / llvmpipe, as CI has): its pow / exp / division round
|
||||
/// differently again, so exact comparisons allow 1 LSB on a few bytes there.
|
||||
cpu: bool,
|
||||
}
|
||||
|
||||
fn device() -> Option<Dev> {
|
||||
let instance = wgpu::Instance::new(wgpu::InstanceDescriptor::new_without_display_handle().with_env());
|
||||
let adapter = pollster::block_on(instance.request_adapter(&wgpu::RequestAdapterOptions::default())).ok()?;
|
||||
let (device, queue) =
|
||||
pollster::block_on(adapter.request_device(&wgpu::DeviceDescriptor { label: Some("display pass tests"), ..Default::default() })).ok()?;
|
||||
let info = adapter.get_info();
|
||||
eprintln!("display pass tests on {:?} ({:?})", info.name, info.device_type);
|
||||
Some(Dev { cpu: info.device_type == wgpu::DeviceType::Cpu, instance, adapter, device, queue })
|
||||
}
|
||||
|
||||
/// `compare`'s result is exact, or (software rasterizers) at most 1 LSB on at most 1 % of bytes.
|
||||
fn exact_enough(d: &Dev, (n, max): (usize, i32), bytes: usize) -> bool {
|
||||
(n, max) == (0, 0) || d.cpu && max <= 1 && n * 100 <= bytes
|
||||
}
|
||||
|
||||
/// egui-wgpu's render state on the test device, as eframe hands it to the app.
|
||||
fn render_state(d: &Dev) -> egui_wgpu::RenderState {
|
||||
let renderer = egui_wgpu::Renderer::new(&d.device, wgpu::TextureFormat::Rgba8Unorm, egui_wgpu::RendererOptions::default());
|
||||
egui_wgpu::RenderState {
|
||||
adapter: d.adapter.clone(),
|
||||
available_adapters: vec![d.adapter.clone()],
|
||||
instance: d.instance.clone(),
|
||||
device: d.device.clone(),
|
||||
queue: d.queue.clone(),
|
||||
target_format: wgpu::TextureFormat::Rgba8Unorm,
|
||||
renderer: Arc::new(egui::mutex::RwLock::new(renderer)),
|
||||
}
|
||||
}
|
||||
|
||||
/// A 17³ LUT that mixes channels and bends the curve (as an ICC profile's device does).
|
||||
fn test_lut(k: f64) -> Arc<effectcraft_engine::color::icc::Lut3d> {
|
||||
Arc::new(effectcraft_engine::color::icc::Lut3d::bake(17, |c| {
|
||||
[c[0].powf(1.3) * 0.9 + c[1] * 0.1, c[1] * 0.8 + c[2] * 0.2, (c[2] * 0.7 + c[0] * 0.3).sqrt() * k]
|
||||
}))
|
||||
}
|
||||
|
||||
fn read(d: &Dev, tex: &wgpu::Texture, level: u32) -> (Vec<[u8; 4]>, [u32; 2]) {
|
||||
let (w, h) = ((tex.width() >> level).max(1), (tex.height() >> level).max(1));
|
||||
let stride = (w * 4).div_ceil(256) * 256;
|
||||
let buf = d.device.create_buffer(&wgpu::BufferDescriptor {
|
||||
label: None,
|
||||
size: (stride * h) as u64,
|
||||
usage: wgpu::BufferUsages::MAP_READ | wgpu::BufferUsages::COPY_DST,
|
||||
mapped_at_creation: false,
|
||||
});
|
||||
let mut e = d.device.create_command_encoder(&Default::default());
|
||||
e.copy_texture_to_buffer(
|
||||
wgpu::TexelCopyTextureInfo { texture: tex, mip_level: level, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
|
||||
wgpu::TexelCopyBufferInfo { buffer: &buf, layout: wgpu::TexelCopyBufferLayout { offset: 0, bytes_per_row: Some(stride), rows_per_image: Some(h) } },
|
||||
wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
|
||||
);
|
||||
d.queue.submit([e.finish()]);
|
||||
buf.slice(..).map_async(wgpu::MapMode::Read, |r| r.expect("map"));
|
||||
d.device.poll(wgpu::PollType::wait_indefinitely()).expect("poll");
|
||||
let data = buf.slice(..).get_mapped_range().expect("mapped").to_vec();
|
||||
let mut out = Vec::with_capacity((w * h) as usize);
|
||||
for y in 0..h {
|
||||
let row = &data[(y * stride) as usize..][..(w * 4) as usize];
|
||||
out.extend(row.as_chunks::<4>().0.iter().copied());
|
||||
}
|
||||
(out, [w, h])
|
||||
}
|
||||
|
||||
/// The pass on the device: level 0 of the displayed texture, and the texture.
|
||||
fn bake(d: &Dev, gpu: &DisplayGpu, src: &Source, pass: &DisplayPass) -> (Vec<[u8; 4]>, wgpu::Texture) {
|
||||
let k = crate::frames::fit_factor(src.size(), gpu.max_side(), 1);
|
||||
let input = gpu.upload(&d.device, &d.queue, src, k, None);
|
||||
let out = DisplayGpu::output(&d.device, input.width(), input.height());
|
||||
let lut = match &pass.color {
|
||||
Some(effectcraft_engine::viewer::PassColor::Lut(l)) => Some(gpu.lut(&d.device, &d.queue, l)),
|
||||
_ => None,
|
||||
};
|
||||
gpu.run(&d.device, &d.queue, &input, &out, &super::uniform(pass), lut.as_ref());
|
||||
(read(d, &out, 0).0, out)
|
||||
}
|
||||
|
||||
fn frame(px: &[[f32; 4]], w: u32, encoding: FrameEncoding) -> Arc<F16Frame> {
|
||||
let mut img = Image::new(w, px.len() as u32 / w);
|
||||
img.data.copy_from_slice(px);
|
||||
Arc::new(F16Frame::from_image(&img, encoding))
|
||||
}
|
||||
|
||||
/// GPU output against the reference on the frame's own (half-float) values: the number of
|
||||
/// bytes that differ and the largest difference.
|
||||
fn compare(got: &[[u8; 4]], f: &F16Frame, pass: &DisplayPass) -> (usize, i32) {
|
||||
let (mut n, mut max) = (0, 0);
|
||||
for (i, g) in got.iter().enumerate() {
|
||||
let want = pass.pixel(f.pixel(i).expect("pixel"));
|
||||
for c in 0..4 {
|
||||
let d = (g[c] as i32 - want[c] as i32).abs();
|
||||
if d > 0 {
|
||||
n += 1;
|
||||
max = max.max(d);
|
||||
}
|
||||
}
|
||||
}
|
||||
(n, max)
|
||||
}
|
||||
|
||||
const SRGB: FrameEncoding = FrameEncoding { curve: PassCurve::Srgb, hdr: HdrMode::Clip };
|
||||
|
||||
fn grey(l: f32) -> [f32; 4] {
|
||||
let e = PassCurve::Srgb.encode(l);
|
||||
[e, e, e, 1.0]
|
||||
}
|
||||
|
||||
/// The issue's values (4.0 at -2 EV is white, 0.5 at +1 EV is white) and the HDR golden table,
|
||||
/// bit for bit on the GPU.
|
||||
#[test]
|
||||
fn hdr_values_on_the_gpu() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let ls = [0.25f32, 0.5, 1.0, 2.0, 4.0, 16.0];
|
||||
let f = frame(&ls.map(grey), 6, SRGB);
|
||||
let want: [[u8; 6]; 5] = [
|
||||
[137, 188, 255, 255, 255, 255],
|
||||
[99, 137, 188, 255, 255, 255],
|
||||
[71, 99, 137, 188, 255, 255],
|
||||
[34, 49, 71, 99, 137, 255],
|
||||
[188, 255, 255, 255, 255, 255],
|
||||
];
|
||||
for (stops, w) in [0.0f32, -1.0, -2.0, -4.0, 1.0].into_iter().zip(want) {
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, stops, SRGB, None);
|
||||
let (got, _) = bake(&d, &gpu, &Source::F16(f.clone()), &pass);
|
||||
let r: Vec<u8> = got.iter().map(|p| p[0]).collect();
|
||||
assert!(r.iter().zip(w).all(|(g, w)| *g == w || d.cpu && g.abs_diff(w) <= 1), "{stops} EV: {r:?} vs {w:?}");
|
||||
let c = compare(&got, &f, &pass);
|
||||
assert!(c == (0, 0) || d.cpu && c.1 <= 1, "{stops} EV: {c:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Exposure 0 on half floats: bit for bit the reference, and the bytes the 8-bit viewer showed
|
||||
/// for values in 0..1 (at least 97.9 % equal, never more than 1 apart: the half-float storage).
|
||||
#[test]
|
||||
fn exposure_zero_parity_on_the_gpu() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, 0.0, SRGB, None);
|
||||
for alpha in [1.0f32, 0.5] {
|
||||
let img = crate::frames::tests::parity_frame(alpha);
|
||||
let f = Arc::new(F16Frame::from_image(&img, SRGB));
|
||||
let (got, _) = bake(&d, &gpu, &Source::F16(f.clone()), &pass);
|
||||
let c = compare(&got, &f, &pass);
|
||||
assert!(exact_enough(&d, c, got.len() * 4), "alpha {alpha}: {c:?}");
|
||||
let old = crate::frames::to_color_image(&img);
|
||||
let (mut same, mut max) = (0, 0);
|
||||
for (g, o) in got.iter().zip(&old.pixels) {
|
||||
let o = o.to_array();
|
||||
for c in 0..3 {
|
||||
same += (g[c] == o[c]) as usize;
|
||||
max = max.max((g[c] as i32 - o[c] as i32).abs());
|
||||
}
|
||||
assert_eq!(g[3], o[3]);
|
||||
}
|
||||
let share = same as f64 / (got.len() * 3) as f64;
|
||||
eprintln!("GPU, alpha {alpha}: {:.2} % of colour bytes as before, max {max}", share * 100.0);
|
||||
assert!(share >= 0.979 && max <= 1, "alpha {alpha}: {share}, max {max}");
|
||||
}
|
||||
// The issue's two values, exactly.
|
||||
let f = frame(&[grey(4.0), grey(0.5)], 2, SRGB);
|
||||
let (m2, _) = bake(&d, &gpu, &Source::F16(f.clone()), &DisplayPass::new(Channel::Rgb, false, -2.0, SRGB, None));
|
||||
let (p1, _) = bake(&d, &gpu, &Source::F16(f), &DisplayPass::new(Channel::Rgb, false, 1.0, SRGB, None));
|
||||
let white = |p: [u8; 4]| p.iter().all(|c| *c == 255 || d.cpu && *c >= 254);
|
||||
assert!(white(m2[0]) && white(p1[1]), "{:?} {:?}", m2[0], p1[1]);
|
||||
}
|
||||
|
||||
/// Every pass setting the viewer has, against the reference on the same half-float input:
|
||||
/// exposure, HDR step (after exposure), Show Channel, curves and display colour.
|
||||
#[test]
|
||||
fn the_shader_is_the_reference() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
// 64 × 64: colours 0..2 in every channel, alpha 1, 0.5 and a few others, some over range.
|
||||
let px: Vec<[f32; 4]> = (0..64 * 64)
|
||||
.map(|i| {
|
||||
let (x, y) = (i % 64, i / 64);
|
||||
let a = [1.0, 0.5, 0.25, 0.8, 1.0, 1.0, 0.0, 1.0][y % 8];
|
||||
// Row 5: negative values too (32 bpc frames can hold them).
|
||||
let lo = if y % 8 == 5 { 0.75 } else { 0.0 };
|
||||
let v = |k: usize| ((x * 7 + y * k) % 64) as f32 / 32.0 - lo;
|
||||
[v(3) * a, v(5) * a, v(11) * a, a]
|
||||
})
|
||||
.collect();
|
||||
let dcs = [
|
||||
None,
|
||||
DisplayColor::new(
|
||||
Some(effectcraft_engine::project::ColorSpace::Srgb),
|
||||
false,
|
||||
effectcraft_engine::project::ColorSpace::Srgb,
|
||||
true,
|
||||
effectcraft_engine::project::ColorSpace::DisplayP3,
|
||||
Simulation::default(),
|
||||
&Default::default(),
|
||||
),
|
||||
DisplayColor::new(
|
||||
Some(effectcraft_engine::project::ColorSpace::Srgb),
|
||||
false,
|
||||
effectcraft_engine::project::ColorSpace::Srgb,
|
||||
true,
|
||||
effectcraft_engine::project::ColorSpace::Srgb,
|
||||
Simulation { profile: SimProfile::Rec709, preserve_rgb: false },
|
||||
&Default::default(),
|
||||
),
|
||||
DisplayColor::new(
|
||||
Some(effectcraft_engine::project::ColorSpace::Srgb),
|
||||
false,
|
||||
effectcraft_engine::project::ColorSpace::Srgb,
|
||||
true,
|
||||
effectcraft_engine::project::ColorSpace::DisplayP3,
|
||||
Simulation { profile: SimProfile::Mac18, preserve_rgb: true },
|
||||
&Default::default(),
|
||||
),
|
||||
// A LUT-based My Custom RGB profile.
|
||||
Some(DisplayColor::from_lut(test_lut(1.0))),
|
||||
];
|
||||
let (mut report, mut off, mut total) = (vec![], 0usize, 0usize);
|
||||
for curve in [PassCurve::Srgb, PassCurve::Gamma(2.4), PassCurve::Linear, PassCurve::Pq, PassCurve::Hlg] {
|
||||
for hdr in [HdrMode::Clip, HdrMode::Compand, HdrMode::ToneMap] {
|
||||
let enc = FrameEncoding { curve, hdr };
|
||||
let f = frame(&px, 64, enc);
|
||||
for stops in [0.0f32, 1.0, -2.0, 2.5] {
|
||||
for (ci, dc) in dcs.iter().enumerate() {
|
||||
for (channel, colorized) in
|
||||
[(Channel::Rgb, false), (Channel::RgbStraight, false), (Channel::Red, false), (Channel::Green, true), (Channel::Alpha, false)]
|
||||
{
|
||||
if ci > 0 && !matches!(channel, Channel::Rgb) {
|
||||
continue;
|
||||
}
|
||||
let pass = DisplayPass::new(channel, colorized, stops, enc, dc.as_ref());
|
||||
let (got, _) = bake(&d, &gpu, &Source::F16(f.clone()), &pass);
|
||||
let (n, max) = compare(&got, &f, &pass);
|
||||
// The device's pow / exp / ln / division are not the CPU's to the last
|
||||
// bit: a value on a rounding tie can land either side of it.
|
||||
// PQ's exponent of 78.8 magnifies those last bits most.
|
||||
// Software rasterizers (CI's lavapipe) differ more.
|
||||
let share = if matches!(curve, PassCurve::Pq | PassCurve::Hlg) { 5 } else { 1 } * if d.cpu { 5 } else { 1 };
|
||||
assert!(max <= 1 && n * 100 <= got.len() * 4 * share, "{curve:?} {hdr:?} {stops} EV {channel:?} dc {ci}: {n} bytes off, max {max}");
|
||||
off += n;
|
||||
total += got.len() * 4;
|
||||
if n > 0 {
|
||||
report.push(format!("{curve:?} {hdr:?} {stops} {channel:?} {ci}: {n}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
eprintln!("{off} of {total} bytes 1 LSB off the reference: {report:?}");
|
||||
}
|
||||
|
||||
/// 8-bit frames go through the same pass (exposure before the display colour there too), equal
|
||||
/// to the CPU pass on their bytes.
|
||||
#[test]
|
||||
fn rgba8_frames_match_the_cpu_pass() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let px: Vec<egui::Color32> = (0..256 * 16)
|
||||
.map(|i| {
|
||||
let (x, y) = (i % 256, i / 256);
|
||||
let a = [255u8, 128, 64, 200][y % 4];
|
||||
let c = |k: usize| ((x * k + y) % 256) as u32 * a as u32 / 255;
|
||||
egui::Color32::from_rgba_premultiplied(c(1) as u8, c(3) as u8, c(7) as u8, a)
|
||||
})
|
||||
.collect();
|
||||
let img = Arc::new(egui::ColorImage::new([256, 16], px));
|
||||
let dc = DisplayColor::new(
|
||||
Some(effectcraft_engine::project::ColorSpace::Srgb),
|
||||
false,
|
||||
effectcraft_engine::project::ColorSpace::Srgb,
|
||||
true,
|
||||
effectcraft_engine::project::ColorSpace::DisplayP3,
|
||||
Simulation::default(),
|
||||
&Default::default(),
|
||||
);
|
||||
for stops in [0.0f32, 1.0, -1.5] {
|
||||
for dc in [None, dc.as_ref()] {
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, stops, SRGB, dc);
|
||||
let (got, _) = bake(&d, &gpu, &Source::Rgba8(img.clone()), &pass);
|
||||
let mut want: Vec<[u8; 4]> = img.pixels.iter().map(|c| c.to_array()).collect();
|
||||
pass.apply_rgba8(&mut want);
|
||||
let off = got.iter().zip(&want).flat_map(|(g, w)| (0..4).map(move |c| (g[c] as i32 - w[c] as i32).abs())).filter(|d| *d > 0).collect::<Vec<_>>();
|
||||
eprintln!("8-bit, {stops} EV, dc {}: {} bytes 1 LSB off", dc.is_some(), off.len());
|
||||
assert!(off.iter().all(|d| *d <= 1) && off.len() * 100 <= got.len() * 4, "{stops} EV, dc {}: {off:?}", dc.is_some());
|
||||
if stops == 0.0 && dc.is_none() {
|
||||
assert_eq!(got, want, "nothing to change: the frame's bytes");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The mips: each texel the rounded mean of the 2 × 2 texels above it, and at the edge of an
|
||||
/// odd-sized level above also of its last column / row, so no frame pixel is dropped.
|
||||
#[test]
|
||||
fn mips_average_the_level_above() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let px: Vec<[f32; 4]> = (0..37 * 21).map(|i| [(i % 37) as f32 / 36.0, (i / 37) as f32 / 20.0, 0.3, 1.0]).collect();
|
||||
let f = frame(&px, 37, SRGB);
|
||||
let (_, tex) = bake(&d, &gpu, &Source::F16(f), &DisplayPass::new(Channel::Rgb, false, 0.5, SRGB, None));
|
||||
assert_eq!(tex.mip_level_count(), 6);
|
||||
for l in 1..tex.mip_level_count() {
|
||||
let (above, [aw, ah]) = read(&d, &tex, l - 1);
|
||||
let (got, [w, h]) = read(&d, &tex, l);
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
let (x0, y0) = ((x * 2).min(aw - 1), (y * 2).min(ah - 1));
|
||||
let x1 = if x == w - 1 { aw } else { (x * 2 + 2).min(aw) };
|
||||
let y1 = if y == h - 1 { ah } else { (y * 2 + 2).min(ah) };
|
||||
let n = ((x1 - x0) * (y1 - y0)) as f32;
|
||||
let want: [u8; 4] = std::array::from_fn(|c| {
|
||||
let s: f32 =
|
||||
(y0..y1).flat_map(|yy| (x0..x1).map(move |xx| (xx, yy))).map(|(xx, yy)| above[(yy * aw + xx) as usize][c] as f32 / 255.0).sum();
|
||||
(s / n * 255.0 + 0.5).floor() as u8
|
||||
});
|
||||
let g = got[(y * w + x) as usize];
|
||||
assert!((0..4).all(|c| (g[c] as i32 - want[c] as i32).abs() <= 1), "level {l} ({x}, {y}): {g:?} vs {want:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One egui frame through egui_wgpu's renderer into an offscreen target (eframe minus the
|
||||
/// swapchain): RGBA8 of the framebuffer.
|
||||
fn render_egui(
|
||||
d: &Dev,
|
||||
renderer: &mut egui_wgpu::Renderer,
|
||||
ctx: &egui::Context,
|
||||
size_pts: [f32; 2],
|
||||
ppp: f32,
|
||||
ui_fn: &mut dyn FnMut(&mut egui::Ui),
|
||||
) -> (Vec<[u8; 4]>, [u32; 2]) {
|
||||
ctx.set_pixels_per_point(ppp);
|
||||
let raw = egui::RawInput { screen_rect: Some(egui::Rect::from_min_size(egui::Pos2::ZERO, egui::vec2(size_pts[0], size_pts[1]))), ..Default::default() };
|
||||
let mut first = ctx.run_ui(raw.clone(), |ui| ui_fn(ui));
|
||||
let mut out = ctx.run_ui(raw, |ui| ui_fn(ui));
|
||||
for o in [&mut first, &mut out] {
|
||||
for (id, delta) in o.textures_delta.set.iter().flat_map(|(id, v)| v.iter().map(move |d| (id, d))) {
|
||||
renderer.update_texture(&d.device, &d.queue, *id, delta);
|
||||
}
|
||||
for id in &o.textures_delta.free {
|
||||
renderer.free_texture(id);
|
||||
}
|
||||
o.textures_delta.clear();
|
||||
}
|
||||
let jobs = ctx.tessellate(out.shapes, out.pixels_per_point);
|
||||
let size = [(size_pts[0] * ppp).round() as u32, (size_pts[1] * ppp).round() as u32];
|
||||
let sd = egui_wgpu::ScreenDescriptor { size_in_pixels: size, pixels_per_point: out.pixels_per_point };
|
||||
let target = d.device.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some("framebuffer"),
|
||||
size: wgpu::Extent3d { width: size[0], height: size[1], depth_or_array_layers: 1 },
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D2,
|
||||
format: wgpu::TextureFormat::Rgba8Unorm,
|
||||
usage: wgpu::TextureUsages::RENDER_ATTACHMENT | wgpu::TextureUsages::COPY_SRC,
|
||||
view_formats: &[],
|
||||
});
|
||||
let view = target.create_view(&Default::default());
|
||||
let mut enc = d.device.create_command_encoder(&Default::default());
|
||||
let user = renderer.update_buffers(&d.device, &d.queue, &mut enc, &jobs, &sd);
|
||||
{
|
||||
let pass = enc.begin_render_pass(&wgpu::RenderPassDescriptor {
|
||||
label: Some("egui"),
|
||||
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
|
||||
view: &view,
|
||||
depth_slice: None,
|
||||
resolve_target: None,
|
||||
ops: wgpu::Operations { load: wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT), store: wgpu::StoreOp::Store },
|
||||
})],
|
||||
..Default::default()
|
||||
});
|
||||
let mut pass = pass.forget_lifetime();
|
||||
renderer.render(&mut pass, &jobs, &sd);
|
||||
}
|
||||
d.queue.submit(user.into_iter().chain([enc.finish()]));
|
||||
(read(d, &target, 0).0, size)
|
||||
}
|
||||
|
||||
/// The displayed texture drawn by egui as the viewer draws it (native texture, the Faster
|
||||
/// sampler: nearest): at 2 pixels per point, zoomed 8× and 13.37× with the frame past every
|
||||
/// edge of a clip rect, each screen pixel shows the texel under its centre; and over egui's
|
||||
/// checkerboard, half-transparent pixels blend premultiplied.
|
||||
#[test]
|
||||
fn the_displayed_texture_lands_where_the_viewer_draws_it() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let mut renderer =
|
||||
egui_wgpu::Renderer::new(&d.device, wgpu::TextureFormat::Rgba8Unorm, egui_wgpu::RendererOptions { dithering: false, ..Default::default() });
|
||||
let ctx = egui::Context::default();
|
||||
let (tw, th) = (64u32, 32u32);
|
||||
let px: Vec<[f32; 4]> = (0..tw * th).map(|i| [(i % tw) as f32 / 63.0, (i / tw) as f32 / 31.0, 1.0, 1.0]).collect();
|
||||
let f = frame(&px, tw, SRGB);
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, -1.0, SRGB, None);
|
||||
let (texels, tex) = bake(&d, &gpu, &Source::F16(f), &pass);
|
||||
let id =
|
||||
renderer.register_native_texture_with_sampler_options(&d.device, &tex.create_view(&Default::default()), crate::panels::viewer::zoom_sampler(false));
|
||||
let ppp = 2.0f32;
|
||||
let clip = egui::Rect::from_min_max(egui::pos2(50.0, 40.0), egui::pos2(350.0, 260.0));
|
||||
for (zoom, origin) in [(2.0f32, egui::pos2(80.0, 60.0)), (8.0, egui::pos2(-100.0, -60.0)), (13.37, egui::pos2(-151.3, -77.7))] {
|
||||
let rect = egui::Rect::from_min_size(origin, egui::vec2(tw as f32 * zoom, th as f32 * zoom));
|
||||
let mut ui_fn = |ui: &mut egui::Ui| {
|
||||
let p = ui.painter().with_clip_rect(clip);
|
||||
p.image(id, rect, egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)), egui::Color32::WHITE);
|
||||
};
|
||||
let (out, sz) = render_egui(&d, &mut renderer, &ctx, [400.0, 300.0], ppp, &mut ui_fn);
|
||||
let (mut bad, mut drawn) = (0, 0);
|
||||
for y in 0..sz[1] {
|
||||
for x in 0..sz[0] {
|
||||
let (cx, cy) = ((x as f32 + 0.5) / ppp, (y as f32 + 0.5) / ppp);
|
||||
let (u, v) = ((cx - rect.min.x) / zoom, (cy - rect.min.y) / zoom);
|
||||
// Centres within a hair of a texel or clip edge can go either way.
|
||||
let near = |t: f32| (t - t.round()).abs() < 0.01;
|
||||
if near(u)
|
||||
|| near(v)
|
||||
|| near(cx * ppp - clip.min.x * ppp)
|
||||
|| !clip.contains(egui::pos2(cx, cy))
|
||||
|| u < 0.0
|
||||
|| v < 0.0
|
||||
|| u >= tw as f32
|
||||
|| v >= th as f32
|
||||
{
|
||||
continue;
|
||||
}
|
||||
drawn += 1;
|
||||
let want = texels[(v as u32 * tw + u as u32) as usize];
|
||||
let got = out[(y * sz[0] + x) as usize];
|
||||
if (0..4).any(|c| (got[c] as i32 - want[c] as i32).abs() > 1) {
|
||||
bad += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert!(drawn > 1000, "{drawn}");
|
||||
assert_eq!(bad, 0, "zoom {zoom}, origin {origin:?}: {bad} of {drawn} pixels wrong");
|
||||
}
|
||||
renderer.free_texture(&id);
|
||||
// Alpha 0.5 over a checkerboard.
|
||||
let src = [0.5f32, 0.25, 0.0, 0.5];
|
||||
let f = frame(&[src; 256], 16, SRGB);
|
||||
let (texels, tex) = bake(&d, &gpu, &Source::F16(f), &DisplayPass::new(Channel::Rgb, false, 0.0, SRGB, None));
|
||||
let id =
|
||||
renderer.register_native_texture_with_sampler_options(&d.device, &tex.create_view(&Default::default()), crate::panels::viewer::zoom_sampler(false));
|
||||
let (c1, c2) = (egui::Color32::from_gray(0x66), egui::Color32::from_gray(0x99));
|
||||
let mut ui_fn = |ui: &mut egui::Ui| {
|
||||
let p = ui.painter();
|
||||
for j in 0..8 {
|
||||
for i in 0..8 {
|
||||
let r = egui::Rect::from_min_size(egui::pos2(i as f32 * 8.0, j as f32 * 8.0), egui::vec2(8.0, 8.0));
|
||||
p.rect_filled(r, 0.0, if (i + j) % 2 == 0 { c1 } else { c2 });
|
||||
}
|
||||
}
|
||||
p.image(
|
||||
id,
|
||||
egui::Rect::from_min_size(egui::Pos2::ZERO, egui::vec2(64.0, 64.0)),
|
||||
egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)),
|
||||
egui::Color32::WHITE,
|
||||
);
|
||||
};
|
||||
let (out, sz) = render_egui(&d, &mut renderer, &ctx, [64.0, 64.0], 2.0, &mut ui_fn);
|
||||
let s = texels[0];
|
||||
let mut bad = 0;
|
||||
for y in 0..sz[1] {
|
||||
for x in 0..sz[0] {
|
||||
let (i, j) = (x / 16, y / 16);
|
||||
let dst = if (i + j) % 2 == 0 { 0x66 } else { 0x99 } as f32;
|
||||
let want: [u8; 3] = std::array::from_fn(|c| (s[c] as f32 + dst * (1.0 - s[3] as f32 / 255.0)).round() as u8);
|
||||
let got = out[(y * sz[0] + x) as usize];
|
||||
if (0..3).any(|c| (got[c] as i32 - want[c] as i32).abs() > 1) {
|
||||
bad += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(bad, 0, "alpha over the checkerboard: {bad} pixels off");
|
||||
}
|
||||
|
||||
/// Timings at 4K (`cargo test -- --ignored display_pass_timings --nocapture`): the f32 → half
|
||||
/// conversion of a frame, today's f32 → RGBA8 conversion, and the display pass.
|
||||
#[test]
|
||||
#[ignore = "timing, not a check"]
|
||||
fn display_pass_timings() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let img = Image::filled(3840, 2160, [0.7, 1.9, 0.2, 1.0]);
|
||||
let time = |f: &mut dyn FnMut()| {
|
||||
let mut v: Vec<f64> = (0..9)
|
||||
.map(|_| {
|
||||
let t = std::time::Instant::now();
|
||||
f();
|
||||
t.elapsed().as_secs_f64() * 1000.0
|
||||
})
|
||||
.collect();
|
||||
v.sort_by(f64::total_cmp);
|
||||
v[4]
|
||||
};
|
||||
let f16 = time(&mut || drop(F16Frame::from_image(&img, SRGB)));
|
||||
let rgba8 = time(&mut || drop(crate::frames::to_color_image(&img)));
|
||||
eprintln!("4K: f32 → f16 {f16:.2} ms, f32 → RGBA8 {rgba8:.2} ms (medians of 9)");
|
||||
let Some(d) = device() else { return };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let f = Arc::new(F16Frame::from_image(&img, SRGB));
|
||||
let input = gpu.upload(&d.device, &d.queue, &Source::F16(f), 1, None);
|
||||
let out = DisplayGpu::output(&d.device, 3840, 2160);
|
||||
let params = super::uniform(&DisplayPass::new(Channel::Rgb, false, -1.0, FrameEncoding { curve: PassCurve::Srgb, hdr: HdrMode::ToneMap }, None));
|
||||
let pass = time(&mut || {
|
||||
gpu.run(&d.device, &d.queue, &input, &out, ¶ms, None);
|
||||
d.device.poll(wgpu::PollType::wait_indefinitely()).expect("poll");
|
||||
});
|
||||
let f = Arc::new(F16Frame::from_image(&img, SRGB));
|
||||
let mut kept = Some(gpu.upload(&d.device, &d.queue, &Source::F16(f.clone()), 1, None));
|
||||
let upload = time(&mut || {
|
||||
kept = Some(gpu.upload(&d.device, &d.queue, &Source::F16(f.clone()), 1, kept.take()));
|
||||
d.device.poll(wgpu::PollType::wait_indefinitely()).expect("poll");
|
||||
});
|
||||
let cpu = time(&mut || drop(f.to_color_image(&DisplayPass::new(Channel::Rgb, false, -1.0, SRGB, None))));
|
||||
eprintln!("4K: half-float upload into the kept texture {upload:.2} ms; the pass on the CPU (no device) {cpu:.2} ms");
|
||||
eprintln!("4K: display pass with mips, submit to idle {pass:.2} ms");
|
||||
}
|
||||
|
||||
/// `Bake::show` on egui-wgpu's render state: it runs the pass again when the frame, the pass's
|
||||
/// settings or the LUT change (a new LUT of the same size included: another ICC profile,
|
||||
/// Preserve RGB, another display), and only then; the texture id stays.
|
||||
#[test]
|
||||
fn bakes_run_again_when_frame_pass_or_lut_change() {
|
||||
let _g = DEVICE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
let Some(d) = device() else { return eprintln!("no wgpu adapter: skipping") };
|
||||
let gpu = DisplayGpu::new(&d.device);
|
||||
let rs = render_state(&d);
|
||||
let px: Vec<[f32; 4]> = (0..16).map(|i| [i as f32 / 15.0, 0.4, 0.7, 1.0]).collect();
|
||||
let f = frame(&px, 4, SRGB);
|
||||
let lut = |k: f64| DisplayColor::from_lut(test_lut(k));
|
||||
let (l1, l2) = (lut(1.0), lut(0.5));
|
||||
let pass = |stops: f32, dc: &DisplayColor| DisplayPass::new(Channel::Rgb, false, stops, SRGB, Some(dc));
|
||||
let mut b = super::Bake::default();
|
||||
let id = b.show(&rs, &gpu, &Source::F16(f.clone()), &pass(0.0, &l1), true).unwrap();
|
||||
let first = read(&d, b.texture().unwrap(), 0).0;
|
||||
assert_eq!(b.runs(), 1);
|
||||
b.show(&rs, &gpu, &Source::F16(f.clone()), &pass(0.0, &l1), true);
|
||||
assert_eq!(b.runs(), 1, "nothing changed");
|
||||
assert_eq!(b.show(&rs, &gpu, &Source::F16(f.clone()), &pass(0.0, &l2), true), Some(id));
|
||||
assert_eq!(b.runs(), 2, "a new LUT of the same size");
|
||||
let second = read(&d, b.texture().unwrap(), 0).0;
|
||||
assert_ne!(first, second);
|
||||
let want: Vec<[u8; 4]> = (0..16).map(|i| pass(0.0, &l2).pixel(f.pixel(i).unwrap())).collect();
|
||||
assert!(second.iter().zip(&want).all(|(g, w)| (0..4).all(|c| g[c].abs_diff(w[c]) <= 1)), "{second:?} vs {want:?}");
|
||||
b.show(&rs, &gpu, &Source::F16(f.clone()), &pass(-1.0, &l2), true);
|
||||
assert_eq!(b.runs(), 3, "another exposure");
|
||||
b.show(&rs, &gpu, &Source::F16(frame(&px, 4, SRGB)), &pass(-1.0, &l2), true);
|
||||
assert_eq!(b.runs(), 4, "another frame");
|
||||
b.free(Some(&rs));
|
||||
assert!(b.id().is_none());
|
||||
}
|
||||
@@ -23,6 +23,10 @@
|
||||
//! to the GPU) frames stay on the GPU: the compositor writes an RGBA8 texture that the viewer
|
||||
//! registers with egui-wgpu and draws directly; pixels are read back only when something needs
|
||||
//! them (Info panel, eyedroppers, histograms).
|
||||
//!
|
||||
//! 32 bpc projects keep half-float frames on the desktop ([`F16Frame`], rendered without the HDR
|
||||
//! step): twice the bytes of an 8-bit frame, so the RAM preview holds half as many. The disk
|
||||
//! cache keeps them as half-float entries.
|
||||
|
||||
use std::collections::{BTreeSet, HashMap, HashSet, VecDeque};
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
@@ -47,10 +51,14 @@ const MIN_GPU_BUDGET: usize = 64 << 20;
|
||||
const GPU_PRESSURE_COOLDOWN: std::time::Duration = std::time::Duration::from_secs(5);
|
||||
const GPU_RECOVERY_INTERVAL: std::time::Duration = std::time::Duration::from_secs(30);
|
||||
|
||||
/// A rendered viewer frame: CPU pixels or a GPU texture.
|
||||
mod f16;
|
||||
pub use f16::{F16_PIXEL_BYTES, F16Frame};
|
||||
|
||||
/// A rendered viewer frame: 8-bit CPU pixels, half-float CPU pixels (32 bpc) or a GPU texture.
|
||||
#[derive(Clone)]
|
||||
pub enum FrameImage {
|
||||
Cpu(Arc<egui::ColorImage>),
|
||||
CpuF16(Arc<F16Frame>),
|
||||
Gpu(Arc<DisplayFrame>),
|
||||
}
|
||||
|
||||
@@ -58,11 +66,14 @@ impl FrameImage {
|
||||
pub fn size(&self) -> [usize; 2] {
|
||||
match self {
|
||||
FrameImage::Cpu(c) => c.size,
|
||||
FrameImage::CpuF16(f) => f.size(),
|
||||
FrameImage::Gpu(f) => [f.width as usize, f.height as usize],
|
||||
}
|
||||
}
|
||||
fn bytes(&self) -> usize {
|
||||
self.size()[0] * self.size()[1] * 4
|
||||
/// Bytes the frame holds in the RAM preview (GPU frames: without their mips).
|
||||
pub fn bytes(&self) -> usize {
|
||||
let [w, h] = self.size();
|
||||
w * h * if matches!(self, FrameImage::CpuF16(_)) { F16_PIXEL_BYTES } else { 4 }
|
||||
}
|
||||
fn is_gpu(&self) -> bool {
|
||||
matches!(self, FrameImage::Gpu(_))
|
||||
@@ -615,6 +626,29 @@ impl Default for Frames {
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether viewer frames of a project with `settings` are half floats ([`F16Frame`], rendered
|
||||
/// with `RenderOpts::skip_hdr_step`): 32 bpc on the desktop. The browser's frames (its frame
|
||||
/// worker's RGBA8, the GPU compositor's display textures) stay 8-bit for now.
|
||||
pub fn f16_frames(settings: &effectcraft_engine::project::ProjectSettings) -> bool {
|
||||
cfg!(not(target_arch = "wasm32")) && settings.bit_depth.levels().is_none()
|
||||
}
|
||||
|
||||
/// Bytes per pixel of the viewer frames of a project with `settings` (see [`f16_frames`]).
|
||||
pub fn frame_pixel_bytes(settings: &effectcraft_engine::project::ProjectSettings) -> usize {
|
||||
if f16_frames(settings) { F16_PIXEL_BYTES } else { 4 }
|
||||
}
|
||||
|
||||
/// A rendered frame as a cached viewer frame: half floats for `skip_hdr_step` renders (see
|
||||
/// [`f16_frames`]), else 8-bit.
|
||||
pub fn cpu_frame(img: &effectcraft_engine::render::Image, project: &Project, opts: &RenderOpts) -> FrameImage {
|
||||
if opts.skip_hdr_step {
|
||||
let encoding = effectcraft_engine::viewer::FrameEncoding::of(&project.settings, true);
|
||||
FrameImage::CpuF16(Arc::new(F16Frame::from_image(img, encoding)))
|
||||
} else {
|
||||
FrameImage::Cpu(Arc::new(to_color_image(img)))
|
||||
}
|
||||
}
|
||||
|
||||
/// Premultiplied f32 → egui premultiplied Color32 (alpha kept, so the viewer can show the
|
||||
/// transparency grid or the comp background colour underneath).
|
||||
pub fn to_color_image(img: &effectcraft_engine::render::Image) -> egui::ColorImage {
|
||||
@@ -974,7 +1008,7 @@ impl Frames {
|
||||
self.request_content_key(src.project.clone(), base.revision, base.comp);
|
||||
return true;
|
||||
};
|
||||
let (missing, converting): (Vec<(u128, Arc<egui::ColorImage>)>, bool) = {
|
||||
let (missing, converting): (Vec<(u128, FrameImage)>, bool) = {
|
||||
let (Ok(c), Ok(mut queued), Ok(mut written)) = (self.cache.lock(), self.persisted.lock(), self.written.lock()) else { return false };
|
||||
if queued.len() > 100_000 {
|
||||
queued.clear();
|
||||
@@ -988,7 +1022,10 @@ impl Frames {
|
||||
.iter()
|
||||
.filter(|(k, _)| k.same_series(base))
|
||||
.filter_map(|(k, img)| {
|
||||
let FrameImage::Cpu(img) = img else { return None };
|
||||
// (GPU frames have no CPU pixels to write)
|
||||
if img.is_gpu() {
|
||||
return None;
|
||||
}
|
||||
let dk = disk_cache::frame_key(content, k.frame, k.scale, k.view ^ oh);
|
||||
let tries = written.1.get(&dk).copied().unwrap_or(0);
|
||||
(tries < 2 && !dc.contains(disk_cache::Kind::Frame, dk) && !dc.frame_pending(dk) && queued.insert(dk)).then(|| (dk, img.clone()))
|
||||
@@ -1002,12 +1039,17 @@ impl Frames {
|
||||
let (persisted, written) = (self.persisted.clone(), self.written.clone());
|
||||
let work = move || {
|
||||
for (dk, img) in missing {
|
||||
let rgba: Vec<u8> = img.pixels.iter().flat_map(|c| c.to_array()).collect();
|
||||
// (accepted: the disk cache tracks it until written, and it counts as a try;
|
||||
// refused by a full queue: retried, without counting)
|
||||
if dc.put_frame(dk, img.size[0] as u32, img.size[1] as u32, &rgba)
|
||||
&& let Ok(mut w) = written.lock()
|
||||
{
|
||||
let accepted = match &img {
|
||||
FrameImage::Cpu(img) => {
|
||||
let rgba: Vec<u8> = img.pixels.iter().flat_map(|c| c.to_array()).collect();
|
||||
dc.put_frame(dk, img.size[0] as u32, img.size[1] as u32, &rgba)
|
||||
}
|
||||
FrameImage::CpuF16(f) => dc.put_frame_f16(dk, f.width, f.height, f.bits()),
|
||||
FrameImage::Gpu(_) => false,
|
||||
};
|
||||
if accepted && let Ok(mut w) = written.lock() {
|
||||
let n = w.1.entry(dk).or_default();
|
||||
*n = n.saturating_add(1);
|
||||
}
|
||||
@@ -1563,10 +1605,20 @@ impl Worker {
|
||||
.disk
|
||||
.as_ref()
|
||||
.map(|dc| (dc.clone(), frame_disk_key(content_keys, &job.src.project, &job.key, opts_hash(&job.opts), epoch, self.virtual_id.as_deref())));
|
||||
let from_disk = disk.as_ref().and_then(|(dc, k)| dc.get_frame(*k));
|
||||
// Half-float frames (32 bpc) have entries of their own (their key has the options).
|
||||
let from_disk = disk.as_ref().and_then(|(dc, k)| {
|
||||
if job.opts.skip_hdr_step {
|
||||
let f = dc.get_frame_f16(*k)?;
|
||||
let encoding = effectcraft_engine::viewer::FrameEncoding::of(&job.src.project.settings, true);
|
||||
Some(FrameImage::CpuF16(Arc::new(F16Frame::from_bits(f.width, f.height, f.rgba, encoding))))
|
||||
} else {
|
||||
let f = dc.get_frame(*k)?;
|
||||
Some(FrameImage::Cpu(Arc::new(egui::ColorImage::from_rgba_premultiplied([f.width as usize, f.height as usize], &f.rgba))))
|
||||
}
|
||||
});
|
||||
let disk_hit = from_disk.is_some();
|
||||
let ci = match from_disk {
|
||||
Some(f) => FrameImage::Cpu(Arc::new(egui::ColorImage::from_rgba_premultiplied([f.width as usize, f.height as usize], &f.rgba))),
|
||||
Some(f) => f,
|
||||
None => match self.render_guarded(&job).filter(|_| !job.cancel.load(Ordering::Relaxed)) {
|
||||
Some(ci) => ci,
|
||||
// Called off (frames it read may be missing) or failed: not cached, and not
|
||||
@@ -1580,10 +1632,20 @@ impl Worker {
|
||||
// Publish the shared frame and wake the viewer before optional RGBA conversion
|
||||
// and compression. Persistence still runs on this worker, using the same snapshot.
|
||||
self.finish(&job, ci.clone(), t0);
|
||||
if !disk_hit && let (Some((dc, k)), FrameImage::Cpu(img)) = (&disk, &ci) {
|
||||
let rgba: Vec<u8> = img.pixels.iter().flat_map(|c| c.to_array()).collect();
|
||||
dc.put_frame(*k, img.size[0] as u32, img.size[1] as u32, &rgba);
|
||||
if let Some(ctx) = &self.ctx {
|
||||
if !disk_hit && let Some((dc, k)) = &disk {
|
||||
let written = match &ci {
|
||||
FrameImage::Cpu(img) => {
|
||||
let rgba: Vec<u8> = img.pixels.iter().flat_map(|c| c.to_array()).collect();
|
||||
dc.put_frame(*k, img.size[0] as u32, img.size[1] as u32, &rgba);
|
||||
true
|
||||
}
|
||||
FrameImage::CpuF16(f) => {
|
||||
dc.put_frame_f16(*k, f.width, f.height, f.bits());
|
||||
true
|
||||
}
|
||||
FrameImage::Gpu(_) => false,
|
||||
};
|
||||
if written && let Some(ctx) = &self.ctx {
|
||||
ctx.request_repaint();
|
||||
}
|
||||
}
|
||||
@@ -1745,18 +1807,18 @@ impl Worker {
|
||||
}
|
||||
}
|
||||
let t0 = web_time::Instant::now();
|
||||
let img = to_color_image(&r.comp_frame_cpu(job.comp, job.t));
|
||||
let img = cpu_frame(&r.comp_frame_cpu(job.comp, job.t), &job.src.project, &job.opts);
|
||||
if let Some((p, key)) = auto {
|
||||
p.record(key, false, t0.elapsed().as_secs_f64() * 1000.0);
|
||||
}
|
||||
return FrameImage::Cpu(Arc::new(img));
|
||||
return img;
|
||||
}
|
||||
if let Some(g) = gpu {
|
||||
// The GPU composites and the frame is read back.
|
||||
return match g.within_memory(|| r.comp_frame(job.comp, job.t)) {
|
||||
Ok(img) => {
|
||||
self.gpu_succeeded();
|
||||
FrameImage::Cpu(Arc::new(to_color_image(&img)))
|
||||
cpu_frame(&img, &job.src.project, &job.opts)
|
||||
}
|
||||
Err(e) => {
|
||||
self.gpu_attempt_failed(g, &e);
|
||||
@@ -1764,7 +1826,7 @@ impl Worker {
|
||||
}
|
||||
};
|
||||
}
|
||||
FrameImage::Cpu(Arc::new(to_color_image(&r.comp_frame(job.comp, job.t))))
|
||||
cpu_frame(&r.comp_frame(job.comp, job.t), &job.src.project, &job.opts)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1863,4 +1925,4 @@ pub(crate) fn render_scale(thousandths: u32, factor: Option<u8>) -> f64 {
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
pub(crate) mod tests;
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
//! Half-float viewer frames (32 bpc projects on the desktop, #2797): the RAM preview keeps the
|
||||
//! rendered frame's full range, without the HDR step, so viewer exposure, Compand / Tone Map and
|
||||
//! the display colour run on it in the display pass (`crate::display_gpu`), in that order.
|
||||
|
||||
use effectcraft_engine::render::Image;
|
||||
use effectcraft_engine::viewer::{DisplayPass, FrameEncoding};
|
||||
use half::f16;
|
||||
use half::slice::{HalfBitsSliceExt, HalfFloatSliceExt};
|
||||
use rayon::prelude::*;
|
||||
|
||||
/// Values converted per piece: 16 KB of input, so the piece stays in L1 while its non-finite
|
||||
/// values are fixed up.
|
||||
const BLOCK: usize = 4096;
|
||||
|
||||
/// A premultiplied RGBA frame in half floats, encoded in the output space as rendered (values
|
||||
/// above 1.0 kept), with how it is encoded. Never NaN or infinite (see [`F16Frame::from_image`]):
|
||||
/// the display pass relies on it.
|
||||
pub struct F16Frame {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
/// RGBA, row by row, as IEEE binary16 bits.
|
||||
data: Vec<u16>,
|
||||
pub encoding: FrameEncoding,
|
||||
}
|
||||
|
||||
/// Bytes per pixel of a half-float frame.
|
||||
pub const F16_PIXEL_BYTES: usize = 8;
|
||||
|
||||
/// Half floats that are NaN become 0, infinities ±65504 (one pass, no branches on the data).
|
||||
fn finite(bits: &mut [u16]) {
|
||||
for b in bits {
|
||||
let v = *b;
|
||||
let special = v & 0x7c00 == 0x7c00;
|
||||
let fixed = if v & 0x03ff != 0 { 0 } else { (v & 0x8000) | 0x7bff };
|
||||
*b = if special { fixed } else { v };
|
||||
}
|
||||
}
|
||||
|
||||
impl F16Frame {
|
||||
/// A rendered frame in half floats. NaN becomes 0 (NaN alpha is transparent, as in
|
||||
/// `display_pixel`) and values past the half-float range become ±65504.
|
||||
pub fn from_image(img: &Image, encoding: FrameEncoding) -> F16Frame {
|
||||
let src = img.data.as_flattened();
|
||||
// A zeroed allocation: fresh pages from the OS, no fill pass.
|
||||
let mut data = vec![0u16; src.len()];
|
||||
data.par_chunks_mut(BLOCK).zip(src.par_chunks(BLOCK)).for_each(|(out, inp)| {
|
||||
out.reinterpret_cast_mut::<f16>().convert_from_f32_slice(inp);
|
||||
finite(out);
|
||||
});
|
||||
F16Frame { width: img.width, height: img.height, data, encoding }
|
||||
}
|
||||
|
||||
/// A frame from binary16 bits (the disk cache), made finite as [`F16Frame::from_image`] does.
|
||||
pub fn from_bits(width: u32, height: u32, mut data: Vec<u16>, encoding: FrameEncoding) -> F16Frame {
|
||||
data.resize(width as usize * height as usize * 4, 0);
|
||||
data.par_chunks_mut(BLOCK).for_each(finite);
|
||||
F16Frame { width, height, data, encoding }
|
||||
}
|
||||
|
||||
/// The pixels as binary16 bits (RGBA, row by row).
|
||||
pub fn bits(&self) -> &[u16] {
|
||||
&self.data
|
||||
}
|
||||
|
||||
pub fn size(&self) -> [usize; 2] {
|
||||
[self.width as usize, self.height as usize]
|
||||
}
|
||||
|
||||
pub fn bytes(&self) -> usize {
|
||||
self.data.len() * 2
|
||||
}
|
||||
|
||||
/// Pixel `i` (row-major) as `f32`.
|
||||
pub fn pixel(&self, i: usize) -> Option<[f32; 4]> {
|
||||
let p = self.data.get(i.checked_mul(4)?..i * 4 + 4)?;
|
||||
Some([0, 1, 2, 3].map(|k| f16::from_bits(p[k]).to_f32()))
|
||||
}
|
||||
|
||||
/// The frame through `pass` on the CPU (no GPU, or a single sample).
|
||||
pub fn to_color_image(&self, pass: &DisplayPass) -> egui::ColorImage {
|
||||
let px: Vec<egui::Color32> = self
|
||||
.data
|
||||
.par_chunks(4)
|
||||
.map(|p| {
|
||||
let [r, g, b, a] = pass.pixel([0, 1, 2, 3].map(|k| f16::from_bits(p[k]).to_f32()));
|
||||
egui::Color32::from_rgba_premultiplied(r, g, b, a)
|
||||
})
|
||||
.collect();
|
||||
egui::ColorImage::new(self.size(), px)
|
||||
}
|
||||
}
|
||||
@@ -1136,6 +1136,175 @@ fn timing_markers_and_expressions_still_change_the_frames() {
|
||||
assert_ne!(at(&s), o2, "an expression can read the marker, so the frames change");
|
||||
}
|
||||
|
||||
const SRGB: effectcraft_engine::viewer::FrameEncoding =
|
||||
effectcraft_engine::viewer::FrameEncoding { curve: effectcraft_engine::viewer::PassCurve::Srgb, hdr: effectcraft_engine::project::HdrMode::Clip };
|
||||
|
||||
/// Half floats keep over-range values (the point of #2797); NaN becomes 0 and values past the
|
||||
/// half-float range ±65504.
|
||||
#[test]
|
||||
fn half_float_frames_keep_over_range_values() {
|
||||
let mut img = effectcraft_engine::render::Image::new(3, 1);
|
||||
img.data.copy_from_slice(&[[4.0, 16.0, 1.825, 1.0], [70_000.0, f32::NAN, -f32::INFINITY, 0.5], [0.1, -0.25, 1000.0, f32::NAN]]);
|
||||
let f = F16Frame::from_image(&img, SRGB);
|
||||
assert_eq!(f.pixel(0), Some([4.0, 16.0, half::f16::from_f32(1.825).to_f32(), 1.0]));
|
||||
assert_eq!(f.pixel(1), Some([65504.0, 0.0, -65504.0, 0.5]));
|
||||
let p = f.pixel(2).unwrap();
|
||||
assert_eq!((p[2], p[3]), (1000.0, 0.0));
|
||||
assert!((p[0] - 0.1).abs() < 1e-4 && (p[1] + 0.25).abs() < 1e-4);
|
||||
assert_eq!(f.bytes(), 3 * F16_PIXEL_BYTES);
|
||||
}
|
||||
|
||||
/// Oversized half-float frames take the CPU pass: display conversion, then averaging down. Linear
|
||||
/// 8 and 0 alternating in Clip mode at -2 EV is white and black (byte 128 averaged); averaging the
|
||||
/// scene values first would give 4.0 x 0.25 = 1.0, byte 255.
|
||||
#[test]
|
||||
fn oversized_half_float_frames_convert_before_averaging() {
|
||||
use crate::display_gpu::Source;
|
||||
use effectcraft_engine::viewer::{Channel, DisplayPass};
|
||||
let mut img = effectcraft_engine::render::Image::new(2, 1);
|
||||
img.data.copy_from_slice(&[[8.0, 8.0, 8.0, 1.0], [0.0, 0.0, 0.0, 1.0]]);
|
||||
let lin =
|
||||
effectcraft_engine::viewer::FrameEncoding { curve: effectcraft_engine::viewer::PassCurve::Linear, hdr: effectcraft_engine::project::HdrMode::Clip };
|
||||
let f = Arc::new(F16Frame::from_image(&img, lin));
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, -2.0, lin, None);
|
||||
let src = Source::F16(f.clone());
|
||||
assert!(src.needs_cpu_pass(1) && !src.needs_cpu_pass(2));
|
||||
assert!(!Source::Rgba8(Arc::new(egui::ColorImage::new([2, 1], vec![egui::Color32::WHITE; 2]))).needs_cpu_pass(1), "8-bit frames are averaged on the GPU");
|
||||
let out = fit_texture_by(f.to_color_image(&pass), 1, 1);
|
||||
assert_eq!(out.size, [1, 1]);
|
||||
assert_eq!(out.pixels[0].to_array(), [128, 128, 128, 255]);
|
||||
}
|
||||
|
||||
/// 65 536 colours spread over 0..1 (each channel on its own 16-bit grid), premultiplied by
|
||||
/// `alpha`: the 8-bit parity input.
|
||||
pub(crate) fn parity_frame(alpha: f32) -> effectcraft_engine::render::Image {
|
||||
let mut img = effectcraft_engine::render::Image::new(256, 256);
|
||||
for (i, p) in img.data.iter_mut().enumerate() {
|
||||
let v = |k: usize| ((i * k) % 65_536) as f32 / 65_535.0;
|
||||
*p = [v(1) * alpha, v(7_919) * alpha, v(104_729) * alpha, alpha];
|
||||
}
|
||||
img
|
||||
}
|
||||
|
||||
/// At 0 EV the half-float viewer shows the bytes of the old 8-bit frames for values in 0..1: at
|
||||
/// least 97.9 % of the colour bytes equal (alpha all), never more than 1 apart. The rest is the
|
||||
/// half-float storage.
|
||||
#[test]
|
||||
fn half_float_frames_show_what_8_bit_frames_showed() {
|
||||
use effectcraft_engine::viewer::{Channel, DisplayPass};
|
||||
let pass = DisplayPass::new(Channel::Rgb, false, 0.0, SRGB, None);
|
||||
for alpha in [1.0f32, 0.5] {
|
||||
let img = parity_frame(alpha);
|
||||
let f = F16Frame::from_image(&img, SRGB);
|
||||
let new = f.to_color_image(&pass);
|
||||
let old = to_color_image(&img);
|
||||
let (mut same, mut total, mut max) = (0, 0, 0);
|
||||
for (n, o) in new.pixels.iter().zip(&old.pixels) {
|
||||
let (n, o) = (n.to_array(), o.to_array());
|
||||
assert_eq!(n[3], o[3]);
|
||||
for c in 0..3 {
|
||||
total += 1;
|
||||
same += (n[c] == o[c]) as usize;
|
||||
max = max.max((n[c] as i32 - o[c] as i32).abs());
|
||||
}
|
||||
}
|
||||
let share = same as f64 / total as f64;
|
||||
eprintln!("alpha {alpha}: {:.2} % of colour bytes equal, max {max}", share * 100.0);
|
||||
assert!(share >= 0.979 && max <= 1, "alpha {alpha}: {share} equal, max {max}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The RAM preview counts half-float frames at 8 bytes a pixel: in a budget of three 8-bit
|
||||
/// frames, a half-float frame of the same size takes the room of two.
|
||||
#[test]
|
||||
fn half_float_frames_count_twice_the_bytes() {
|
||||
let f16 = || FrameImage::CpuF16(Arc::new(F16Frame::from_image(&effectcraft_engine::render::Image::new(2, 2), SRGB)));
|
||||
assert_eq!(f16().bytes(), 32);
|
||||
assert_eq!(img().bytes(), 16);
|
||||
let frames = Frames::default();
|
||||
frames.set_budget(48);
|
||||
for i in 0..3 {
|
||||
frames.cache.lock().unwrap().insert(key(1, i, 0), img());
|
||||
}
|
||||
assert_eq!(frames.cached_frames(&key(1, 0, 0)).len(), 3);
|
||||
frames.cache.lock().unwrap().insert(key(2, 0, 0), f16());
|
||||
let c = frames.cache.lock().unwrap();
|
||||
assert!(c.bytes <= 48, "{}", c.bytes);
|
||||
assert_eq!(c.map.len(), 2, "the half-float frame took two 8-bit frames' room");
|
||||
}
|
||||
|
||||
/// 32 bpc viewer frames go to the disk cache as half floats and come back from it bit for bit
|
||||
/// in a new session (over-range values kept); the timeline's disk bar sees them.
|
||||
#[test]
|
||||
fn half_float_frames_survive_in_the_disk_cache() {
|
||||
use effectcraft_engine::Session;
|
||||
use serde_json::json;
|
||||
// Removed on drop, a failed assertion included.
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let disk = DiskCache::open(dir.path(), 1 << 30).unwrap();
|
||||
let mut s = Session::default();
|
||||
let cid = ItemId(s.execute("comp.new", json!({"width": 16, "height": 8, "duration": 1})).unwrap()["comp"].as_u64().unwrap());
|
||||
s.execute("file.projectSettings", json!({"bitDepth": 32, "workingSpace": "srgb", "hdr": "compand"})).unwrap();
|
||||
let lid = s.execute("layer.newSolid", json!({"color": "#ffffff"})).unwrap()["layer"].as_u64().unwrap();
|
||||
s.execute("effect.apply", json!({"layers": [lid], "effect": "Exposure"})).unwrap();
|
||||
s.execute("prop.set", json!({"layer": lid, "path": "effects/#1/master/exposure", "value": 2.0})).unwrap();
|
||||
let src = |disk: &Arc<DiskCache>| RenderSource {
|
||||
project: s.project.clone(),
|
||||
footage: s.footage.clone(),
|
||||
expr: None,
|
||||
layer_cache: Arc::default(),
|
||||
gpu: None,
|
||||
gpu_display: false,
|
||||
disk: Some(disk.clone()),
|
||||
};
|
||||
let opts = RenderOpts { skip_hdr_step: true, ..Default::default() };
|
||||
let shown = |f: &Frames, disk: &Arc<DiskCache>| {
|
||||
let key = FrameKey {
|
||||
revision: s.revision,
|
||||
content: f.content_of(&s.project, s.revision, cid),
|
||||
comp: cid.0,
|
||||
frame: 0,
|
||||
scale: 1000,
|
||||
view: 0,
|
||||
opts: opts_hash(&opts),
|
||||
};
|
||||
f.request_urgent(&src(disk), key, cid, Tick::ZERO, opts);
|
||||
wait_for_renders(f);
|
||||
match f.get(&key) {
|
||||
Some(FrameImage::CpuF16(img)) => (key, img),
|
||||
_ => panic!("a half-float frame"),
|
||||
}
|
||||
};
|
||||
let first = Frames::default();
|
||||
let (key, rendered) = shown(&first, &disk);
|
||||
assert!(rendered.pixel(0).unwrap()[0] > 1.0, "over range, without the HDR step");
|
||||
// The worker releases the frame before it queues the disk write: wait for the entry, not
|
||||
// only for the queue to drain.
|
||||
for _ in 0..400 {
|
||||
disk.flush();
|
||||
if disk.stats().entries >= 1 {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
}
|
||||
assert_eq!(disk.stats().entries, 1, "written as a frame entry");
|
||||
// The timeline's blue bar: on disk, not in this (new) RAM cache.
|
||||
let again = Frames::default();
|
||||
let key2 = FrameKey { content: again.content_of(&s.project, s.revision, cid), ..key };
|
||||
for _ in 0..200 {
|
||||
if !again.disk_frames(&src(&disk), &key2, 0..1, &opts).is_empty() {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
}
|
||||
assert_eq!(again.disk_frames(&src(&disk), &key2, 0..1, &opts), vec![0]);
|
||||
let hits = disk.stats().hits;
|
||||
let (_, read) = shown(&again, &disk);
|
||||
assert_eq!(disk.stats().hits, hits + 1, "read from the disk cache");
|
||||
assert_eq!(read.bits(), rendered.bits());
|
||||
assert_eq!(read.encoding, rendered.encoding);
|
||||
}
|
||||
|
||||
/// The oracle for [`Comp::changed_spans`]: every frame an edit keeps renders the same in the edited
|
||||
/// project. Moves, trims, splits, deletes and duplicates of moving clips, under adjustment
|
||||
/// layers, with seeded noise, motion blur, mattes, parents and effects that read other times.
|
||||
@@ -1415,9 +1584,19 @@ fn disk_keys_survive_save_and_reopen() {
|
||||
/// Roto Brush model stamp in its opts; the disk keys are the worker's, without it.
|
||||
#[test]
|
||||
fn ram_frames_missing_on_disk_are_written() {
|
||||
ram_frames_missing_on_disk_are_written_as(false);
|
||||
}
|
||||
|
||||
/// Same for a 32 bpc project, whose RAM frames are half-float.
|
||||
#[test]
|
||||
fn half_float_ram_frames_missing_on_disk_are_written() {
|
||||
ram_frames_missing_on_disk_are_written_as(true);
|
||||
}
|
||||
|
||||
fn ram_frames_missing_on_disk_are_written_as(half: bool) {
|
||||
use effectcraft_engine::Session;
|
||||
use serde_json::json;
|
||||
let dir = std::env::temp_dir().join(format!("ec-persist-{}", std::process::id()));
|
||||
let dir = std::env::temp_dir().join(format!("ec-persist-{}-{half}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
let disk = DiskCache::open(&dir, 1 << 30).unwrap();
|
||||
let mut s = Session::default();
|
||||
@@ -1436,7 +1615,8 @@ fn ram_frames_missing_on_disk_are_written() {
|
||||
let series =
|
||||
FrameKey { revision: 1, content: f.content_of(&src.project, 1, cid), comp: cid.0, opts: opts_hash(&RenderOpts::default()) ^ 1, ..key(0, 0, 0) };
|
||||
for fr in 0..10 {
|
||||
insert(&f, FrameKey { frame: fr, ..series });
|
||||
let frame = if half { FrameImage::CpuF16(Arc::new(F16Frame::from_image(&effectcraft_engine::render::Image::new(8, 8), SRGB))) } else { img() };
|
||||
f.cache.lock().unwrap().insert(FrameKey { frame: fr, ..series }, frame);
|
||||
}
|
||||
let on_disk = |fr: i64| {
|
||||
disk.contains(
|
||||
|
||||
@@ -24,6 +24,7 @@ pub mod bench;
|
||||
pub mod color_picker;
|
||||
pub mod control;
|
||||
pub mod credits;
|
||||
pub mod display_gpu;
|
||||
pub mod dock;
|
||||
pub mod dock_ui;
|
||||
mod expression_diagnostics;
|
||||
@@ -314,12 +315,28 @@ pub struct EffectcraftApp {
|
||||
/// The last GPU frame shown: its egui texture id (registered with egui-wgpu), key and texture.
|
||||
pub(crate) viewer_native: Option<(egui::TextureId, FrameKey, Arc<effectcraft_gpu::DisplayFrame>)>,
|
||||
/// The texture the viewer draws and its frame key (CPU or GPU frame).
|
||||
pub(crate) viewer_shown: Option<(egui::TextureId, FrameKey)>,
|
||||
/// (`None` texture: a half-float frame, drawn by the display pass.)
|
||||
pub(crate) viewer_shown: Option<(Option<egui::TextureId>, FrameKey)>,
|
||||
/// A Cryptomatte eyedropper probe is being built (one at a time, re-arming included).
|
||||
pub(crate) crypto_probe_busy: Arc<std::sync::atomic::AtomicBool>,
|
||||
/// Last displayed image (Info panel colour, eyedroppers, histograms). GPU frames fill it on
|
||||
/// demand ([`EffectcraftApp::viewer_pixels`]).
|
||||
/// Last displayed image (Info panel colour, eyedroppers, histograms). GPU frames and
|
||||
/// half-float frames fill it on demand ([`EffectcraftApp::viewer_pixels`]).
|
||||
pub(crate) viewer_image: Option<Arc<egui::ColorImage>>,
|
||||
/// The half-float frame on screen (32 bpc projects; its key is `viewer_shown`'s).
|
||||
pub(crate) viewer_f16: Option<Arc<frames::F16Frame>>,
|
||||
/// The window's egui-wgpu device (with or without a GPU compositor on it): the display pass
|
||||
/// runs there.
|
||||
pub(crate) window_wgpu: Option<eframe::egui_wgpu::RenderState>,
|
||||
/// The display pass on that device (made with the first frame that needs it), and whether
|
||||
/// making it failed (the pass then runs on the CPU).
|
||||
pub(crate) display_gpu: Option<Arc<display_gpu::DisplayGpu>>,
|
||||
pub(crate) display_gpu_failed: bool,
|
||||
/// The Composition viewer's and the snapshot's displayed textures (display pass).
|
||||
pub(crate) viewer_bake: display_gpu::Bake,
|
||||
pub(crate) snapshot_bake: display_gpu::Bake,
|
||||
/// The snapshot in half floats for the display pass (the image it was made from, not kept
|
||||
/// alive by this).
|
||||
pub(crate) snapshot_f16: Option<(std::sync::Weak<effectcraft_engine::render::Image>, Arc<frames::F16Frame>)>,
|
||||
/// wasm32: an asynchronous readback of a GPU viewer frame (key asked for, result).
|
||||
viewer_readback: (Option<FrameKey>, Arc<std::sync::Mutex<Option<(FrameKey, Arc<egui::ColorImage>)>>>),
|
||||
/// egui-wgpu's device, once the first frame arrives (desktop and WebGPU).
|
||||
|
||||
@@ -18,16 +18,28 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
let p = ui.painter().with_clip_rect(rect);
|
||||
let x0 = rect.min.x + 12.0;
|
||||
let mut y = rect.min.y + 14.0 - scroll.offset;
|
||||
if app.pointer_comp.is_some() {
|
||||
let comp_size = app.session.active_comp().map_or([1.0, 1.0], |c| [c.width as f32, c.height as f32]);
|
||||
let region = || super::viewer_tools::shown_region(ui.ctx(), comp_size);
|
||||
// 32 bpc (half-float frames): the frame's float values, straight colour as rendered (before
|
||||
// Project Settings ▸ HDR and the viewer's exposure), as After Effects shows them in 32 bpc.
|
||||
let floats = match (app.pointer_comp, app.viewer_f16()) {
|
||||
(Some([cx, cy]), Some(f)) => {
|
||||
let p = super::fx_widgets::frame_pixel_index(f.size(), region(), [cx as f64, cy as f64]).and_then(|i| f.pixel(i));
|
||||
let un = |p: [f32; 4]| {
|
||||
let a = p[3].clamp(0.0, 1.0);
|
||||
if a > 0.0 { [p[0] / a, p[1] / a, p[2] / a, a] } else { [0.0; 4] }
|
||||
};
|
||||
Some((p.map(un), (cx, cy)))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
if app.pointer_comp.is_some() && floats.is_none() {
|
||||
// GPU frames: read the shown frame back for sampling.
|
||||
app.viewer_pixels();
|
||||
}
|
||||
let (rgba, xy) = match (app.pointer_comp, &app.viewer_image) {
|
||||
(Some([cx, cy]), Some(img)) => {
|
||||
let comp = app.session.active_comp().map_or([1.0, 1.0], |c| [c.width as f32, c.height as f32]);
|
||||
let region = super::viewer_tools::shown_region(ui.ctx(), comp);
|
||||
(super::fx_widgets::frame_pixel(img, region, [cx as f64, cy as f64]), Some((cx, cy)))
|
||||
}
|
||||
_ if floats.is_some() => (None, floats.map(|(_, xy)| xy)),
|
||||
(Some([cx, cy]), Some(img)) => (super::fx_widgets::frame_pixel(img, region(), [cx as f64, cy as f64]), Some((cx, cy))),
|
||||
_ => (None, None),
|
||||
};
|
||||
let row = |p: &egui::Painter, y: f32, k: &str, v: String, col: Color32| {
|
||||
@@ -43,20 +55,26 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
[f(c.r()), f(c.g()), f(c.b()), a]
|
||||
}
|
||||
};
|
||||
let px = rgba.map(unpre).unwrap_or([0, 0, 0, 0]);
|
||||
row(&p, y, "R :", format!("{}", px[0]), Color32::from_rgb(0xe0, 0x60, 0x60));
|
||||
row(&p, y + 18.0, "G :", format!("{}", px[1]), Color32::from_rgb(0x60, 0xd0, 0x60));
|
||||
row(&p, y + 36.0, "B :", format!("{}", px[2]), Color32::from_rgb(0x60, 0x90, 0xf0));
|
||||
row(&p, y + 54.0, "A :", format!("{}", px[3]), t.text_dim);
|
||||
let (texts, swatch): ([String; 4], Option<[u8; 4]>) = match floats {
|
||||
Some((p, _)) => (p.unwrap_or([0.0; 4]).map(|v| format!("{v:.4}")), p.map(|p| p.map(|v| (v.clamp(0.0, 1.0) * 255.0 + 0.5) as u8))),
|
||||
None => {
|
||||
let px = rgba.map(unpre).unwrap_or([0, 0, 0, 0]);
|
||||
(px.map(|v| format!("{v}")), rgba.map(|_| px))
|
||||
}
|
||||
};
|
||||
let [tr_, tg, tb, ta] = texts;
|
||||
row(&p, y, "R :", tr_, Color32::from_rgb(0xe0, 0x60, 0x60));
|
||||
row(&p, y + 18.0, "G :", tg, Color32::from_rgb(0x60, 0xd0, 0x60));
|
||||
row(&p, y + 36.0, "B :", tb, Color32::from_rgb(0x60, 0x90, 0xf0));
|
||||
row(&p, y + 54.0, "A :", ta, t.text_dim);
|
||||
let cx = rect.min.x + rect.width() * 0.5;
|
||||
if let Some((x, yy)) = xy {
|
||||
p.text(pos2(cx, y), Align2::LEFT_CENTER, format!("X : {x:.0}"), Tokens::mono(12.0), t.text);
|
||||
p.text(pos2(cx, y + 18.0), Align2::LEFT_CENTER, format!("Y : {yy:.0}"), Tokens::mono(12.0), t.text);
|
||||
}
|
||||
if let Some(c) = rgba {
|
||||
if let Some(px) = swatch {
|
||||
let sw = egui::Rect::from_min_size(pos2(cx, y + 34.0), vec2(36.0, 30.0));
|
||||
p.rect_filled(sw, 3.0, Color32::from_rgb(px[0], px[1], px[2]));
|
||||
let _ = c;
|
||||
}
|
||||
y += 82.0;
|
||||
p.line_segment([pos2(rect.min.x + 8.0, y), pos2(rect.max.x - 8.0, y)], egui::Stroke::new(1.0, t.separator));
|
||||
|
||||
@@ -352,6 +352,7 @@ fn phase_frame(app: &mut EffectcraftApp, v: &mut Vw<'_>) {
|
||||
let stale = app.viewer_shown.as_ref().is_none_or(|(_, s)| *s != k);
|
||||
// A CPU frame goes up again when the magnification calls for another factor.
|
||||
let minify = minify_factor(app.session.prefs.viewer_zoom_smooth(), scale, (zoom * ppp) as f64);
|
||||
// (Half-float frames have no such texture: the display pass makes mips.)
|
||||
let refit = !app.viewer_on_gpu() && app.viewer_tex.as_ref().is_some_and(|(_, _, m)| *m != minify);
|
||||
if stale || refit {
|
||||
show_frame(app, &ctx, k, img, minify);
|
||||
|
||||
@@ -140,7 +140,8 @@ pub(super) fn rect_of(map: &ViewerMap, r: Option<[f64; 4]>) -> Rect {
|
||||
|
||||
/// Put a rendered frame on screen: CPU pixels go into an egui texture, averaged down by `minify`
|
||||
/// (see [`minify_factor`]); GPU frames are drawn straight from their wgpu texture (registered with
|
||||
/// egui-wgpu, no readback).
|
||||
/// egui-wgpu, no readback). Half-float frames (32 bpc) are kept for the display pass, which
|
||||
/// `viewer_tools::draw_frame` runs (exposure, HDR step, display colour).
|
||||
pub(super) fn show_frame(app: &mut EffectcraftApp, ctx: &egui::Context, key: crate::frames::FrameKey, img: crate::frames::FrameImage, minify: usize) {
|
||||
use crate::frames::FrameImage;
|
||||
// Settings ▸ Previews ▸ Viewer Zoom Quality.
|
||||
@@ -158,8 +159,18 @@ pub(super) fn show_frame(app: &mut EffectcraftApp, ctx: &egui::Context, key: cra
|
||||
}
|
||||
_ => app.viewer_tex = Some((ctx.load_texture("viewer-frame", fitted, opts), key, minify)),
|
||||
}
|
||||
app.viewer_shown = app.viewer_tex.as_ref().map(|(t, k, _)| (t.id(), *k));
|
||||
app.viewer_shown = app.viewer_tex.as_ref().map(|(t, k, _)| (Some(t.id()), *k));
|
||||
app.viewer_image = Some(img);
|
||||
app.viewer_f16 = None;
|
||||
vt::set_texture_roi(ctx, app.session.state.region_of_interest.or(app.ui.viewer.extended));
|
||||
}
|
||||
FrameImage::CpuF16(f) => {
|
||||
// Drawn through the display pass; the 8-bit texture of an earlier frame goes.
|
||||
app.viewer_tex = None;
|
||||
// No texture of its own: the display pass draws it.
|
||||
app.viewer_shown = Some((None, key));
|
||||
app.viewer_image = None;
|
||||
app.viewer_f16 = Some(f);
|
||||
vt::set_texture_roi(ctx, app.session.state.region_of_interest.or(app.ui.viewer.extended));
|
||||
}
|
||||
FrameImage::Gpu(f) => {
|
||||
@@ -175,9 +186,10 @@ pub(super) fn show_frame(app: &mut EffectcraftApp, ctx: &egui::Context, key: cra
|
||||
};
|
||||
drop(ren);
|
||||
app.viewer_native = Some((id, key, f));
|
||||
app.viewer_shown = Some((id, key));
|
||||
app.viewer_shown = Some((Some(id), key));
|
||||
// Pixels are read back on demand (viewer_pixels).
|
||||
app.viewer_image = None;
|
||||
app.viewer_f16 = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -343,14 +343,34 @@ pub(crate) fn showing_snapshot(app: &EffectcraftApp, ctx: &egui::Context) -> boo
|
||||
app.session.snapshot.is_some() && (app.session.state.viewer.show_snapshot || f5)
|
||||
}
|
||||
|
||||
/// Show Channel and exposure, then the display colour conversion (View ▸ Use Display Color
|
||||
/// Management / Simulate Output).
|
||||
fn transformed(img: &egui::ColorImage, ch: Channel, colorized: bool, stops: f32, dc: Option<vw::DisplayColor>) -> egui::ColorImage {
|
||||
let mut px: Vec<[u8; 4]> = img.pixels.iter().map(|c| c.to_array()).collect();
|
||||
if let Some(dc) = dc.filter(|_| matches!(ch, Channel::Rgb | Channel::RgbStraight)) {
|
||||
dc.apply(&mut px);
|
||||
/// The viewer's display pass for frames encoded as `encoding`: Show Channel, exposure, the HDR
|
||||
/// step such frames still need, then the display colour conversion `dc` (View ▸ Use Display
|
||||
/// Color Management / Simulate Output).
|
||||
pub(crate) fn display_pass(app: &EffectcraftApp, encoding: vw::FrameEncoding, dc: Option<&vw::DisplayColor>) -> vw::DisplayPass {
|
||||
let o = &app.session.state.viewer;
|
||||
vw::DisplayPass::new(o.channel, o.colorized, o.exposure, encoding, dc)
|
||||
}
|
||||
|
||||
/// How the project's 8-bit viewer frames are encoded (rendered with the HDR step).
|
||||
pub(crate) fn rgba8_encoding(app: &EffectcraftApp) -> vw::FrameEncoding {
|
||||
vw::FrameEncoding::of(&app.session.project.settings, false)
|
||||
}
|
||||
|
||||
/// A key of a display pass's parameters, for the texture caches.
|
||||
pub(crate) fn pass_key(pass: &vw::DisplayPass) -> u64 {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
crate::display_gpu::uniform(pass).hash(&mut h);
|
||||
if let Some(vw::PassColor::Lut(l)) = &pass.color {
|
||||
(std::sync::Arc::as_ptr(l) as usize).hash(&mut h);
|
||||
}
|
||||
vw::display_transform(&mut px, ch, colorized, stops);
|
||||
h.finish()
|
||||
}
|
||||
|
||||
/// The display pass on the CPU over an 8-bit frame.
|
||||
fn transformed(img: &egui::ColorImage, pass: &vw::DisplayPass) -> egui::ColorImage {
|
||||
let mut px: Vec<[u8; 4]> = img.pixels.iter().map(|c| c.to_array()).collect();
|
||||
pass.apply_rgba8(&mut px);
|
||||
egui::ColorImage::new(img.size, px.into_iter().map(|a| Color32::from_rgba_premultiplied(a[0], a[1], a[2], a[3])).collect())
|
||||
}
|
||||
|
||||
@@ -362,9 +382,60 @@ pub(crate) fn display_key(app: &EffectcraftApp, has_conversion: bool) -> String
|
||||
+ &format!("{:?}{:?}{}", app.session.project.settings.working_space, app.session.project.settings.output_space, opts.display_color_management)
|
||||
}
|
||||
|
||||
/// Draw `src` through `pass` into `rect`: the display pass on egui-wgpu's device (one texture per
|
||||
/// slot, with mips), else on the CPU into a texture averaged down for the magnification (#417).
|
||||
fn draw_through_pass(
|
||||
app: &mut EffectcraftApp,
|
||||
ctx: &egui::Context,
|
||||
painter: &egui::Painter,
|
||||
snapshot: bool,
|
||||
src: crate::display_gpu::Source,
|
||||
pass: &vw::DisplayPass,
|
||||
rect: Rect,
|
||||
) {
|
||||
let smooth = app.session.prefs.viewer_zoom_smooth();
|
||||
if let Some(gpu) = app.display_gpu()
|
||||
&& let Some(rs) = &app.window_wgpu
|
||||
{
|
||||
let bake = if snapshot { &mut app.snapshot_bake } else { &mut app.viewer_bake };
|
||||
if let Some(id) = bake.show(rs, &gpu, &src, pass, smooth) {
|
||||
painter.image(id, rect, bake.uv(), Color32::WHITE);
|
||||
return;
|
||||
}
|
||||
}
|
||||
let size = src.size();
|
||||
let minify = super::viewer::minify_factor(smooth, size[0] as f64, (rect.width() * ctx.pixels_per_point()) as f64);
|
||||
let key = {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
let ptr = match &src {
|
||||
crate::display_gpu::Source::F16(f) => std::sync::Arc::as_ptr(f) as usize,
|
||||
crate::display_gpu::Source::Rgba8(c) => std::sync::Arc::as_ptr(c) as usize,
|
||||
};
|
||||
(ptr, pass_key(pass), minify).hash(&mut h);
|
||||
h.finish()
|
||||
};
|
||||
let id = egui::Id::unique(if snapshot { "viewer-snapshot-tex" } else { "viewer-display-tex" });
|
||||
let tex = match ctx.data(|d| d.get_temp::<(u64, egui::TextureHandle)>(id)) {
|
||||
Some((k, tex)) if k == key => tex,
|
||||
_ => {
|
||||
let img = match &src {
|
||||
crate::display_gpu::Source::F16(f) => f.to_color_image(pass),
|
||||
crate::display_gpu::Source::Rgba8(c) => transformed(c, pass),
|
||||
};
|
||||
let img = crate::frames::fit_texture_by(img, crate::frames::max_texture_side(ctx), minify);
|
||||
let tex = ctx.load_texture(if snapshot { "viewer-snapshot" } else { "viewer-display" }, img, super::viewer::zoom_texture_options(smooth));
|
||||
ctx.data_mut(|d| d.insert_temp(id, (key, tex.clone())));
|
||||
tex
|
||||
}
|
||||
};
|
||||
painter.image(tex.id(), rect, fitted_uv(ctx, size, minify), Color32::WHITE);
|
||||
}
|
||||
|
||||
/// Draw the frame into `comp_rect`: the rendered frame (covering the region of interest when one
|
||||
/// is set), or the snapshot while Show Snapshot is on / F5 is held, through Show Channel and the
|
||||
/// exposure. Fast Previews ▸ Wireframe draws layer outlines instead.
|
||||
/// is set), or the snapshot while Show Snapshot is on / F5 is held, through the display pass (Show
|
||||
/// Channel, exposure, the HDR step of 32 bpc frames, display colour). Fast Previews ▸ Wireframe
|
||||
/// draws layer outlines instead.
|
||||
pub(crate) fn draw_frame(app: &mut EffectcraftApp, ctx: &egui::Context, painter: &egui::Painter, comp_rect: Rect, cid: ItemId, ectx: &EvalCtx) {
|
||||
clear_sample_source(ctx);
|
||||
let opts = app.session.state.viewer.clone();
|
||||
@@ -391,47 +462,58 @@ pub(crate) fn draw_frame(app: &mut EffectcraftApp, ctx: &egui::Context, painter:
|
||||
}
|
||||
return;
|
||||
}
|
||||
// The snapshot went (Purge, a new one): its half floats and texture go too.
|
||||
if app
|
||||
.snapshot_f16
|
||||
.as_ref()
|
||||
.is_some_and(|(img, _)| app.session.snapshot.as_ref().is_none_or(|s| !std::ptr::eq(img.as_ptr(), std::sync::Arc::as_ptr(&s.image))))
|
||||
{
|
||||
app.snapshot_f16 = None;
|
||||
app.snapshot_bake.free(app.window_wgpu.as_ref());
|
||||
}
|
||||
let dc = vw::DisplayColor::of(&app.session);
|
||||
let dc_key = display_key(app, dc.is_some());
|
||||
let plain = opts.channel == Channel::Rgb && opts.exposure == 0.0 && dc.is_none();
|
||||
let display = SampleDisplay { channel: opts.channel, colorized: opts.colorized, exposure: opts.exposure, conversion: dc.clone() };
|
||||
let region = shown_region(ctx, [ectx.comp.width as f32, ectx.comp.height as f32]);
|
||||
let uv = Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0));
|
||||
if snap {
|
||||
let Some(s) = app.session.snapshot.clone() else { return };
|
||||
let key = {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
(std::sync::Arc::as_ptr(&s.image) as usize, opts.channel.id(), opts.colorized, opts.exposure.to_bits(), &dc_key).hash(&mut h);
|
||||
h.finish()
|
||||
};
|
||||
let id = egui::Id::unique("viewer-snapshot-tex");
|
||||
let tex = match ctx.data(|d| d.get_temp::<(u64, egui::TextureHandle)>(id)) {
|
||||
Some((k, tex)) if k == key => tex,
|
||||
_ => {
|
||||
let img = crate::frames::to_color_image(&s.image);
|
||||
let img = if plain { img } else { transformed(&img, opts.channel, opts.colorized, opts.exposure, dc) };
|
||||
let opts = super::viewer::zoom_texture_options(app.session.prefs.viewer_zoom_smooth());
|
||||
let tex = crate::frames::load_fitted(ctx, "viewer-snapshot", img, opts);
|
||||
ctx.data_mut(|d| d.insert_temp(id, (key, tex.clone())));
|
||||
tex
|
||||
let pass = display_pass(app, s.encoding, dc.as_ref());
|
||||
// The snapshot in half floats, made once (it holds the render's full range).
|
||||
let f = match &app.snapshot_f16 {
|
||||
Some((_, f)) => f.clone(),
|
||||
None => {
|
||||
let f = std::sync::Arc::new(crate::frames::F16Frame::from_image(&s.image, s.encoding));
|
||||
app.snapshot_f16 = Some((std::sync::Arc::downgrade(&s.image), f.clone()));
|
||||
f
|
||||
}
|
||||
};
|
||||
painter.image(tex.id(), comp_rect, uv, Color32::WHITE);
|
||||
note_sample_source(
|
||||
ctx,
|
||||
SampleSource::Snapshot(s.image.clone()),
|
||||
[0.0, 0.0, ectx.comp.width as f64, ectx.comp.height as f64],
|
||||
(!plain).then_some(display),
|
||||
);
|
||||
draw_through_pass(app, ctx, painter, true, crate::display_gpu::Source::F16(f), &pass, comp_rect);
|
||||
note_sample_source(ctx, SampleSource::Snapshot(s.image.clone()), [0.0, 0.0, ectx.comp.width as f64, ectx.comp.height as f64], Some(pass));
|
||||
painter.text(comp_rect.left_top() + vec2(6.0, 6.0), Align2::LEFT_TOP, crate::i18n::tr("Snapshot"), Tokens::ui(11.0), SNAP_COLOR);
|
||||
return;
|
||||
}
|
||||
let zoom = comp_rect.width() as f64 / ectx.comp.width.max(1) as f64;
|
||||
let roi_rect = || match texture_roi(ctx) {
|
||||
Some([x, y, w, h]) => Rect::from_min_size(comp_rect.min + vec2((x * zoom) as f32, (y * zoom) as f32), vec2((w * zoom) as f32, (h * zoom) as f32)),
|
||||
None => comp_rect,
|
||||
};
|
||||
// A half-float frame (32 bpc): always through the display pass, which applies the HDR step
|
||||
// after the exposure.
|
||||
if let Some(f) = app.viewer_f16.clone() {
|
||||
if app.viewer_shown.is_none_or(|(_, k)| k.comp != cid.0) {
|
||||
return;
|
||||
}
|
||||
let pass = display_pass(app, f.encoding, dc.as_ref());
|
||||
draw_through_pass(app, ctx, painter, false, crate::display_gpu::Source::F16(f.clone()), &pass, roi_rect());
|
||||
note_sample_source(ctx, SampleSource::F16(f), region, Some(pass));
|
||||
return;
|
||||
}
|
||||
let pass = display_pass(app, rgba8_encoding(app), dc.as_ref());
|
||||
let plain = pass.is_identity();
|
||||
// A GPU frame (the web viewer): drawn straight from its texture; Show Channel / exposure
|
||||
// read it back first (asynchronously in the browser: from the next frame on).
|
||||
let gpu = matches!((&app.viewer_shown, &app.viewer_native), (Some(s), Some(n)) if s.0 == n.0 && s.1 == n.1 && n.1.comp == cid.0);
|
||||
let gpu = matches!((&app.viewer_shown, &app.viewer_native), (Some(s), Some(n)) if s.0 == Some(n.0) && s.1 == n.1 && n.1.comp == cid.0);
|
||||
let rect = if gpu {
|
||||
let Some(id) = app.viewer_shown.map(|s| s.0) else { return };
|
||||
let Some(id) = app.viewer_shown.and_then(|s| s.0) else { return };
|
||||
if plain || app.viewer_pixels().is_none() {
|
||||
if !plain {
|
||||
ctx.request_repaint();
|
||||
@@ -448,11 +530,7 @@ pub(crate) fn draw_frame(app: &mut EffectcraftApp, ctx: &egui::Context, painter:
|
||||
if k.comp != cid.0 {
|
||||
return;
|
||||
}
|
||||
let zoom = comp_rect.width() as f64 / ectx.comp.width.max(1) as f64;
|
||||
let rect = match texture_roi(ctx) {
|
||||
Some([x, y, w, h]) => Rect::from_min_size(comp_rect.min + vec2((x * zoom) as f32, (y * zoom) as f32), vec2((w * zoom) as f32, (h * zoom) as f32)),
|
||||
None => comp_rect,
|
||||
};
|
||||
let rect = roi_rect();
|
||||
if plain {
|
||||
let uv = app.viewer_image.as_ref().map_or(uv, |src| fitted_uv(ctx, src.size, minify));
|
||||
painter.image(tex.id(), rect, uv, Color32::WHITE);
|
||||
@@ -464,51 +542,24 @@ pub(crate) fn draw_frame(app: &mut EffectcraftApp, ctx: &egui::Context, painter:
|
||||
rect
|
||||
};
|
||||
let Some(src) = app.viewer_image.clone() else { return };
|
||||
let smooth = app.session.prefs.viewer_zoom_smooth();
|
||||
// Averaged down for the magnification, as the frame's own texture is (#417).
|
||||
let minify = super::viewer::minify_factor(smooth, src.size[0] as f64, (rect.width() * ctx.pixels_per_point()) as f64);
|
||||
let key = {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
(std::sync::Arc::as_ptr(&src) as usize, opts.channel.id(), opts.colorized, opts.exposure.to_bits(), &dc_key, minify).hash(&mut h);
|
||||
h.finish()
|
||||
};
|
||||
let id = egui::Id::unique("viewer-display-tex");
|
||||
let tex = match ctx.data(|d| d.get_temp::<(u64, egui::TextureHandle)>(id)) {
|
||||
Some((k, tex)) if k == key => tex,
|
||||
_ => {
|
||||
let img =
|
||||
crate::frames::fit_texture_by(transformed(&src, opts.channel, opts.colorized, opts.exposure, dc), crate::frames::max_texture_side(ctx), minify);
|
||||
let tex = ctx.load_texture("viewer-display", img, super::viewer::zoom_texture_options(smooth));
|
||||
ctx.data_mut(|d| d.insert_temp(id, (key, tex.clone())));
|
||||
tex
|
||||
}
|
||||
};
|
||||
painter.image(tex.id(), rect, fitted_uv(ctx, src.size, minify), Color32::WHITE);
|
||||
note_sample_source(ctx, SampleSource::Cpu(src), region, Some(display));
|
||||
draw_through_pass(app, ctx, painter, false, crate::display_gpu::Source::Rgba8(src.clone()), &pass, rect);
|
||||
note_sample_source(ctx, SampleSource::Cpu(src), region, Some(pass));
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
enum SampleSource {
|
||||
Cpu(std::sync::Arc<egui::ColorImage>),
|
||||
F16(std::sync::Arc<crate::frames::F16Frame>),
|
||||
Gpu(crate::frames::FrameKey),
|
||||
Snapshot(std::sync::Arc<effectcraft_engine::render::Image>),
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct SampleDisplay {
|
||||
channel: Channel,
|
||||
colorized: bool,
|
||||
exposure: f32,
|
||||
conversion: Option<vw::DisplayColor>,
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct DisplayedSample {
|
||||
pass: u64,
|
||||
source: SampleSource,
|
||||
region: [f64; 4],
|
||||
display: Option<SampleDisplay>,
|
||||
display: Option<vw::DisplayPass>,
|
||||
}
|
||||
|
||||
fn sample_source_id() -> egui::Id {
|
||||
@@ -519,36 +570,40 @@ pub(crate) fn clear_sample_source(ctx: &egui::Context) {
|
||||
ctx.data_mut(|data| data.remove::<DisplayedSample>(sample_source_id()));
|
||||
}
|
||||
|
||||
fn note_sample_source(ctx: &egui::Context, source: SampleSource, region: [f64; 4], display: Option<SampleDisplay>) {
|
||||
fn note_sample_source(ctx: &egui::Context, source: SampleSource, region: [f64; 4], display: Option<vw::DisplayPass>) {
|
||||
let sample = DisplayedSample { pass: ctx.cumulative_pass_nr(), source, region, display };
|
||||
ctx.data_mut(|data| data.insert_temp(sample_source_id(), sample));
|
||||
}
|
||||
|
||||
/// Sample only a frame actually drawn this pass, with that draw's channel, exposure and
|
||||
/// display conversion. Hidden and wireframe views deliberately have no sample source.
|
||||
/// Sample only a frame actually drawn this pass, with that draw's display pass (channel,
|
||||
/// exposure, HDR step and display conversion). Hidden and wireframe views deliberately have no
|
||||
/// sample source.
|
||||
pub(crate) fn sample_displayed(app: &EffectcraftApp, ctx: &egui::Context, point: [f64; 2]) -> Option<[f32; 4]> {
|
||||
let sample = ctx.data(|data| data.get_temp::<DisplayedSample>(sample_source_id()))?;
|
||||
if sample.pass != ctx.cumulative_pass_nr() {
|
||||
return None;
|
||||
}
|
||||
let pixel = match sample.source {
|
||||
SampleSource::Cpu(image) => super::fx_widgets::frame_pixel(&image, sample.region, point)?,
|
||||
let rgba8 = |c: Color32| c.to_array().map(|v| v as f32 / 255.0);
|
||||
// The frame's premultiplied pixel.
|
||||
let raw: [f32; 4] = match sample.source {
|
||||
SampleSource::Cpu(image) => rgba8(super::fx_widgets::frame_pixel(&image, sample.region, point)?),
|
||||
SampleSource::F16(f) => f.pixel(super::fx_widgets::frame_pixel_index(f.size(), sample.region, point)?)?,
|
||||
SampleSource::Gpu(key) => {
|
||||
if app.viewer_shown.map(|(_, shown)| shown) != Some(key) {
|
||||
return None;
|
||||
}
|
||||
super::fx_widgets::frame_pixel(app.viewer_image.as_ref()?, sample.region, point)?
|
||||
rgba8(super::fx_widgets::frame_pixel(app.viewer_image.as_ref()?, sample.region, point)?)
|
||||
}
|
||||
SampleSource::Snapshot(image) => {
|
||||
let index = super::fx_widgets::frame_pixel_index([image.width as usize, image.height as usize], sample.region, point)?;
|
||||
crate::frames::display_pixel(*image.data.get(index)?)
|
||||
*image.data.get(index)?
|
||||
}
|
||||
};
|
||||
let image = egui::ColorImage::new([1, 1], vec![pixel]);
|
||||
let image = match sample.display {
|
||||
Some(display) => transformed(&image, display.channel, display.colorized, display.exposure, display.conversion),
|
||||
None => image,
|
||||
let shown = match &sample.display {
|
||||
Some(pass) => pass.pixel(raw),
|
||||
None => crate::frames::display_pixel(raw).to_array(),
|
||||
};
|
||||
let image = egui::ColorImage::new([1, 1], vec![Color32::from_rgba_premultiplied(shown[0], shown[1], shown[2], shown[3])]);
|
||||
super::fx_widgets::sample_frame(&image, [0.0, 0.0, 1.0, 1.0], [0.5, 0.5])
|
||||
}
|
||||
|
||||
|
||||
@@ -22,13 +22,25 @@ use crate::theme::Tokens;
|
||||
pub enum PassiveTexture {
|
||||
Cpu(egui::TextureHandle),
|
||||
Gpu(egui::TextureId, Arc<effectcraft_gpu::DisplayFrame>),
|
||||
/// A half-float frame (32 bpc) through the display pass on egui-wgpu's device.
|
||||
Baked(Box<crate::display_gpu::Bake>),
|
||||
}
|
||||
|
||||
impl PassiveTexture {
|
||||
fn id(&self) -> egui::TextureId {
|
||||
/// The texture to draw (`None`: the display pass has not made one).
|
||||
fn id(&self) -> Option<egui::TextureId> {
|
||||
match self {
|
||||
PassiveTexture::Cpu(h) => h.id(),
|
||||
PassiveTexture::Gpu(id, _) => *id,
|
||||
PassiveTexture::Cpu(h) => Some(h.id()),
|
||||
PassiveTexture::Gpu(id, _) => Some(*id),
|
||||
PassiveTexture::Baked(b) => b.id(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Where the frame is in the texture (a frame past the texture limit is averaged down).
|
||||
fn uv(&self) -> Rect {
|
||||
match self {
|
||||
PassiveTexture::Baked(b) => b.uv(),
|
||||
_ => Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -106,9 +118,9 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, id: u32, rect: Rect) {
|
||||
let dkey = display_hash(&super::viewer_tools::display_key(app, dc.is_some()));
|
||||
set_texture(app, &ctx, id, key, img, dc, dkey);
|
||||
}
|
||||
match app.passive_tex.get(&id).filter(|(k, _)| k.comp == cid.0) {
|
||||
Some((_, tex)) => {
|
||||
p.image(tex.id(), r, Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE);
|
||||
match app.passive_tex.get(&id).filter(|(k, _)| k.comp == cid.0).and_then(|(_, tex)| Some((tex.id()?, tex.uv()))) {
|
||||
Some((tex, uv)) => {
|
||||
p.image(tex, r, uv, Color32::WHITE);
|
||||
}
|
||||
None => {
|
||||
let bg = c.background;
|
||||
@@ -128,7 +140,7 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, id: u32, rect: Rect) {
|
||||
}
|
||||
|
||||
/// Keep the frame viewer `id` shows in its texture.
|
||||
fn set_texture(
|
||||
pub(crate) fn set_texture(
|
||||
app: &mut EffectcraftApp,
|
||||
ctx: &egui::Context,
|
||||
id: u32,
|
||||
@@ -149,16 +161,19 @@ fn set_texture(
|
||||
return;
|
||||
}
|
||||
ctx.data_mut(|d| d.insert_temp(dc_id, dkey));
|
||||
let opts = super::viewer::zoom_texture_options(app.session.prefs.viewer_zoom_smooth());
|
||||
// The monitor conversion the active viewer applies (Use Display Color Management).
|
||||
let img = match (img, &dc) {
|
||||
(FrameImage::Cpu(img), Some(dc)) => {
|
||||
let mut px: Vec<[u8; 4]> = img.pixels.iter().map(|c| c.to_array()).collect();
|
||||
dc.apply(&mut px);
|
||||
let pixels = px.into_iter().map(|a| Color32::from_rgba_premultiplied(a[0], a[1], a[2], a[3])).collect();
|
||||
FrameImage::Cpu(Arc::new(egui::ColorImage::new(img.size, pixels)))
|
||||
let smooth = app.session.prefs.viewer_zoom_smooth();
|
||||
let opts = super::viewer::zoom_texture_options(smooth);
|
||||
let pass = |encoding| passive_pass(encoding, dc.as_ref());
|
||||
let gpu = app.display_gpu();
|
||||
let img = match img {
|
||||
// Frames past the display texture limit take the CPU pass (display conversion first, then
|
||||
// the averaging down), which the GPU bake leaves out.
|
||||
FrameImage::CpuF16(f)
|
||||
if app.window_wgpu.is_some() && gpu.as_ref().is_some_and(|g| !crate::display_gpu::Source::F16(f.clone()).needs_cpu_pass(g.max_side())) =>
|
||||
{
|
||||
FrameImage::CpuF16(f)
|
||||
}
|
||||
(img, _) => img,
|
||||
img => cpu_passive_image(app, img, dc.as_ref()),
|
||||
};
|
||||
let tex = match (img, app.passive_tex.remove(&id)) {
|
||||
(FrameImage::Cpu(img), Some((_, PassiveTexture::Cpu(mut h))))
|
||||
@@ -171,10 +186,23 @@ fn set_texture(
|
||||
free(app, old.map(|(_, t)| t));
|
||||
PassiveTexture::Cpu(crate::frames::load_fitted(ctx, format!("viewer-{id}"), (*img).clone(), opts))
|
||||
}
|
||||
(FrameImage::CpuF16(f), old) => {
|
||||
let Some(gpu) = gpu else { return };
|
||||
let mut bake = match old {
|
||||
Some((_, PassiveTexture::Baked(b))) => b,
|
||||
old => {
|
||||
free(app, old.map(|(_, t)| t));
|
||||
Box::default()
|
||||
}
|
||||
};
|
||||
let Some(rs) = &app.window_wgpu else { return };
|
||||
bake.show(rs, &gpu, &crate::display_gpu::Source::F16(f.clone()), &pass(f.encoding), smooth);
|
||||
PassiveTexture::Baked(bake)
|
||||
}
|
||||
(FrameImage::Gpu(f), old) => {
|
||||
let Some(rs) = &app.wgpu else { return };
|
||||
let view = f.texture.create_view(&Default::default());
|
||||
let sampler = || super::viewer::zoom_sampler(app.session.prefs.viewer_zoom_smooth());
|
||||
let sampler = || super::viewer::zoom_sampler(smooth);
|
||||
let mut ren = rs.renderer.write();
|
||||
let tid = match old {
|
||||
Some((_, PassiveTexture::Gpu(tid, _))) => {
|
||||
@@ -189,6 +217,35 @@ fn set_texture(
|
||||
app.passive_tex.insert(id, (key, tex));
|
||||
}
|
||||
|
||||
/// A passive viewer's display pass: the monitor conversion the active viewer applies (Use
|
||||
/// Display Color Management), with no channel or exposure of its own; for half-float frames
|
||||
/// also the HDR step.
|
||||
fn passive_pass(
|
||||
encoding: effectcraft_engine::viewer::FrameEncoding,
|
||||
dc: Option<&effectcraft_engine::viewer::DisplayColor>,
|
||||
) -> effectcraft_engine::viewer::DisplayPass {
|
||||
effectcraft_engine::viewer::DisplayPass::new(effectcraft_engine::viewer::Channel::Rgb, false, 0.0, encoding, dc)
|
||||
}
|
||||
|
||||
/// A frame as a passive viewer shows it without the display pass on the GPU: CPU frames through
|
||||
/// [`passive_pass`] (GPU frames unchanged).
|
||||
pub(crate) fn cpu_passive_image(app: &EffectcraftApp, img: FrameImage, dc: Option<&effectcraft_engine::viewer::DisplayColor>) -> FrameImage {
|
||||
match img {
|
||||
FrameImage::Cpu(img) => {
|
||||
let pass = passive_pass(super::viewer_tools::rgba8_encoding(app), dc);
|
||||
if pass.is_identity() {
|
||||
return FrameImage::Cpu(img);
|
||||
}
|
||||
let mut px: Vec<[u8; 4]> = img.pixels.iter().map(|c| c.to_array()).collect();
|
||||
pass.apply_rgba8(&mut px);
|
||||
let pixels = px.into_iter().map(|a| Color32::from_rgba_premultiplied(a[0], a[1], a[2], a[3])).collect();
|
||||
FrameImage::Cpu(Arc::new(egui::ColorImage::new(img.size, pixels)))
|
||||
}
|
||||
FrameImage::CpuF16(f) => FrameImage::Cpu(Arc::new(f.to_color_image(&passive_pass(f.encoding, dc)))),
|
||||
img => img,
|
||||
}
|
||||
}
|
||||
|
||||
/// A hash of the display conversion's inputs (see `viewer_tools::display_key`).
|
||||
fn display_hash(key: &str) -> u64 {
|
||||
use std::hash::{Hash, Hasher};
|
||||
@@ -199,8 +256,10 @@ fn display_hash(key: &str) -> u64 {
|
||||
|
||||
/// Give a passive viewer's texture back (GPU textures are registered with the renderer).
|
||||
fn free(app: &EffectcraftApp, tex: Option<PassiveTexture>) {
|
||||
if let (Some(PassiveTexture::Gpu(id, _)), Some(rs)) = (tex, &app.wgpu) {
|
||||
rs.renderer.write().free_texture(&id);
|
||||
match (tex, &app.wgpu) {
|
||||
(Some(PassiveTexture::Gpu(id, _)), Some(rs)) => rs.renderer.write().free_texture(&id),
|
||||
(Some(PassiveTexture::Baked(mut b)), _) => b.free(app.window_wgpu.as_ref()),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -242,3 +242,42 @@ fn a_lost_device_draws_nothing_autosaves_and_closes() {
|
||||
assert_eq!(told.borrow().len(), 1);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// #2797 in the real viewer on a device: a 32 bpc frame of white +2 stops (about 4.0 linear)
|
||||
/// goes through the display pass on egui-wgpu's device, so -2 EV still shows it white and -4 EV
|
||||
/// a quarter of white (the 8-bit viewer showed a sixteenth of white there).
|
||||
#[test]
|
||||
fn the_viewer_exposes_32_bpc_frames_on_the_gpu() {
|
||||
let _device = crate::gpu_test::lock();
|
||||
if !any_adapter() {
|
||||
eprintln!("no wgpu adapter: skipping");
|
||||
return;
|
||||
}
|
||||
let mut s = Session::default();
|
||||
s.execute("comp.new", json!({"name": "Hdr", "width": 64, "height": 64, "duration": 1})).unwrap();
|
||||
s.execute("file.projectSettings", json!({"bitDepth": 32, "workingSpace": "srgb"})).unwrap();
|
||||
let lid = s.execute("layer.newSolid", json!({"color": "#ffffff"})).unwrap()["layer"].as_u64().unwrap();
|
||||
s.execute("effect.apply", json!({"layers": [lid], "effect": "Exposure"})).unwrap();
|
||||
s.execute("prop.set", json!({"layer": lid, "path": "effects/#1/master/exposure", "value": 2.0})).unwrap();
|
||||
let mut h = Harness::builder().with_size(egui::vec2(1200.0, 800.0)).wgpu().build_eframe(move |_| EffectcraftApp::new(s));
|
||||
settle(&mut h);
|
||||
let p = h.state().viewer_f16().expect("a half-float viewer frame").pixel(0).unwrap();
|
||||
let l = effectcraft_engine::color::srgb_to_linear(p[0] / p[3]);
|
||||
assert!(l > 3.9, "{l}");
|
||||
// Screen pixels of the frame's grey (the comp is 64 × 64 points in the viewer at most).
|
||||
let count = |h: &mut Harness<'_, EffectcraftApp>, stops: f32| {
|
||||
h.state_mut().session.state.viewer.exposure = stops;
|
||||
for _ in 0..4 {
|
||||
h.step();
|
||||
}
|
||||
assert!(h.state().viewer_display_texture().is_some(), "drawn by the display pass on the device");
|
||||
let want = (effectcraft_engine::color::linear_to_srgb(l * 2f32.powf(stops)).clamp(0.0, 1.0) * 255.0).round() as i32;
|
||||
let img = h.render().expect("render");
|
||||
let n = img.pixels().filter(|px| px.0[..3].iter().all(|c| (*c as i32 - want).abs() <= 1)).count();
|
||||
(want, n)
|
||||
};
|
||||
let (white, n) = count(&mut h, -2.0);
|
||||
assert!(white == 255 && n >= 1000, "-2 EV: {n} white pixels");
|
||||
let (grey, n) = count(&mut h, -4.0);
|
||||
assert!((130..=145).contains(&grey) && n >= 1000, "-4 EV: {n} pixels of {grey}");
|
||||
}
|
||||
|
||||
@@ -116,7 +116,7 @@ fn source(s: &Session, gpu: Option<Gpu>) -> RenderSource {
|
||||
fn cpu_pixels(f: &FrameImage) -> Vec<egui::Color32> {
|
||||
match f {
|
||||
FrameImage::Cpu(c) => c.pixels.clone(),
|
||||
FrameImage::Gpu(_) => panic!("expected a CPU frame"),
|
||||
FrameImage::Gpu(_) | FrameImage::CpuF16(_) => panic!("expected an 8-bit CPU frame"),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -136,7 +136,7 @@ fn viewer_frames_render_remotely_from_project_diffs() {
|
||||
assert_eq!(frames.remote_busy(), 0);
|
||||
let img = match frames.get(&key(&s, 10)).expect("viewer frame cached") {
|
||||
FrameImage::Cpu(c) => c,
|
||||
FrameImage::Gpu(_) => unreachable!(),
|
||||
FrameImage::Gpu(_) | FrameImage::CpuF16(_) => unreachable!(),
|
||||
};
|
||||
let local = effectcraft_ui_egui::frames::to_color_image(&s.render(comp, c.frame_rate.tick_of(10), opts));
|
||||
assert_eq!(img.size, local.size);
|
||||
|
||||
+22
-1
@@ -220,12 +220,33 @@ instead of skipping pixels (Viewer Zoom Quality More Accurate): GPU frames carry
|
||||
sampled trilinearly, CPU frames go up averaged down by the whole factor that leaves about one
|
||||
texel per screen pixel.
|
||||
|
||||
32 bpc projects keep **half-float frames** on the desktop (`frames::F16Frame`, 8 bytes a pixel:
|
||||
the RAM preview holds half as many as of 8-bit frames). They are rendered without Project
|
||||
Settings ▸ HDR (`RenderOpts::skip_hdr_step`), so the values above 1.0 stay. The viewer's
|
||||
**display pass** (`engine::viewer::DisplayPass`; on the GPU `ui-egui/src/display_gpu.rs` +
|
||||
`display_gpu.wgsl`) then turns a frame into what the panel shows, in this order: decode with the
|
||||
output space's curve, the viewer's exposure in linear light, the HDR step the frame was rendered
|
||||
without (Compand / Tone Map), encode, clamp to 0..1, the display colour conversion (Use Display
|
||||
Color Management, Simulate Output, LUT profiles), Show Channel, premultiply, round to 8 bits.
|
||||
So exposure reaches the full float range, and tone mapping and the monitor conversion see the
|
||||
exposed light. The pass runs on egui-wgpu's device (it needs no GPU compositor) into one RGBA8
|
||||
texture with mips per viewer slot (main viewer, snapshot, each passive viewer: about 110 MB of
|
||||
video memory per slot for a 4K frame, input included); without a device the same steps run on
|
||||
the CPU (`DisplayPass::pixel`, which is also the shader's test reference). 8 and 16 bpc frames
|
||||
stay 8-bit with the HDR step applied when rendered; they go through the same pass when exposure,
|
||||
a channel view or a display conversion applies, else they are drawn as they are. Panels that
|
||||
read the viewer's pixels as 8-bit (Video Preview, keyers, thumbnails) get the half-float frame
|
||||
converted on demand with the HDR step and without the exposure; the Info panel shows the
|
||||
frame's float values. The browser keeps 8-bit frames for now (its frame workers send RGBA8).
|
||||
|
||||
A persistent **disk cache** (`render::disk_cache`, Settings ▸ Media & Disk Cache) backs both
|
||||
the layer cache and the viewer's RAM preview: layer buffers that were slow to render and every
|
||||
preview frame are written (LZ4, atomic rename, checksummed) under 128-bit content keys — the
|
||||
comp's content hash (the comp, what it uses, footage file size and time) plus frame, scale and
|
||||
view for frames; the layer key salted with the footage fingerprint for layers — so entries
|
||||
survive restarts and never go stale. LRU eviction keeps it under the size limit; Edit ▸ Purge
|
||||
survive restarts and never go stale. Frame entries are 8-bit RGBA, or for 32 bpc viewer frames
|
||||
half floats with their bytes shuffled into planes before LZ4 (the entry header's format byte
|
||||
tells them apart; the render options in the key keep the two kinds apart). LRU eviction keeps it under the size limit; Edit ▸ Purge
|
||||
empties it, `cache.diskStats` reports it, and the timeline draws disk-only frames in blue. In the
|
||||
browser the frames live in the Origin Private File System: frame workers write them, the page
|
||||
keeps the index (`DiskIndex`) and reads hits ([web.md](web.md#disk-cache)).
|
||||
|
||||
Reference in New Issue
Block a user