craft-cms: LUT file parsing fixes and GamutCheck::mask_rgba (ports of open PhotoCraft PRs) (#860)

* craft-cms: the changes PhotoCraft made to its copy since extraction

PhotoCraft kept editing crates/cms after craft-cms was extracted from it at
storytold/photocraft@b3222ee8. This ports its 41 commits up to
storytold/photocraft@0143b148 with a three-way merge per file, keeping the
changes craft-cms made on its own (Result-returning API, white preservation,
FOGRA39, named colours, profile ID).

Ported: the .3dl shaper, Mesh header, UTF-16 and .look fixes in lutfile; the
checked ICC offsets and the per-profile curve sample budget; same_bytes and the
byte-confirmed transform cache; the blocked matrix/TRC path and the 8-bit
first-stage table; GpuMatrixShaper; cicp_rgb; the ACEScg and Dot Gain 20 %
built-ins (the latter now the default gray); bisection for a table's flat ends;
PhotoCraft spelled with a capital C in the built-in descriptions, with the
shipped coated CMYK profile's bytes as PhotoCraft regenerated them.

Not taken, because craft-cms already does it another way: Curve::inverse_table
(inverse64_many), the effective-dimension loop in multilinear interpolation,
the curve-count and mluc checks, and the .cube DOMAIN writer.

The 8-bit first-stage table checks device white as the exact path does.

Refs #779

* craft-cms: Lut3d::from_fn

PhotoCraft's proof and HDR display tables are built from a chain of several
transforms, which it wrote into Lut3d's fields. Those are private here, so
from_fn samples any RGB function, with the same size bounds as from_transform.

Refs #779

* craft-cms: README names Lut3d::from_fn

* craft-cms: a UTF-8 byte-order mark before a .cube or .3dl file is skipped

A LUT file saved with a UTF-8 BOM failed to parse: the mark stayed on the
first keyword. decode_text now strips it, as it already handles UTF-16.

Ported from storytold/photocraft#9776.

* craft-cms: a .cube data line with a field that is not a number is an error

A data line's fields after the first were parsed with filter_map, so
'0.25 nope 0.5 0.75' silently became the three numbers that did parse. Every
field must now parse.

Ported from storytold/photocraft#10112.

* craft-cms tests: .look data is pinned to hand-written little-endian bytes

The test built its input with the same to_le_bytes the parser mirrors, so a
wrong byte order on both sides would pass. The bytes are now written out.

Ported from storytold/photocraft#10389.

* craft-cms: a 1D .cube whose domain the 33-point bake cannot hold is refused

A 1D LUT is resampled onto a fixed 33-point grid. A steep curve over a
narrow DOMAIN_MIN/DOMAIN_MAX (or LUT_1D_INPUT_RANGE) does not survive that:
a two-node ramp over 0.24..0.26 baked 42 RGB8 levels off. The parser now
measures the baked table against the exact curve for a non-default domain
and refuses one that shifts by more than two RGB8 levels, naming the
channel. Default-domain tables and gentle curves are unchanged.

Ported from storytold/photocraft#9101.

* craft-cms: only a quote that opens a .cube TITLE protects '#'

A quote anywhere on a TITLE line made the text up to the next quote part of
the title, so 'TITLE Look # "draft"' took its comment into the title. Only
a quote at the start of the value opens a quoted title now; an unterminated
one is an error, and an unquoted title ends at '#' like any other line.

Ported from storytold/photocraft#8506 (its lutfile.rs part).

* craft-cms: GamutCheck::mask_rgba computes a whole gamut-warning mask in parallel

A caller building a gamut-warning mask made one delta_e call per pixel on
one thread. mask_rgba fills the mask (255 where alpha > 0 and the pixel is
out of gamut) in 16K-pixel chunks on the rayon pool, serial below that and
on wasm, and returns the count. The result is identical to out_of_gamut per
pixel. A check not built from an RGB source profile is refused before
anything is written, since an RGBA pixel cannot stand for a CMYK, gray or
Lab colour.

Ported from storytold/photocraft#9748.

---------

Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
Co-authored-by: Max Mittelstädt <62167861+TheGalitube@users.noreply.github.com>
Co-authored-by: smittie2000 <smittie2000@gmail.com>
Co-authored-by: bfenty <1398236+bfenty@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: vangel76 <vangel1976@googlemail.com>
This commit is contained in:
Dylan M. Taylor
2026-11-11 19:19:03 -05:00
committed by GitHub
co-authored by NetterSilver Max Mittelstädt smittie2000 bfenty FredApps vangel76
parent 4a7024cd69
commit eadfaf85ed
3 changed files with 240 additions and 16 deletions
+2
View File
@@ -30,6 +30,8 @@ Buffers are checked once per call, before any pixel is written.
* `Transform::proof(src, proof, display, intent, bpc, simulate_paper)`: soft-proof chain.
* `GamutCheck::new(src, proof, ΔE)`: out-of-gamut test (gamut warning).
`check.mask_rgba(&rgba, &mut mask)?` fills a whole mask (255 where alpha > 0 and out of gamut) on
the thread pool and returns the count; it needs an RGB source profile.
* `Lut3d::from_transform(&t, 33)?.to_rgba16f_bytes()`: display transform as a 3D texture;
`Lut3d::from_fn(size, f)` samples any RGB function.
* `Transform::gpu_matrix_shaper()`: accurate RGB input curves + matrix + output curves for
+89 -3
View File
@@ -2,13 +2,18 @@
use crate::clut::Clut;
use crate::pipeline::{Pipeline, Stage};
use crate::profile::Profile;
use crate::profile::{ColorSpace, Profile};
use crate::transform::link_stages;
use crate::{CmsError, Intent, check_len};
use crate::{CmsError, Intent, check_len, invalid_arg};
/// Default ΔE76 threshold above which a colour counts as out of gamut.
pub const DEFAULT_THRESHOLD: f32 = 4.0;
#[cfg(not(target_arch = "wasm32"))]
const PAR_MIN_PIXELS: usize = 16 * 1024;
#[cfg(not(target_arch = "wasm32"))]
const PAR_CHUNK_PIXELS: usize = 16 * 1024;
/// Precomputed out-of-gamut test for colours of `src` against `proof`.
///
/// A colour is out of gamut when sending it to the proof device (relative colorimetric,
@@ -17,6 +22,7 @@ pub const DEFAULT_THRESHOLD: f32 = 4.0;
#[derive(Debug)]
pub struct GamutCheck {
inputs: usize,
source_space: ColorSpace,
grid: Option<Clut>,
exact: (Pipeline, Pipeline),
pub threshold: f32,
@@ -44,7 +50,7 @@ impl GamutCheck {
4 => Some(Clut::sample(vec![17; 4], 1, de)?),
_ => None,
};
Ok(GamutCheck { inputs: n, grid, exact: (direct, via), threshold })
Ok(GamutCheck { inputs: n, source_space: src.color_space, grid, exact: (direct, via), threshold })
}
pub fn inputs(&self) -> usize {
@@ -76,4 +82,84 @@ impl GamutCheck {
pub fn out_of_gamut(&self, px: &[f32]) -> Result<bool, CmsError> {
Ok(self.delta_e(px)? > self.threshold)
}
/// Out-of-gamut mask for RGBA pixels: `255` where the pixel has alpha > 0 and its RGB
/// channels are out of gamut, `0` elsewhere. Returns the number of flagged pixels. Only
/// `min(px.len(), mask.len())` pixels are processed. Runs on the rayon pool (not on wasm);
/// the result is identical to calling [`Self::out_of_gamut`] per pixel.
///
/// # Errors
/// [`CmsError::InvalidArgument`] when the check was not built from an RGB source profile: an
/// RGBA pixel carries only RGB colour channels, so it cannot be tested against a CMYK, gray
/// or Lab source. Nothing is written to `mask` in that case.
pub fn mask_rgba(&self, px: &[[f32; 4]], mask: &mut [u8]) -> Result<usize, CmsError> {
if self.source_space != ColorSpace::Rgb || self.inputs != 3 {
return Err(invalid_arg(format!("gamut mask needs an RGB source profile, got {:?} with {} channels", self.source_space, self.inputs)));
}
let n = px.len().min(mask.len());
let (Some(px), Some(mask)) = (px.get(..n), mask.get_mut(..n)) else { return Ok(0) };
let run = |p: &[[f32; 4]], m: &mut [u8]| {
let mut count = 0;
for (p, m) in p.iter().zip(m.iter_mut()) {
// The source has three channels (checked above), so the check cannot fail.
let hit = p[3] > 0.0 && self.out_of_gamut(&[p[0], p[1], p[2]]).unwrap_or(false);
*m = if hit { 255 } else { 0 };
count += usize::from(hit);
}
count
};
#[cfg(not(target_arch = "wasm32"))]
if n >= PAR_MIN_PIXELS {
use rayon::prelude::*;
return Ok(px.par_chunks(PAR_CHUNK_PIXELS).zip(mask.par_chunks_mut(PAR_CHUNK_PIXELS)).map(|(p, m)| run(p, m)).sum());
}
Ok(run(px, mask))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::Builtin;
fn image(n: usize) -> Vec<[f32; 4]> {
let mut s = 0x9E37_79B9u32;
let mut rnd = move || {
s ^= s << 13;
s ^= s >> 17;
s ^= s << 5;
(s >> 8) as f32 / (1u32 << 24) as f32
};
// Includes out-of-range values and zero alpha.
(0..n).map(|i| [rnd() * 1.2 - 0.1, rnd(), rnd(), if i % 7 == 0 { 0.0 } else { rnd() }]).collect()
}
#[test]
fn mask_rgba_matches_per_pixel_check() {
let src = Builtin::Srgb.profile();
// Sizes straddle the parallel threshold and the chunk size, odd ones included.
for (proof, threshold) in [(Builtin::CoatedCmyk, 4.0), (Builtin::CoatedCmyk, 1.5), (Builtin::GrayGamma22, 4.0), (Builtin::DisplayP3, 0.5)] {
let check = GamutCheck::new(src, proof.profile(), threshold).unwrap();
for n in [0, 1, 1023, 16 * 1024 - 1, 16 * 1024, 16 * 1024 + 1, 50_001] {
let px = image(n);
let want: Vec<u8> = px.iter().map(|p| if p[3] > 0.0 && check.out_of_gamut(&p[..3]).unwrap() { 255 } else { 0 }).collect();
let mut got = vec![7u8; n];
let count = check.mask_rgba(&px, &mut got).unwrap();
assert_eq!(got, want, "{proof:?} threshold {threshold} n {n}");
assert_eq!(count, want.iter().filter(|&&m| m == 255).count());
}
}
}
#[test]
fn mask_rgba_rejects_non_rgb_source() {
// A CMYK (4-channel), gray (1-channel) and Lab (3-channel, not RGB) source cannot be tested from RGBA pixels.
for src in [Builtin::CoatedCmyk, Builtin::GrayGamma22, Builtin::LabD50] {
let check = GamutCheck::new(src.profile(), Builtin::Srgb.profile(), 4.0).unwrap();
let px = image(64);
let mut mask = vec![9u8; 64];
assert!(matches!(check.mask_rgba(&px, &mut mask), Err(CmsError::InvalidArgument(_))), "{src:?}");
assert!(mask.iter().all(|&m| m == 9), "mask must stay untouched");
}
}
}
+149 -13
View File
@@ -135,7 +135,8 @@ pub fn parse_as(format: Format, bytes: &[u8]) -> Result<LutFile, LutError> {
}
}
/// UTF-16 with a byte-order mark (Photoshop loads such `.look` files, #9118), else UTF-8.
/// UTF-16 with a byte-order mark (Photoshop loads such `.look` files, #9118), else UTF-8
/// (a UTF-8 BOM is stripped).
fn decode_text(bytes: &[u8]) -> std::borrow::Cow<'_, str> {
let utf16 = |rest: &[u8], from: fn([u8; 2]) -> u16| {
let units: Vec<u16> = rest.as_chunks::<2>().0.iter().map(|&c| from(c)).collect();
@@ -144,6 +145,7 @@ fn decode_text(bytes: &[u8]) -> std::borrow::Cow<'_, str> {
match bytes {
[0xFF, 0xFE, rest @ ..] => utf16(rest, u16::from_le_bytes),
[0xFE, 0xFF, rest @ ..] => utf16(rest, u16::from_be_bytes),
[0xEF, 0xBB, 0xBF, rest @ ..] => String::from_utf8_lossy(rest),
_ => String::from_utf8_lossy(bytes),
}
}
@@ -155,14 +157,19 @@ pub fn parse_cube(text: &str) -> Result<LutFile, LutError> {
let (mut dmin, mut dmax) = ([0.0f32; 3], [1.0f32; 3]);
let mut rows: Vec<f32> = Vec::new();
for line in text.lines() {
// A '#' inside a quoted TITLE is part of the title, not a comment.
let raw = line.trim_start();
let skip = if raw.starts_with("TITLE") { raw.find('"').and_then(|q| raw[q + 1..].find('"').map(|e| q + e + 2)) } else { None }.unwrap_or(0);
let line = match raw[skip..].find('#') {
Some(i) => &raw[..skip + i],
None => raw,
// Only a quote at the beginning of a TITLE value protects '#'. Quotes inside an
// unquoted title's comment must not turn that comment into part of the title.
if let Some(quoted) = line
.trim_start()
.strip_prefix("TITLE")
.filter(|rest| rest.starts_with(char::is_whitespace))
.and_then(|rest| rest.trim_start().strip_prefix('"'))
{
let Some((value, _)) = quoted.split_once('"') else { return err("unterminated quoted TITLE") };
title = value.to_string();
continue;
}
.trim();
let line = line.split('#').next().unwrap_or("").trim();
if line.is_empty() {
continue;
}
@@ -187,7 +194,9 @@ pub fn parse_cube(text: &str) -> Result<LutFile, LutError> {
}
k if k.starts_with(|c: char| c.is_ascii_digit() || c == '-' || c == '+' || c == '.') => {
let mut v = vec![k.parse::<f32>().map_err(|_| LutError(format!("bad number in `{line}`")))?];
v.extend(it.filter_map(|s| s.parse::<f32>().ok()));
for value in it {
v.push(value.parse::<f32>().map_err(|_| LutError(format!("bad number in `{line}`")))?);
}
if v.len() != 3 {
return err(format!("bad data line `{line}`"));
}
@@ -209,6 +218,15 @@ pub fn parse_cube(text: &str) -> Result<LutFile, LutError> {
if rows.len() != expected_rows {
return err("no LUT_3D_SIZE in .cube file");
}
// #8913: the resample below bakes the curve onto a fixed 33-point grid, so a steep
// curve over a narrow domain cannot survive the resample — the baked table silently
// shifts colours (measured 42 RGB8 levels for a two-node ramp over 0.24..0.26, where
// Photoshop renders the ramp exactly). Rather than proxy the risk by domain width (a
// 0.1..0.17 ramp spans two grid cells yet still loses ~28 levels), measure the
// resample's actual error against the exact curve and refuse only tables whose baked
// output shifts by more than a couple of RGB8 levels. Default-domain tables keep the
// existing bake untouched; gentle curves over any domain bake within tolerance and
// stay accepted.
// The 1D table is resampled onto a 0..1 cube anyway, so its input domain
// (`DOMAIN_MIN`/`MAX`, `LUT_1D_INPUT_RANGE`) is baked into the resample: an input `v`
// reads the table at `(v - min) / (max - min)`, saturating outside.
@@ -221,6 +239,36 @@ pub fn parse_cube(text: &str) -> Result<LutFile, LutError> {
};
let mut l = LutFile::from_fn(&title, 33, |c| [curve(0, c[0]), curve(1, c[1]), curve(2, c[2])])?;
l.title = title;
let non_default = dmin.iter().any(|v| *v != 0.0) || dmax.iter().any(|v| *v != 1.0);
if non_default {
// Worst baked-vs-exact error per channel, in RGB8 levels: the baked cube is read
// by linear interpolation between grid points (red fastest, so channel `ch` sits
// at stride 33^ch), while `curve` is what the file actually says.
const MAX_BAKE_ERROR: f32 = 2.0 / 255.0;
for ch in 0..3 {
let stride = [1usize, 33, 33 * 33][ch];
let mut worst = 0.0f32;
for k in 0..=256 {
let v = k as f32 / 256.0;
let x = v * 32.0;
let i = (x as usize).min(31);
let f = x - i as f32;
let baked = l.data[i * stride * 3 + ch] * (1.0 - f) + l.data[(i + 1).min(32) * stride * 3 + ch] * f;
worst = worst.max((baked - curve(ch, v)).abs());
}
if worst > MAX_BAKE_ERROR {
return err(format!(
"1D LUT domain on {} is {min}..{max}: the curve changes faster than the 33-point grid can \
hold (baking shifts colours by up to {levels:.0} RGB8 levels); export the look as a 3D LUT \
or widen DOMAIN_MIN/DOMAIN_MAX",
["red", "green", "blue"][ch],
min = dmin[ch],
max = dmax[ch],
levels = worst * 255.0,
));
}
}
}
return l.check();
}
err("no LUT_3D_SIZE in .cube file")
@@ -554,6 +602,48 @@ pub fn builtin(id: &str) -> Option<LutFile> {
mod tests {
use super::*;
#[test]
fn cube_title_hashes_and_comment_quotes_are_distinct() {
let rows = "LUT_1D_SIZE 2\n0 0 0 # low\n1 1 1 # high\n";
for (line, title) in [
(r#"TITLE "Look #3" # comment "ignored""#, "Look #3"),
(r#"TITLE Look # "draft""#, "Look"),
(" TITLE\t\"Été #2\"", "Été #2"),
(r#"TITLE "" # empty"#, ""),
] {
assert_eq!(parse_cube(&format!("{line}\n{rows}")).unwrap().title, title);
}
let lut = LutFile::from_fn("Look #3", 2, |c| c).unwrap();
assert_eq!(parse_cube(&write_cube(&lut)).unwrap().title, "Look #3");
assert!(parse_cube(&format!("TITLE \"unterminated # title\n{rows}")).is_err());
}
#[test]
fn cube_rejects_invalid_numeric_fields() {
let text = "LUT_3D_SIZE 2\n0.25 nope 0.5 0.75\n0 0 1\n0 1 0\n0 1 1\n1 0 0\n1 0 1\n1 1 0\n1 1 1\n";
assert!(parse_cube(text).is_err(), "invalid numeric field must not be ignored");
}
#[test]
fn utf8_bom_is_stripped() {
let l = LutFile::identity(3).unwrap();
let mut bytes = vec![0xEF, 0xBB, 0xBF];
bytes.extend_from_slice(write_cube(&l).as_bytes());
let p = parse("a.cube", &bytes).unwrap();
assert_eq!(p.size, l.size);
assert_eq!(p.title, l.title);
let titled = LutFile::from_fn("Look", 3, |c| c).unwrap();
let mut bytes = vec![0xEF, 0xBB, 0xBF];
bytes.extend_from_slice(write_cube(&titled).as_bytes());
assert_eq!(parse("a.cube", &bytes).unwrap().title, "Look");
let mut bytes = vec![0xEF, 0xBB, 0xBF];
bytes.extend_from_slice(b"LUT_3D_SIZE 2\n");
for i in 0..8 {
bytes.extend_from_slice(format!("{} {} {}\n", i & 1, (i >> 1) & 1, i >> 2).as_bytes());
}
assert_eq!(parse("a.cube", &bytes).unwrap().size, 2);
}
/// An identity `.3dl` of edge `n` (12-bit nodes, blue fastest) under `shaper`.
fn identity_3dl(shaper: &[u32], n: usize) -> String {
let mut t = shaper.iter().map(u32::to_string).collect::<Vec<_>>().join(" ") + "\n";
@@ -811,6 +901,49 @@ DOMAIN_MAX 0.5 1 1
assert!(e.0.contains("below DOMAIN_MAX"), "{e}");
}
#[test]
fn cube_1d_narrow_domain_is_refused() {
// #8913: the issue's fixture — a two-node ramp over 0.24..0.26. Baking it onto the
// 33-point grid rendered grey-65 as 148 where Photoshop renders 190 (42-level error).
// It must now be an actionable parse error rather than a silently wrong table.
let e = parse_cube("LUT_1D_SIZE 2\nDOMAIN_MIN 0.24 0.24 0.24\nDOMAIN_MAX 0.26 0.26 0.26\n0 0 0\n1 1 1\n").unwrap_err();
assert!(e.0.contains("changes faster than the 33-point grid"), "{e}");
// The same domain via LUT_1D_INPUT_RANGE is refused identically.
let e = parse_cube("LUT_1D_SIZE 2\nLUT_1D_INPUT_RANGE 0.24 0.26\n0 0 0\n1 1 1\n").unwrap_err();
assert!(e.0.contains("changes faster"), "{e}");
// A per-channel narrow domain names the offending channel.
let e = parse_cube("LUT_1D_SIZE 2\nDOMAIN_MIN 0 0 0.24\nDOMAIN_MAX 1 1 0.26\n0 0 0\n1 1 1\n").unwrap_err();
assert!(e.0.contains("blue"), "{e}");
}
#[test]
fn cube_1d_wide_domain_still_bakes() {
// A steep ramp over 0.1..0.17 spans two grid cells but still loses ~28 RGB8 levels
// to the resample, so the measured-error check refuses it even though the old width
// check let it through.
let e = parse_cube("LUT_1D_SIZE 2\nLUT_1D_INPUT_RANGE 0.1 0.17\n0 0 0\n1 1 1\n").unwrap_err();
assert!(e.0.contains("changes faster"), "steep ramp over 0.1..0.17 refused: {e}");
// A gentle (here linear) curve over a grid-aligned domain bakes exactly and stays
// accepted — the check measures the bake's error, not the domain's width. Value
// assertions, not just a successful parse: the baked table must reproduce the curve.
// Table row k sits at t = k/32 of the domain 0.25..0.75; the curve y = 0.4 + 0.4t
// is gentle (a half-level per grid step), so the bake must reproduce it exactly.
let mut table = String::from("LUT_1D_SIZE 33\nLUT_1D_INPUT_RANGE 0.25 0.75\n");
for k in 0..33 {
let y = 0.4 + 0.4 * (k as f32 / 32.0);
table.push_str(&format!("{y} {y} {y}\n"));
}
let l = parse_cube(&table).expect("gentle curve over a non-default domain still bakes");
let at = |r: usize| l.data[r * 3];
// Grid entry r is input r/32: 0.5 is the middle of the domain (t=0.5) -> 0.6.
assert!((at(16) - 0.6).abs() < 1.0 / 255.0, "baked value at 0.5 tracks the curve: {}", at(16));
// Domain top (input 0.75 = grid 24, t=1) -> 0.8.
assert!((at(24) - 0.8).abs() < 1.0 / 255.0, "baked value at the domain top tracks the curve: {}", at(24));
// Below the domain saturates to the first row.
assert!((at(8) - 0.4).abs() < 1.0 / 255.0, "below the domain saturates: {}", at(8));
parse_cube("LUT_1D_SIZE 2\n0 0 0\n1 1 1\n").expect("default domain");
}
#[test]
fn cube_1d_domain_is_baked_into_the_resample() {
// An identity ramp over the domain 0.25..0.75: 0.25 maps to 0, 0.5 to 0.5, 0.75 to 1.
@@ -1042,11 +1175,14 @@ DOMAIN_MAX 0.5 1 1
}
#[test]
fn look_hex_floats() {
let id = LutFile::identity(2).unwrap();
let hex: String = id.data.iter().flat_map(|v| v.to_le_bytes()).map(|b| format!("{b:02X}")).collect();
fn look_data_is_little_endian_float32() {
// Bytes written out by hand rather than by the encoder the parser mirrors, so a wrong byte
// order fails: 0.25, 0.5, 1.0 (3E800000, 3F000000, 3F800000 stored low byte first), then zeros.
let hex = format!("0000803E 0000003F 0000803F {}", "00000000 ".repeat(21));
let xml = format!("<?xml version=\"1.0\"?><look><LUT><size>\"2\"</size><data>\"{hex}\"</data></LUT></look>");
assert_eq!(parse("a.look", xml.as_bytes()).unwrap().data, id.data);
let data = parse("a.look", xml.as_bytes()).unwrap().data;
assert_eq!(data[..3], [0.25, 0.5, 1.0]);
assert!(data[3..].iter().all(|&v| v == 0.0));
}
#[test]