mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-12 10:03:51 +00:00
* craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time Bands reads the headers of a sequential Huffman-coded (SOF0, SOF1) 8-bit file whose first scan holds every component, and a Reader decodes it one line of MCUs at a time: Huffman decoding, dequantization, inverse transform, upsampling and colour conversion, holding at most three MCU lines of samples. Nothing the size of the image is allocated, so a frame up to the format's 65,535 px sides can be decoded straight into the caller's storage. The samples equal decode()'s for the same rows: both go through the same block transform (idct_block, factored out of reconstruct), the same upsampling maps and the same colour conversion (to_color, to_u8). A component subsampled down reads the MCU line above and below its band, as the whole-image upsampling does. A band that begins a restart interval is a seek point once find_seek_points has found every restart marker, in sequence; readers on other threads can decode from there. Damaged data leaves the rest of its restart interval at zero and decoding resumes at the next marker, so the result is the same however the bands were read. Bits::restart now also clears the exhausted flag: decode() never restarts after running out of data, the band decoder does. parse_frame is split into parse_header (validation and geometry) and the whole-image allocation; parse_scan takes the component identifiers. decode() is unchanged. Tests: bands against decode() sample for sample over jpeg-encoder streams (gray, YCbCr, CMYK, YCCK; 4:4:4, 4:2:2, 4:2:0, 4:4:0; four restart intervals) and a test-only encoder's interleaved layouts with factors of 3 and 4, in order and by seeking; zune-jpeg within the existing tolerance; damage with and without restarts; limits; unsupported files; the libjpeg test images; the mutation run with a seek-equals-in-order check; and a `bands` fuzz target. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: faster inverse transform, rounding, Huffman codes and band rows Each change keeps every sample as it was: decode() and the bands give bit-identical output before and after on the test images, 12-bit and arithmetic-coded files, 48 generated streams and a 48,000 × 34,200 file decoded in bands. - idct_block works on rows of eight: each sum keeps its order (over u, then over v), eight values of x at a time, and rows of zero coefficients are skipped (a zero term changes only the sign of a zero, which the level shift removes). The transposed basis is built once. - to_u8 rounds half away from zero as truncation plus one from .5 up, exact on 0..=255, instead of the `round` library call targets without SSE4.1 make. - Huffman::decode resolves codes of up to 9 bits from a table of the next 9 bits, built by running the bit-by-bit procedure on each, so both give the same symbol. - The band rows are colour converted in loops written out for gray, YCbCr and RGB; YCbCr to RGB is one helper shared with the whole-image decode. On the 48,000 × 34,200 4:4:4 file with a restart interval per line of MCUs (Ryzen 7 7800X3D, idle), one thread decodes a band of 8 rows in about 3.0 ms (5.8 ms before), and 16 threads decode the whole image in 2.05 s (3.8 s before). Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: bands report damage, never fail on it, and see data left before a restart Two cases a mutation run through PhotoCraft's import turned up. - Data that fails in the first MCU made a band read fail, as decode() refuses a stream with no image data. The baseline decoder PhotoCraft uses for other files opens such files (grey, with restarts the rest of the image), so a band read now marks it damaged and goes on: with restart intervals everything after the first decodes. Refusing a file with nothing decodable is the caller's, from the size of its scan data. - An interval that decodes without an invalid code but leaves whole bytes before its restart marker was decoded from cut, shifted or added data. It now counts as damage (libjpeg warns "extraneous bytes before marker"). The check runs after an interval's last MCU, so readers that seek and readers that read in order agree. Over 1,074 sequential 8-bit JPEGs on one machine the damage flag agrees with djpeg's warnings on every file, reading from seek points equals reading in order on all 151 with restart intervals, and the samples are within mean 0.57, max 8 of zune-jpeg's. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: seek points follow the restart markers in the data, as reading in order does find_seek_points gave up on a file whose markers were not all there in sequence, so a damaged or cut-short file with a restart interval at every line of MCUs decoded in order on one core (about 13 s for an undamaged 48,000 × 34,200 file on an idle 8-core machine, against 2 s on 16 threads). With these seek points the same file with one marker renumbered, cut at 60%, or its first interval overwritten decodes in 2.8 to 4.4 s on 16 threads. A reader that reads in order pairs the k-th restart marker in the data with interval k, whatever its number, and resumes after each, so seek points built the same way give the same samples for any data: missing, misnumbered or added markers only change which bands are seek points. Tested: misnumbered, missing (file cut), added and removed markers read in order and by seeking; the mutation run's seek-equals-in-order check (more of its cases now seek); 600,000 runs of the bands fuzz target, which asserts the same; 169 local files with restarts. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: Bands::halo, the MCU lines a reader decodes around its bands A reader that seeks decodes the MCU line above and below its bands when a component is subsampled down. A caller splitting the work across threads needs to know it: reading one band from each seek point of a 4:2:0 file decodes three lines per band (4.97 s for a 48,000 × 34,200 file on 16 threads, against 3.09 s reading four bands from each), while for 4:4:4 the run length makes no difference. Tested against the sampling factors of every interleaved layout. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: band decoder review fixes - Data left over after the last MCU, before EOI, is damage, as it already was before a restart marker. - A scan that names a component twice is Malformed (T.81 B.2.3), not Unsupported. - The upsampling maps are sized from the header and checked against max_alloc before they are made. - The bands fuzz target and the mutation test fail on a read or seek error once Bands::new has accepted a stream; damage never surfaces as an error. - Known-sample CMYK and YCCK fixtures check the ink polarity directly; the bands-equal-decode tests cannot, since both share the colour conversion. - Docs: a single-component frame's bands are 8 rows; Reader::damaged and the scan's size cannot tell a failure at the first MCU from later damage; with a halo a seek decodes from the previous seek point; each fuzz target writes to its own corpus; doc comments on the remaining new functions. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: test YCCK chroma and trailing-data samples - The ink fixture gains a YCCK case with Cb and Cr away from neutral, with the T.871 values worked out by hand: a conversion that drops or swaps the chroma now fails it. - Bytes added before EOI leave the samples as the clean stream's; the test now checks that, not only that reading in order and seeking agree. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> --------- Co-authored-by: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
204 lines
8.2 KiB
Rust
204 lines
8.2 KiB
Rust
//! Deterministic mutation run over the test images and fuzz seeds: decoding corrupted input never panics and
|
||
//! stays within the caller's limits.
|
||
|
||
use craft_jpeg::{Bands, Limits, Samples, decode, frame};
|
||
|
||
/// Inputs and mutation counts. The 227×149 test images take tens of milliseconds each to decode
|
||
/// in a debug build; the small fuzz seeds (SOF9 with restarts, 12-bit SOF9, SOF10, 12-bit SOF1)
|
||
/// take microseconds and get most of the cases.
|
||
const INPUTS: [(&str, usize); 8] = [
|
||
("tests/data/testorig.jpg", 12),
|
||
("tests/data/testimgari.jpg", 12),
|
||
("tests/data/testimgari-huffman.jpg", 12),
|
||
("tests/data/testorig12.jpg", 12),
|
||
("fuzz/corpus/decode/sof9-restart.jpg", 75),
|
||
("fuzz/corpus/decode/sof9-12bit.jpg", 75),
|
||
("fuzz/corpus/decode/sof10-progressive.jpg", 75),
|
||
("fuzz/corpus/decode/sof1-12bit.jpg", 75),
|
||
];
|
||
|
||
/// Room for the 227×149 fixtures, too little for a mutated frame header that grows them much.
|
||
fn test_limits() -> Limits {
|
||
let mut limits = Limits::default();
|
||
limits.max_pixels = 1 << 16;
|
||
limits.max_alloc = 1 << 20;
|
||
limits
|
||
}
|
||
|
||
struct Rng(u64);
|
||
|
||
impl Rng {
|
||
fn next(&mut self) -> u64 {
|
||
// xorshift64*
|
||
self.0 ^= self.0 >> 12;
|
||
self.0 ^= self.0 << 25;
|
||
self.0 ^= self.0 >> 27;
|
||
self.0.wrapping_mul(0x2545_F491_4F6C_DD1D)
|
||
}
|
||
|
||
fn below(&mut self, n: usize) -> usize {
|
||
(self.next() % n.max(1) as u64) as usize
|
||
}
|
||
|
||
fn byte(&mut self) -> u8 {
|
||
self.next() as u8
|
||
}
|
||
}
|
||
|
||
/// Offset of the first SOFn segment's body, if any.
|
||
fn sof_body(b: &[u8]) -> Option<usize> {
|
||
(2..b.len().saturating_sub(1))
|
||
.find(|&i| b[i] == 0xFF && matches!(b[i + 1], 0xC0..=0xCF) && !matches!(b[i + 1], 0xC4 | 0xC8 | 0xCC))
|
||
.map(|i| i + 4)
|
||
}
|
||
|
||
fn mutate(rng: &mut Rng, original: &[u8]) -> Vec<u8> {
|
||
let mut b = original.to_vec();
|
||
match rng.below(6) {
|
||
0 => b.truncate(rng.below(original.len())),
|
||
1 => {
|
||
for _ in 0..1 + rng.below(8) {
|
||
let at = rng.below(b.len());
|
||
b[at] ^= 1 << rng.below(8);
|
||
}
|
||
}
|
||
2 => {
|
||
for _ in 0..1 + rng.below(4) {
|
||
let at = rng.below(b.len());
|
||
b[at] = rng.byte();
|
||
}
|
||
}
|
||
3 => {
|
||
// Frame header fields: precision, height, width, component count and the
|
||
// per-component sampling and table selectors.
|
||
if let Some(body) = sof_body(&b) {
|
||
for _ in 0..1 + rng.below(3) {
|
||
let at = body + rng.below(15);
|
||
if let Some(v) = b.get_mut(at) {
|
||
*v = rng.byte();
|
||
}
|
||
}
|
||
}
|
||
}
|
||
4 => {
|
||
let at = rng.below(b.len());
|
||
let len = rng.below(64).min(b.len() - at);
|
||
b.drain(at..at + len);
|
||
}
|
||
_ => {
|
||
// Repeat a span, e.g. a marker segment or part of the entropy-coded data.
|
||
let at = rng.below(b.len());
|
||
let len = rng.below(256).min(b.len() - at);
|
||
let span = b[at..at + len].to_vec();
|
||
let to = rng.below(b.len());
|
||
b.splice(to..to, span);
|
||
}
|
||
}
|
||
b
|
||
}
|
||
|
||
#[test]
|
||
fn mutated_streams_never_panic_and_stay_within_limits() {
|
||
let mut rng = Rng(0x9E37_79B9_7F4A_7C15);
|
||
let limits = test_limits();
|
||
let (mut ok, mut errors) = (0, 0);
|
||
for (name, cases) in INPUTS {
|
||
let path = format!("{}/{name}", env!("CARGO_MANIFEST_DIR"));
|
||
let original = std::fs::read(&path).unwrap_or_else(|e| panic!("{path}: {e}"));
|
||
assert!(decode(&original, &limits, None).is_ok(), "{name} must decode within the test limits");
|
||
for case in 0..cases {
|
||
let bytes = mutate(&mut rng, &original);
|
||
let _ = frame(&bytes);
|
||
match decode(&bytes, &limits, None) {
|
||
Ok(img) => {
|
||
let pixels = u64::from(img.width) * u64::from(img.height);
|
||
let (len, width) = match &img.samples {
|
||
Samples::U8(v) => (v.len() as u64, 1),
|
||
Samples::U16(v) => (v.len() as u64, 2),
|
||
};
|
||
assert!(pixels <= limits.max_pixels, "{name} case {case}: {pixels} pixels");
|
||
assert!(len * width <= limits.max_alloc, "{name} case {case}: {} bytes", len * width);
|
||
assert_eq!(len, pixels * u64::from(img.components), "{name} case {case}");
|
||
ok += 1;
|
||
}
|
||
Err(_) => errors += 1,
|
||
}
|
||
}
|
||
}
|
||
eprintln!("{ok} decoded, {errors} rejected");
|
||
// A mutation run that only produces errors would no longer reach the entropy decoders.
|
||
assert!(ok > 50 && errors > 50, "{ok} decoded, {errors} rejected");
|
||
}
|
||
|
||
/// Every band read in order, and (when the restart markers make more than band 0 a seek point)
|
||
/// read again from each seek point; `None` when `Bands::new` rejects the stream. Once it accepts
|
||
/// one, damage is reported, never an error: a failed read or seek fails the test.
|
||
fn read_bands(bytes: &[u8], limits: &Limits) -> Option<(Vec<u8>, Option<Vec<u8>>)> {
|
||
let mut bands = Bands::new(bytes, limits, None).ok()?;
|
||
assert!(bands.reader_bytes() <= limits.max_alloc);
|
||
assert!(u64::from(bands.width()) * u64::from(bands.height()) <= limits.max_pixels);
|
||
let mut reader = bands.reader();
|
||
let mut in_order = Vec::new();
|
||
while let Some(band) = reader.next_band().expect("a read after Bands::new accepted the stream") {
|
||
in_order.extend_from_slice(band.samples);
|
||
}
|
||
assert_eq!(in_order.len(), bands.width() as usize * bands.height() as usize * usize::from(bands.components()));
|
||
bands.find_seek_points();
|
||
let points: Vec<usize> = bands.seek_points().collect();
|
||
if points.len() < 2 {
|
||
return Some((in_order, None));
|
||
}
|
||
let mut sought = vec![0u8; in_order.len()];
|
||
let band_bytes = bands.band_bytes();
|
||
let mut reader = bands.reader();
|
||
for (k, &p) in points.iter().enumerate().rev() {
|
||
reader.seek(p).expect("a seek to an advertised seek point");
|
||
for _ in p..points.get(k + 1).copied().unwrap_or(bands.bands()) {
|
||
let band = reader.next_band().expect("a read after a seek").expect("a band after a seek");
|
||
let at = band.index * band_bytes;
|
||
sought[at..at + band.samples.len()].copy_from_slice(band.samples);
|
||
}
|
||
}
|
||
Some((in_order, Some(sought)))
|
||
}
|
||
|
||
/// A 4:2:0 baseline stream with a restart interval at every line of MCUs, from `jpeg-encoder`.
|
||
fn restart_stream() -> Vec<u8> {
|
||
let (w, h) = (48u16, 40u16);
|
||
let px: Vec<u8> = (0..usize::from(w) * usize::from(h)).flat_map(|i| [(i * 7) as u8, (i / 3) as u8, (i % 97) as u8]).collect();
|
||
let mut out = Vec::new();
|
||
let mut enc = jpeg_encoder::Encoder::new(&mut out, 80);
|
||
enc.set_sampling_factor(jpeg_encoder::SamplingFactor::R_4_2_0);
|
||
enc.set_restart_interval(3);
|
||
enc.encode(&px, w, h, jpeg_encoder::ColorType::Rgb).unwrap();
|
||
out
|
||
}
|
||
|
||
/// The band decoder on the same kind of mutations: it never panics, stays within the limits,
|
||
/// and wherever seek points survive, reading from them gives the samples reading in order does.
|
||
#[test]
|
||
fn mutated_streams_never_panic_in_bands() {
|
||
let mut rng = Rng(0xD1B5_4A32_D192_ED03);
|
||
let limits = test_limits();
|
||
let fixture = |name: &str| std::fs::read(format!("{}/{name}", env!("CARGO_MANIFEST_DIR"))).unwrap();
|
||
let inputs = [(fixture("tests/data/testorig.jpg"), 40), (fixture("tests/data/testimgari-huffman.jpg"), 40), (restart_stream(), 400)];
|
||
let (mut ok, mut errors, mut sought) = (0, 0, 0);
|
||
for (original, cases) in &inputs {
|
||
assert!(read_bands(original, &limits).is_some());
|
||
for case in 0..*cases {
|
||
let bytes = mutate(&mut rng, original);
|
||
match read_bands(&bytes, &limits) {
|
||
Some((in_order, Some(by_seeking))) => {
|
||
assert!(in_order == by_seeking, "case {case}: seeking changes the samples");
|
||
sought += 1;
|
||
ok += 1;
|
||
}
|
||
Some(_) => ok += 1,
|
||
None => errors += 1,
|
||
}
|
||
}
|
||
}
|
||
eprintln!("{ok} decoded ({sought} also by seeking), {errors} rejected");
|
||
assert!(ok > 50 && errors > 20 && sought > 50, "{ok} decoded ({sought} by seeking), {errors} rejected");
|
||
}
|