- unpack: checked sample, row-bit and output products, capped at MAX_SAMPLES before allocating
- every chunk is charged what its decoder may allocate (lossless: coded rows + 1/8; lossy: the
32-pixel MCU-padded tile) against the aggregate budget before decoding, the lossless decoder
is held to that, and truncated capacity is released
- the lossy encoded-size floor allows 64 x spp samples per two-bit block (chroma subsampling)
One opcode-list envelope implementation: opcode_raw::Records walks the records (stopping at a
truncated one) for both the strict opcode_raw::parse and the typed opcodes::parse_list, and
opcodes::write_list encodes through opcode_raw::encode instead of truncating lengths with as.
- uncompressed rows go through crate::unpack (read_u16s / unpack_msb), as in dng.rs
- lossy JPEG tiles (34892) use tiffraw::lossy_jpeg_with_tile (zune-jpeg) with its tile-size checks
- line-dng now implies dng instead of pulling in image
- tests: every layout and compression read against the full reader, refusals, unpack depths
Problem: DNG opcode lists were ignored, so lens shading gain maps
(phones and many cameras), bad pixel maps, vignette and distortion
corrections and trims that a file asks for were silently left out,
and an opcode a reader must apply could not be refused.
Fix (DNG 1.7.1.0 chapter 7): a big-endian opcode list parser (at most
256 opcodes, every count checked against the parameter bytes before
allocating) and the opcodes WarpRectilinear, WarpFisheye,
WarpRectilinear2 (with the skip rule), FixVignetteRadial,
FixBadPixelsConstant / List, TrimBounds, MapTable, MapPolynomial,
GainMap, DeltaPerRow / Column and ScalePerRow / Column. OpcodeList1 runs
on the stored samples (0-65535), OpcodeList2 on the linear values
before white balance (the fused normalise splits only then),
OpcodeList3 on the demosaiced camera values; values are clipped after
each opcode. Unknown opcodes or ones newer than 1.7.1.0 are skipped
when optional and an error naming them when mandatory; preview-only
opcodes are skipped on the reduced-resolution path, and a mandatory
OpcodeList3 keeps full resolution. Interpretations where the spec
leaves room are recorded in the README. Files without opcode lists
develop exactly as before.
Tests: unit tests per opcode with known results (tables, polynomials,
deltas, scales, gain map interpolation and edge replication, vignette,
warps reproducing a ramp, fisheye, the skip rule, bad pixels, trim,
hostile parameters) and whole-file tests of each list's stage, the
optional flag, the preview path, and 3000 mutations under catch_unwind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
unpack accepted a bit depth of 0: the input requirement became zero bytes,
so one source byte produced a header-sized buffer of zero samples. Depths
outside 1..=16 (the u16 output range) are now rejected up front.
- raw: a lossy JPEG tile must have as many components as SamplesPerPixel.
- process::develop_with: use the opcode lists only when apply_opcodes is set.
- writer: lossy_jpeg returns an error instead of an empty chunk or an encoder panic; new try_write rejects unsupported sample counts and bits/CFA options for lossy JPEG.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Problem: lossy DNGs (DNG 1.4 Compression 34892, used for proxy and
"lossy compressed" DNGs, 8-bit LinearRaw tiles usually with a
LinearizationTable) were refused as unsupported.
Fix: decode each tile or strip with the baseline JPEG decoder the
workspace already depends on (`image` with only its jpeg feature, as in
crates/codecs and crates/media; no new crate in Cargo.lock, builds for
wasm32), one component as greyscale or three as colour, the samples
filling the chunk in raster order as with lossless JPEG. As the spec
allows it only for 8-bit LinearRaw, CFA or other depths are refused with
a message naming lossy JPEG. The decoder runs under catch_unwind with an
allocation limit, its picture must match the chunk, and a tiny file
cannot claim a huge picture (at least two bits per 8 x 8 block). Info
reports "lossy JPEG". The test writer gains a lossy JPEG path through
`image`'s encoder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: a 4096 x 2160 12-bit lossless-JPEG CinemaDNG frame took about
50 ms (tiles) to 115 ms (one strip) on 16 threads and about 200 ms on
one, too slow for smooth 4K playback, and every frame was developed at
full size even for small monitors and proxies.
Fix, without changing the output (f32 bit patterns compared):
- Lossless JPEG: one 13-bit lookup gives a code and its difference
bits together; the bit reader takes eight bytes at once when they
hold no 0xFF; per-predictor line loops.
- Tiles decode in parallel by bands straight into place; a single
strip covering the image is used without a copy.
- Normalisation: without black-level deltas, a lookup table per
CFA / black-level site replaces the per-sample float arithmetic
(each entry computed by the same expression).
- Demosaic, crop and colour run row by row from the normalised plane
(no padded copy, no RGB intermediate), with 5 x 5 windows sliding
along the five mirrored rows; the bilinear (Fast) Bayer demosaic
sums in the reference order.
- ProcessOptions::target_size now develops Bayer, LinearRaw and mono
pictures at 1/2, 1/4 or 1/8 size (the smallest covering the
target) by averaging each colour over the blocks, without demosaic.
The first version is kept as `reference` (feature `writer`, tests and
benchmarks only): tests/equivalence.rs checks identical raw samples and
developed bits over all Bayer phases, both qualities, odd crops,
active areas, linearisation, black deltas, codes above the bit depth,
LinearRaw, mono, other CFAs and tiny pictures; examples/bench.rs times
both interleaved at 1/4/16 threads. Best of 9 under load ~12: tiles
2.5x (1 thread) to 2.9x (16), one strip 2.1-2.6x; half size 3-4x.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* dng: bound lossless-JPEG decoding by the data it has and reject shared tile bytes (#2776)
Remaining amplification holes after the decoded-sample cap: the per-chunk
decode cap had 256 Ki samples of slack on top of the chunk's size, a patched
frame header could ask for far more samples than the entropy-coded bytes can
hold (zero bits were fed past the end), and strips/tiles pointing at the same
bytes were each decoded in full.
ljpeg::decode now fails with Truncated when width*height*components needs more
than one bit per sample of the remaining data, the caller caps at exactly the
chunk's sample count, and raw::chunks rejects overlapping strip/tile ranges.
Tested with a tiny lj92 stream whose SOF is patched to 512x512 and a tiled
file whose second tile reuses the first tile's bytes.
Fixes#2776
* dng: keep the slack for a lossless JPEG coded larger than its chunk
The entropy-length bound already caps the allocation; a last strip coded at
full RowsPerStrip holds more samples than the chunk needs and is truncated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Problem: camera RAW is at 0% (docs/codec-parity.md, #383). CinemaDNG,
the one cinema RAW format with a public specification, could not be
read at all.
Fix: new L0 crate filmcraft-dng, written from the public DNG 1.4-1.7,
TIFF 6.0 / TIFF-EP, ITU-T T.81 and CinemaDNG specifications only (no
other decoder's source read; editions in the crate README):
- TIFF container with SubIFDs, bounded (64 IFDs, 4 levels, cycles end
the walk) and checked offsets; the main raw IFD is the largest
NewSubFileType 0 CFA / LinearRaw image.
- Raw data: 1-16-bit uncompressed (MSB-first packing) and lossless JPEG
(process 14, predictors 1-7, 1-4 components, point transform, line
restarts), strips and tiles, tiles decoded in parallel; allocations
capped by size and by the data present.
- Processing per the DNG model: ActiveArea, LinearizationTable, per-site
black levels with deltas, white levels, as-shot white balance with
neutral highlight clipping, Malvar-He-Cutler Bayer demosaic (bilinear
for other patterns), DefaultCrop, ColorMatrix / CameraCalibration /
ForwardMatrix with illuminant interpolation, Bradford to linear
Rec. 709, BaselineExposure; output scene-linear RGBA floats.
- CinemaDNG FrameRate, TimeCodes and ReelName tags.
- A test-only writer (feature `writer`) synthesises DNGs. Tests: raw
samples bit-exact for every depth / predictor / layout, processing
against hand-computed values, hostile files (every truncation, 12k
mutations, cyclic IFDs, huge sizes, bad offsets), and LibRaw's
unprocessed_raw as an external oracle reading the same samples
(skipped when not installed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extract the format portion of LightCraft cc3eb1c8 (lossy DNG export) preserving tile/quality clamps and RGB edge replication. Application rendering/resize/transfer policy stays in the consumer. Callback ownership and fallible allocation integration by Dylan M. Taylor.
Reuse existing writer and image layout validation rather than cloning RawImage samples or importing another wire writer. Metadata matrices are value copies; profile/opcode tables and pixel data remain borrowed.
Port reusable DCP wire parsing/writing and synthetic tests from LightCraft e87f92fe crates/raw/src/dcp.rs, originally introduced by OneSlip in 8972565249b496ccdf3a7f23321c84622615a353. Reuse existing craft-raw DNG color/profile tag reader and craft-tiff writer instead of copying helpers. App applied_to policy stays in LightCraft.
DNG chapter 7 runs OpcodeList1 on the stored codes, then the
LinearizationTable maps them to sensor values. decode was applying the
table first, so a develop pipeline that later applied list 1 saw already
linearized samples.
When list 1 is present the table is now carried on RawImage and left
unapplied (`linearized` stays false). apply_linearization maps it after
the list. Files without list 1 are unchanged.
The dng-only build already compiles on main (d16ee792 gated
cr3::orientation). A comment at that gate records the E0433 failure
mode.
Part of #778.
A tile's QP map error is reported by its first plane job, so a later tile's
corrupt map no longer wins over an earlier tile's plane error. Equivalence
tests also compare error texts; the corpus test fails when the folder holds
none of the pinned files; docs/cr3.md says when the sensor buffer is allocated.
Part of #1654
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The vertical lifting combined strided columns one at a time; it now lifts
whole rows, per pair of output rows in parallel, with the same integer
equations and edge rules (i64 floor shifts equal the div_euclid it replaces).
The horizontal pass runs its rows in parallel and drops the Result per sample;
overflow is still an error. Differential tests compare both passes with the
previous implementation for every size 1..=17 and every edge combination.
Part of #1654
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every plane of every tile is one parallel task that ends in sensor samples;
the mosaic is then filled per output row pair in parallel, without a modulo,
division, bounds check or Result per sample. The first error in tile and
plane order is still the one reported, and the sensor buffer is allocated
only after every plane has decoded. The CC0 corpus test also pins the four
EOS R5 Mark II files and skips files that are not present.
Part of #1654
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>