Commit Graph
58 Commits
Author SHA1 Message Date
FredApps 1a83fd70a3 Merge pull request #1761 from storytold/extract/dcp-wire
craft-raw: extract DCP camera-profile reader and writer
2026-11-16 01:30:25 +01:00
FredApps 920c745f42 raw: initialize linearization_table in the dngwrite test image 2026-11-16 01:13:47 +01:00
Dylan M. Taylor c599f64832 Merge pull request #1662 from vangel76/perf/raw-ljpeg-nef-1655
CR2, DNG, ARW lossless-JPEG and NEF/RW2 Huffman raw decoding is up to 2x as fast
2026-11-15 18:49:54 -05:00
FredApps b24a1ee25c Merge pull request #1745 from storytold/fix/raw-dng-lin-before-opcodes
raw: leave LinearizationTable unapplied when OpcodeList1 is present
2026-11-16 00:44:49 +01:00
Dylan M. Taylor f60aec302b Merge pull request #1659 from vangel76/perf/raw-dng-parallel-copy-1651
DNG files decode up to 2.4x faster on many cores (5x uncompressed)
2026-11-15 18:27:55 -05:00
Dylan M. Taylor ba240b89a3 Merge pull request #1677 from vangel76/perf/raw-raf-1656
Fujifilm compressed RAF files decode about 1.4x as fast
2026-11-15 18:25:44 -05:00
Dylan M. Taylor 4608875afb Merge pull request #1683 from vangel76/perf/raw-cr3-1654
Canon C-RAW (CR3) files decode about 1.5x as fast, lossless CR3 up to 1.4x
2026-11-15 18:23:39 -05:00
Dylan M. Taylor 7bc122ed0b fix(raw): validate serialized DCP first matrix with reader rules 2026-11-15 17:51:42 -05:00
Dylan M. Taylor 42d8933a29 fix(raw): reject DCP writes without the first color matrix 2026-11-15 17:36:40 -05:00
Dylan M. Taylor ffb280d65e test(raw): preserve DCP profile boundaries and shot-field exclusion 2026-11-15 16:34:17 -05:00
OneSlip 39578d914c craft-raw: share DNG camera profile file reader and writer
Port reusable DCP wire parsing/writing and synthetic tests from LightCraft e87f92fe crates/raw/src/dcp.rs, originally introduced by OneSlip in 8972565249b496ccdf3a7f23321c84622615a353. Reuse existing craft-raw DNG color/profile tag reader and craft-tiff writer instead of copying helpers. App applied_to policy stays in LightCraft.
2026-11-15 16:33:42 -05:00
Dylan M. Taylor da591dc6b2 fix(raw): bound LJ92 writer tile padding before allocation 2026-11-15 16:21:37 -05:00
FredApps a99c8f8e36 raw: leave LinearizationTable unapplied when OpcodeList1 is present
DNG chapter 7 runs OpcodeList1 on the stored codes, then the
LinearizationTable maps them to sensor values. decode was applying the
table first, so a develop pipeline that later applied list 1 saw already
linearized samples.

When list 1 is present the table is now carried on RawImage and left
unapplied (`linearized` stays false). apply_linearization maps it after
the list. Files without list 1 are unchanged.

The dng-only build already compiles on main (d16ee792 gated
cr3::orientation). A comment at that gate records the E0433 failure
mode.

Part of #778.
2026-11-15 21:26:02 +01:00
vangel76andClaude Opus 5.5 4904d6367a test(raw): a corrupt CR3 QP map is reported as a QP error, not a plane error
Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:35:06 +01:00
vangel76andClaude Opus 5.5 ee87ab6c9e style(raw): rustfmt and clippy for the CR3 decoder changes
Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:33:35 +01:00
vangel76andClaude Opus 5.5 68181b7d15 fix(raw): keep tile-then-plane error order for QP maps; doc and test review points
A tile's QP map error is reported by its first plane job, so a later tile's
corrupt map no longer wins over an earlier tile's plane error. Equivalence
tests also compare error texts; the corpus test fails when the folder holds
none of the pinned files; docs/cr3.md says when the sensor buffer is allocated.

Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:31:35 +01:00
vangel76andClaude Opus 5.5 308e5c8519 test(raw): name the plane type in the interleave test
Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:27:31 +01:00
vangel76andClaude Opus 5.5 7566f61bad perf(raw): run CRX wavelet synthesis on whole rows, in parallel
The vertical lifting combined strided columns one at a time; it now lifts
whole rows, per pair of output rows in parallel, with the same integer
equations and edge rules (i64 floor shifts equal the div_euclid it replaces).
The horizontal pass runs its rows in parallel and drops the Result per sample;
overflow is still an error. Differential tests compare both passes with the
previous implementation for every size 1..=17 and every edge combination.

Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:20:39 +01:00
vangel76andClaude Opus 5.5 9ea6cf1ec3 perf(raw): decode all CR3 tiles in parallel and interleave them by row pairs
Every plane of every tile is one parallel task that ends in sensor samples;
the mosaic is then filled per output row pair in parallel, without a modulo,
division, bounds check or Result per sample. The first error in tile and
plane order is still the one reported, and the sensor buffer is allocated
only after every plane has decoded. The CC0 corpus test also pins the four
EOS R5 Mark II files and skips files that are not present.

Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:18:47 +01:00
vangel76andClaude Sonnet 5.5 40e7af21d6 Record corpus hashes and cover every quantizer table in the differential test
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 18:18:37 +01:00
vangel76andClaude Opus 5.5 ae11e0ae47 perf(raw): decode C-RAW subbands in parallel, one quantization step per band or QP cell
The ten subbands of a plane are separate entropy streams; the first error in
band order is still the one reported. The step is looked up once per band, or
once per QP cell for adaptive gains, instead of once per coefficient.

Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:16:45 +01:00
vangel76andClaude Opus 5.5 1d950d5597 perf(raw): read CRX entropy streams through a 64-bit bit cache
One 8-byte load per ~56 bits instead of a checked 5-byte window per read;
the Rice prefix and its low bits come from one cached word. Escapes and the
end of the stream keep the bit-by-bit path, so results and errors match.

Part of #1654

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 18:15:56 +01:00
vangel76andClaude Sonnet 5.5 df89f6b6b7 craft-raw: test tables where code plus extra bits fill the lookup, canonical reference, clippy allow
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 18:13:19 +01:00
vangel76andClaude Sonnet 5.5 daed8c7212 Fujifilm compressed RAF decodes about 1.2x as fast
k from leading_zeros, neighbour reads from row slices of one reused
colour-vector buffer, per-quantizer coding parameters computed once,
placement through a precomputed map. Output unchanged; adds a
differential test against the previous decoder and an optional corpus test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 18:09:15 +01:00
vangel76andClaude Sonnet 5.5 fed17b6bb3 craft-raw: one lookup per Huffman symbol and a fixed predictor per run in LJ92 and NEF/RW2 decoding
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 18:08:34 +01:00
vangel76andClaude Sonnet 5.5 ed3f746bcd craft-raw: test odd chunk placement, fix strip chunk sizes in layout tests
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 18:08:14 +01:00
vangel76andClaude Sonnet 5.5 911e3ae315 craft-raw: place DNG chunks into the image in parallel, reuse row buffers
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 18:04:55 +01:00
Dylan M. Taylor d16ee7926c raw: share line-restart lossless JPEG decoder 2026-11-15 16:36:35 +00:00
Dylan M. Taylor ce608a6405 test(raw): omit CFA tag in CR2 fallback regression 2026-11-14 23:10:28 +00:00
Dylan M. Taylor b5778c4f20 test(raw): cover missing-tag CR2 gradient decode 2026-11-14 22:52:35 +00:00
Dylan M. Taylor dd692d907e fix(raw): keep missing-tag CR2 CFA conservative 2026-11-14 22:22:35 +00:00
Dylan M. Taylor 465fa74789 fix(metadata,raw,svg): address CodeRabbit review findings 2026-11-14 21:31:04 +00:00
Terra854 db82f95711 Merge upstream main while preserving RAW import safeguards 2026-11-15 03:57:39 +08:00
Terra854 58ef208c13 fix: preserve RAW identity and SVG byte-limit reporting 2026-11-15 03:49:35 +08:00
Terra854 eec8408c53 fix: bound CR3 sample tables and accept supported RAF Bayer layouts 2026-11-14 22:47:42 +08:00
Terra854 15cd3900e1 fix: preserve RAW samples and reject malformed codec metadata 2026-11-14 22:35:52 +08:00
Terra854 176810ad2e fix: validate RAW framing and preserve decoder metadata 2026-11-14 21:49:09 +08:00
Alex Chambers 5151457b99 craft-raw: Nikon Z 6 and Z 7 in-file distortion; modes 1 and 3 only
Ported from storytold/lightcraft#3796. The Z 6 and Z 7 join the bodies whose
NikonNEFInfo distortion becomes an OpcodeList3 warp. Three Z 6 and three Z 7
CC0 raw.pixls.us files (NIKKOR Z 24-70mm f/4 S) were matched against the
camera's embedded JPEG: the corrected radial median error is 0.88-0.96 px on
the Z 6 and 1.37-1.46 px on the Z 7, from 83-84 px and 124-125 px uncorrected.
The method and results are in crates/raw/docs/nikon-lens-corrections.md.

DistortionInfo's mode byte must now be 1 (On, optional) or 3 (On, required).
Mode 2 (Off) and unknown modes give no warp, where before only 0 was skipped,
so mode-2 files from the Z 6II, Z 8 and Z f no longer get a warp.
2026-11-14 12:20:57 +01:00
Dylan M. Taylor eed3c28cf0 Reject truncated unsupported DNG samples 2026-11-13 05:19:00 -05:00
Dylan M. Taylor 6417d6b90a Preserve shared format boundary behavior 2026-11-13 04:49:49 -05:00
Dylan M. Taylor b9992dbebd Canonicalize DNG CFA at sensor origin 2026-11-13 02:56:57 -05:00
Dylan M. Taylor c6ae974743 Expose shared decoded DNG JPEG XL previews 2026-11-13 01:09:49 -05:00
Dylan M. Taylor 8bac629c0f Preserve CFA anchor in shared raw metadata 2026-11-13 00:46:33 -05:00
Dylan M. Taylor 80144afd9c craft-raw: count a TIFF image's chunks from its offsets table
craft-tiff caps chunk descriptors at the file's length (storytold/craft-libs#827), so a grid of tiles sharing one payload lists fewer chunks than the image declares. Comparing that capped list against the grid refused a file shorter than its tile count for "missing chunks" instead of for the data it actually holds. Compare the offsets table, which is the source of truth, and cover the short-file case.
2026-11-12 06:22:37 -05:00
eca919d7a6 craft-raw: where black and white levels came from (LevelSource) (#1073)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW

A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).

The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.

Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.

* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)

RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.

Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.

* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C

Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.

ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.

* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding

Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.

* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over

Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.

* craft-raw: test for the Deflate padding bound

* craft-raw: Pentax PEF and Samsung SRW

Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.

* craft-raw: Canon CR2, with sRAW and mRAW

Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).

* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved

Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).

* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)

* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level

* craft-raw: Panasonic RW2 / RWL / RAW, every raw format

* craft-raw: RW2 format 8 refuses strips stating fewer bits than samples

Ported from storytold/lightcraft#3910.

* craft-raw: RW2 format 8 refuses samples above the declared bit depth

Ported from storytold/photocraft#11838.

* craft-raw: the RW2 format 8 robustness seed carries a valid stream

Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes.

* craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections

* craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits

Ported from storytold/lightcraft#3910.

* craft-raw: test RAF orientation and camera from the embedded JPEG's Exif

* WIP craft-raw: Olympus ORF

* craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview)

* craft-raw: Olympus ORF, the uncompressed layouts, with its maker-note preview and WB_GLevel

* WIP craft-raw: Sony ARW

* craft-raw: tiled lossless ARW decodes hold one group of tiles at a time

Ported from storytold/lightcraft#3910.

* craft-raw: dark 12-bit uncompressed ARWs keep Sony's 14-bit levels, and the DNG writer's depth holds the white level

Ported from storytold/lightcraft#3429.

* craft-raw: Sony ARW family, docs, seeds and the derivation records

* WIP craft-raw: Canon CR3

* craft-raw: Canon CR3 (CRX) through craft-metadata's container, header plans, CR3 previews and the coverage records

* craft-raw: say where a raw's black and white levels came from (LevelSource)

Ported from storytold/lightcraft#3319.

* craft-raw: decode_format builds with every format feature off

---------

Co-authored-by: radialmonster <radial@gmail.com>
Co-authored-by: wnanbei <wnanbei@gmail.com>
Co-authored-by: lownlyrider <lownlyrider@gmail.com>
Co-authored-by: Julien LEQUEN <jlequen@crypt-0n.fr>
2026-11-11 22:10:30 -05:00
e9e1da3a58 craft-raw: Canon CR3 (CRX), header plans and the remaining preview branches (#1070)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW

A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).

The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.

Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.

* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)

RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.

Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.

* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C

Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.

ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.

* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding

Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.

* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over

Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.

* craft-raw: test for the Deflate padding bound

* craft-raw: Pentax PEF and Samsung SRW

Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.

* craft-raw: Canon CR2, with sRAW and mRAW

Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).

* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved

Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).

* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)

* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level

* craft-raw: Panasonic RW2 / RWL / RAW, every raw format

* craft-raw: RW2 format 8 refuses strips stating fewer bits than samples

Ported from storytold/lightcraft#3910.

* craft-raw: RW2 format 8 refuses samples above the declared bit depth

Ported from storytold/photocraft#11838.

* craft-raw: the RW2 format 8 robustness seed carries a valid stream

Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes.

* craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections

* craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits

Ported from storytold/lightcraft#3910.

* craft-raw: test RAF orientation and camera from the embedded JPEG's Exif

* WIP craft-raw: Olympus ORF

* craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview)

* craft-raw: Olympus ORF, the uncompressed layouts, with its maker-note preview and WB_GLevel

* WIP craft-raw: Sony ARW

* craft-raw: tiled lossless ARW decodes hold one group of tiles at a time

Ported from storytold/lightcraft#3910.

* craft-raw: dark 12-bit uncompressed ARWs keep Sony's 14-bit levels, and the DNG writer's depth holds the white level

Ported from storytold/lightcraft#3429.

* craft-raw: Sony ARW family, docs, seeds and the derivation records

* WIP craft-raw: Canon CR3

* craft-raw: Canon CR3 (CRX) through craft-metadata's container, header plans, CR3 previews and the coverage records

---------

Co-authored-by: radialmonster <radial@gmail.com>
Co-authored-by: wnanbei <wnanbei@gmail.com>
Co-authored-by: lownlyrider <lownlyrider@gmail.com>
2026-11-11 22:01:07 -05:00
2eddd3073e craft-raw: Sony ARW, ARW6, SR2 and lens corrections (#1051)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW

A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).

The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.

Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.

* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)

RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.

Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.

* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C

Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.

ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.

* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding

Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.

* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over

Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.

* craft-raw: test for the Deflate padding bound

* craft-raw: Pentax PEF and Samsung SRW

Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.

* craft-raw: Canon CR2, with sRAW and mRAW

Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).

* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved

Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).

* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)

* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level

* craft-raw: Panasonic RW2 / RWL / RAW, every raw format

* craft-raw: RW2 format 8 refuses strips stating fewer bits than samples

Ported from storytold/lightcraft#3910.

* craft-raw: RW2 format 8 refuses samples above the declared bit depth

Ported from storytold/photocraft#11838.

* craft-raw: the RW2 format 8 robustness seed carries a valid stream

Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes.

* craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections

* craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits

Ported from storytold/lightcraft#3910.

* craft-raw: test RAF orientation and camera from the embedded JPEG's Exif

* WIP craft-raw: Olympus ORF

* craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview)

* craft-raw: Olympus ORF, the uncompressed layouts, with its maker-note preview and WB_GLevel

* WIP craft-raw: Sony ARW

* craft-raw: tiled lossless ARW decodes hold one group of tiles at a time

Ported from storytold/lightcraft#3910.

* craft-raw: dark 12-bit uncompressed ARWs keep Sony's 14-bit levels, and the DNG writer's depth holds the white level

Ported from storytold/lightcraft#3429.

* craft-raw: Sony ARW family, docs, seeds and the derivation records

---------

Co-authored-by: radialmonster <radial@gmail.com>
Co-authored-by: wnanbei <wnanbei@gmail.com>
Co-authored-by: lownlyrider <lownlyrider@gmail.com>
2026-11-11 21:42:48 -05:00
Dylan M. Taylor b20e7370a6 craft-raw: mrw::parse, the MRW block container as public API (#1052)
The MRW block reader (blocks by name and the sensor data offset) was
crate-private, so a metadata reader that wants only the TTW Exif block had
to keep its own copy. craft_raw::mrw::{parse, Mrw, MAX_BLOCKS} are now
public, behind the existing mrw feature; decoding is unchanged. A new test
covers the block limit.
2026-11-11 21:34:56 -05:00
a7a00c7b0a craft-raw: Olympus ORF (uncompressed) (#1032)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW

A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).

The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.

Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.

* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)

RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.

Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.

* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C

Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.

ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.

* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding

Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.

* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over

Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.

* craft-raw: test for the Deflate padding bound

* craft-raw: Pentax PEF and Samsung SRW

Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.

* craft-raw: Canon CR2, with sRAW and mRAW

Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).

* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved

Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).

* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)

* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level

* craft-raw: Panasonic RW2 / RWL / RAW, every raw format

* craft-raw: RW2 format 8 refuses strips stating fewer bits than samples

Ported from storytold/lightcraft#3910.

* craft-raw: RW2 format 8 refuses samples above the declared bit depth

Ported from storytold/photocraft#11838.

* craft-raw: the RW2 format 8 robustness seed carries a valid stream

Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes.

* craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections

* craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits

Ported from storytold/lightcraft#3910.

* craft-raw: test RAF orientation and camera from the embedded JPEG's Exif

* WIP craft-raw: Olympus ORF

* craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview)

* craft-raw: Olympus ORF, the uncompressed layouts, with its maker-note preview and WB_GLevel

---------

Co-authored-by: radialmonster <radial@gmail.com>
Co-authored-by: wnanbei <wnanbei@gmail.com>
2026-11-11 20:48:30 -05:00
b471b84fd4 craft-raw: Fujifilm RAF (#1016)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW

A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).

The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.

Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.

* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)

RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.

Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.

* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C

Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.

ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.

* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding

Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.

* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over

Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.

* craft-raw: test for the Deflate padding bound

* craft-raw: Pentax PEF and Samsung SRW

Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.

* craft-raw: Canon CR2, with sRAW and mRAW

Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).

* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved

Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).

* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)

* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level

* craft-raw: Panasonic RW2 / RWL / RAW, every raw format

* craft-raw: RW2 format 8 refuses strips stating fewer bits than samples

Ported from storytold/lightcraft#3910.

* craft-raw: RW2 format 8 refuses samples above the declared bit depth

Ported from storytold/photocraft#11838.

* craft-raw: the RW2 format 8 robustness seed carries a valid stream

Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes.

* craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections

* craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits

Ported from storytold/lightcraft#3910.

* craft-raw: test RAF orientation and camera from the embedded JPEG's Exif

* craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview)

---------

Co-authored-by: radialmonster <radial@gmail.com>
Co-authored-by: wnanbei <wnanbei@gmail.com>
2026-11-11 20:33:51 -05:00