Commit Graph
2404 Commits
Author SHA1 Message Date
Dylan M. Taylor a05083e2f0 Expose original AVIF sample depth during decode 2026-11-15 17:37:40 +00:00
FredApps 9399c25341 image-codecs: decode AVIF pixels with craft-video's pure-Rust AV1 decoder
New optional `avif-decode` feature (native targets) reads still AVIF
files: the HEIF/ISOBMFF item graph, grids, clap/irot/imir, alpha
auxiliary images (including premultiplied), 8/10/12-bit 4:0:0, 4:2:0,
4:2:2 and 4:4:4, nclx/ICC colour, Exif and XMP, and PQ/HLG to linear
F32 through cicp::linearize as the HEIF path does.

The container walk follows LightCraft #3017 by @emmertex. Every size is
checked against the caller's Limits before the AV1 decoder runs, the
decoder's memory budget is capped by max_alloc, and the decode is
wrapped in catch_unwind so a decoder panic becomes an error.

Fixtures are tiny ffmpeg/libaom files; the 4:2:0 planes match libdav1d
bit for bit. Adds a decode_avif fuzz target.

Part of storytold/craft-libs#778.
2026-11-15 16:10:22 +01:00
Dylan M. Taylor e2e3b4213a Merge pull request #1585 from storytold/dot/plist-binary-v2
Extract bounded binary Apple property-list reader into craft-plist
2026-11-15 09:32:06 -05:00
Dylan M. Taylor ca64580510 Merge pull request #1577 from storytold/codex/zip-seek-boundary
Share bounded seekable ZIP indexing
2026-11-15 09:31:45 -05:00
Dylan M. Taylor 2284a6bf76 Merge pull request #1582 from storytold/extract/psd-annotations
craft-psd: bounded annotation record views and writer
2026-11-15 09:17:31 -05:00
Dylan M. Taylor 09188c3bbf Merge pull request #1565 from storytold/extract/psd-linked-views
craft-psd: bounded header-only linked records for deferred consumers
2026-11-15 08:47:29 -05:00
Dylan M. Taylor 09a962def8 Merge pull request #1575 from storytold/codex/jsonrpc-envelope
Add bounded JSON-RPC server envelopes
2026-11-15 08:37:05 -05:00
Dylan M. Taylor 5c077a7b5e Merge pull request #1574 from storytold/codex/cms-commented-cube
Share commented CUBE export serialization in craft-cms
2026-11-15 08:34:18 -05:00
Dylan M. Taylor b6d9feed23 Merge pull request #1569 from storytold/codex/shared-mcmat
Share bounded portable material archives
2026-11-15 08:06:54 -05:00
Dylan M. Taylor 4cf2a6f4cb Inspect seekable archive entry budgets and overlaps 2026-11-15 13:06:02 +00:00
Dylan M. Taylor 478c50560d Merge pull request #1448 from storytold/fix/avif-exif-hlg
fix(image-codecs): preserve AVIF EXIF and HLG levels
2026-11-15 08:02:58 -05:00
Dylan M. Taylor ac7586ed9f Merge pull request #1507 from storytold/dot/photo-ipp-shared
print: reuse direct CUPS transport with explicit app parity API
2026-11-15 08:01:42 -05:00
Dylan M. Taylor cc4fc29cdb Add bounded selective plist array lookup for locale parity 2026-11-15 08:00:09 -05:00
Dylan M. Taylor 1ec7ab92b5 Expose bounded cross-platform binary plist reader with explicit limits 2026-11-15 07:57:29 -05:00
BFlat 0bc1989610 Import plist regression from Craft Anything 15de3661 2026-11-15 07:55:30 -05:00
BFlat b2f888d788 Import plist regression from Craft Anything 15de3661 2026-11-15 07:55:30 -05:00
Afloat e20c7c1a6e Import binary plist parser changes from Craft Anything 0c386e8a 2026-11-15 07:55:30 -05:00
RRFZ d307449ee4 Import binary plist parser changes from Craft Anything 0a8cc088 2026-11-15 07:55:30 -05:00
BFlat feea4e316d Import binary plist parser changes from Craft Anything 15de3661 2026-11-15 07:55:30 -05:00
Brandon Thomas 72ee990e03 Import binary plist parser changes from Craft Anything f64ae3bf 2026-11-15 07:55:30 -05:00
Dylan M. Taylor 9fda042654 Merge pull request #1568 from storytold/codex/av1-buffered-output
Support buffered output from tiled AV1 encoder
2026-11-15 07:55:24 -05:00
Dylan M. Taylor 22c9a0933d psd: expose bounded annotation record views and serialization 2026-11-15 07:43:59 -05:00
David Mann 2c3171d734 psd: retain the non-advancing annotation length guard 2026-11-15 07:43:59 -05:00
Brandon Thomas 2af0575279 psd: import annotation reader primitives from PhotoCraft 2026-11-15 07:39:16 -05:00
Dylan M. Taylor 3e970577b9 Merge pull request #1564 from storytold/dot/shared-pctip
pcraft-format: share PCTIP1 brush-tip serialization
2026-11-15 07:29:12 -05:00
Dylan M. Taylor 163f1b2031 Merge pull request #1563 from storytold/extract/camera-exif-fields
craft-metadata: shared camera-field EXIF reader and writer for PhotoCraft cutover
2026-11-15 07:12:42 -05:00
Dylan M. Taylor 576fdd5e1a Support explicit JSON-RPC ID policies 2026-11-15 12:06:30 +00:00
Dylan M. Taylor d6c11b40b7 Share bounded seekable ZIP indexing 2026-11-15 12:00:28 +00:00
Dylan M. Taylor 6d1c6188f3 Expose validated single JSON-RPC request inspection 2026-11-15 11:56:16 +00:00
Dylan M. Taylor 10ef64f183 Merge remote-tracking branch 'origin/main' into codex/jsonrpc-envelope 2026-11-15 11:51:21 +00:00
Dylan M. Taylor e67d770c09 Add bounded JSON-RPC server envelopes 2026-11-15 11:51:16 +00:00
Dylan M. Taylor 02c639c357 cms: share commented CUBE export layout 2026-11-15 11:40:59 +00:00
Dylan M. Taylor 4780f75080 Merge remote-tracking branch 'origin/main' into fix/avif-exif-hlg
# Conflicts:
#	Cargo.lock
2026-11-15 11:39:31 +00:00
Dylan M. Taylor 26980ff331 Bound portable material detection 2026-11-15 11:38:06 +00:00
Dylan M. Taylor fe6b443acf Merge pull request #1505 from storytold/fix/svg-embedded-image-accounting
craft-svg: account for malformed embedded image data
2026-11-15 06:37:46 -05:00
Dylan M. Taylor 55b1d7f3c5 Merge pull request #1501 from storytold/extract/apng-container
craft-image-codecs: extract bounded APNG container assembly
2026-11-15 06:30:57 -05:00
Dylan M. Taylor 923ae70549 Reject duplicate manifest fields on write 2026-11-15 11:30:20 +00:00
Dylan M. Taylor 7dbfafa356 Share bounded portable material archives 2026-11-15 11:26:24 +00:00
Dylan M. Taylor 9d088c77ea Support buffered output from tiled AV1 encoder 2026-11-15 11:25:41 +00:00
Dylan M. Taylor dab6a8ea21 Flush delayed AV1 frames in HLG regression 2026-11-15 11:13:51 +00:00
keithcsqueensu 5a0caf286e craft-xlsx: .xlsb cell formulas, shared, array and dynamic arrays (ports of gridcraft#3810, #3818, #3821) (#1166)
craft-xlsx decodes `.xlsb` cell formulas, including shared, array and dynamic-array formulas, into the text an `.xlsx` file holds for them. Until now they were skipped and only their saved values kept. Part of #778.

Ported from storytold/gridcraft#3810, #3818 and #3821 (issue gridcraft#3806), my own series; the commits keep me (kiwi4boy) as their author. The three PRs are one series, each containing the one before.

## Commits

1. **The XLSB function table** (gridcraft#3810): `xlsb/ftab.rs`, [MS-XLSB] 2.5.98.10. It is craft-xls's table up to 379 (0x17B), which matches GridCraft's before the PR, plus the PR's 0x17C–0x1E4 entries (CUBE*, the Analysis ToolPak functions, IFERROR, COUNTIFS, SUMIFS, AVERAGEIF(S)). It is craft-xlsx's own copy, not shared with craft-xls: the formats follow separate specifications and their readers develop separately. They can share one table later once both settle.
2. **Cell formulas** (gridcraft#3818, with its two review fixes): `xlsb/formula.rs` is a BIFF12 decoder, separate from craft-xls's BIFF8 one, because nearly every operand differs. `xlsb/expr.rs` is a copy of craft-xls's expression tree and A1 printer, with `BinOp::precedence` added for grouping. A formula cell's `CellFormula.text` is in the file's spelling (`_xlfn.CONCAT`, `_xlpm.x`, no implicit `@`), and a `BrtCellMeta` before a cell sets `cell_metadata`, as `cm` does. The workbook's sheet links, supporting links and names are read with the workbook part, and `Package` keeps them for its sheets. A new `Limits::binary_formula_text` (128 MiB) bounds decoded text.
3. **Shared, array and dynamic-array formulas** (gridcraft#3821, with its review fixes): `PtgExp` cells, `BrtShrFmla` and `BrtArrFmla`. See the mapping below.
4. **A shared formula stops decoding copies once one is kept as its value**: a fix for CodeRabbit's finding that is still open on gridcraft#3821, so it isn't carried over.
5. **README**: what the decoder gives, what keeps its saved result, and the new limit.
6. **Review fixes**: 468 (0x1D4) is CONVERT. The specification's table has no row for it, but Excel writes CONVERT as `PtgFunc` 0x1D4. A shared formula's tokens are borrowed from the part instead of copied. A formula kept as its value loses its cell metadata.
7. **Review fixes, round 2**: a chain of strings joined with `&` is flattened only up to the decoder's depth bound (256 strings), so a balanced tree of thousands can't become a chain that overflows the stack when printed or dropped. A first cell whose range record never comes loses its cell metadata.
8. **Excel's spelling, from a second Excel round:**
   - Spaces and line breaks typed in a formula (`PtgAttrSpace`) are kept where `.xlsx` keeps them (`1 +  2`, `SUM( A1, B1 )`), and the text is trimmed as the `.xlsx` reader trims it.
   - A number is written out in full when that takes at most 21 characters without the sign, and in scientific notation otherwise (`100000000000000000000`, `1E+21`), as Excel writes it. The decoder switched at 1E+15 and 1E-9.
   - ISO.CEILING, ECMA.CEILING, NETWORKDAYS.INTL and WORKDAY.INTL are called through hidden `_xlfn.` names in `.xlsb`, but `.xlsx` spells them without the prefix.
9. **Macro sheets and spans over chart sheets:** a reference to an XLM macro sheet (`Macro1!$A$1`) is named, since macro sheets have cells. A 3-D span with a chart sheet between its ends is spelled by its ends, as `.xlsx` has it. An end with no cells still keeps the saved result.

## How it maps onto craft-xlsx's model

- A formula's text is what the `.xlsx` reader gives for the same formula. GridCraft's reading of that text stays in GridCraft: `fmla::from_file`, the implicit `@` on legacy formulas, its parser check, and Python-cell adoption.
- **Shared formulas:** every cell of the group gets `FormulaKind::Shared` with the group's `si` (counted per sheet), and the first cell also gets the range. Unlike `.xlsx`, where the other cells have empty text, each cell here gets its own text, decoded from the shared tokens at that cell. [MS-XLSB] defines it that way, with relative offsets that wrap around the grid. Shifting the first cell's text is not exact; the review of the GridCraft PR found that. `CellFormula.text`'s doc says so.
- **Arrays:** the first cell gets `FormulaKind::Array` and its range. The other cells keep their cached values and have no formula, as in `.xlsx`. A dynamic array also has its `cell_metadata`, even across a `BrtValueMeta` written in between. Clearing the other cells, and deciding from `fDynamic` whether to spill, is the consumer's call (gridcraft#3825).
- **Kept as values:** a formula that needs something the reader doesn't read keeps only its saved result, with a warning: user defined names (XLSB names aren't read yet), table references, other workbooks, `PY(index, …)` (code stored elsewhere in the package) and data tables. So do malformed tokens or records, and formulas past the allowance. A workbook whose links can't be read opens with values only, as before.

## Changed from the GridCraft PRs

- craft-xlsx's own function table, tree and printer, instead of GridCraft's formula crate and the table and helpers shared with its `.xls` decoder.
- A single-cell dynamic-array formula is read in commit 2, where #3818 left dynamic arrays to #3821: with `cell_metadata` there is nothing more to decide.
- The test of GridCraft's 8,192-character parser limit is replaced by one showing that a shared copy may be longer than its first cell. Tests that saved to `.xlsx` with GridCraft are dropped. The rest of the PRs' tests come along in `tests/xlsb.rs`, with craft-xlsx's model in the expectations.
- `Limits` gets a new public field, `binary_formula_text`. Code that builds `Limits` with `..Limits::default()` is unaffected.

## Checked against Excel

Excel saved the same workbook (`make_probe.ps1` in the GridCraft work: refs, 3-D refs, names, shared, CSE and dynamic arrays, newer functions, LET, a table, an external link, unions and intersections, array constants) as `.xlsb` and as `.xlsx`. Read through craft-xlsx, all 33 formulas this decodes from the `.xlsb` match the `.xlsx`: same text, kind, range and shared index, with the same cell metadata. The other 5 use tables, an external link or defined names, and keep their values with those warnings.

A second, wider round (commits 8 and 9) saved three more workbooks both ways:
- 487 functions Excel accepts on a worksheet: every name in the function table except 113 macro-sheet-only commands, and about 120 newer `_xlfn.` functions, LET, LAMBDA and its helpers, GROUPBY, the REGEX functions, `TRIMRANGE` and `A1:.A5`, IMAGE, RTD, CUBEVALUE and an add-in function;
- operators and precedence, unary and percent chains, unions, intersections, whole rows and columns, 3-D references and spans, quoted and cell-like sheet names, array constants, edge-magnitude numbers, a 255-character string, unicode, errors and missing arguments, legacy and dynamic-array `@`, references to a deleted sheet, column and row;
- 246 shared-formula cells, including relative references that wrap at the grid's edges, legacy array formulas, spills, spill references and a data table;
- 60 formulas typed with spaces and line breaks, and 62 number literals around Excel's switch to scientific notation.

Before commit 8, 725 formulas matched and 20 differed. Now 744 of the 745 match, with the same text, kind, range and cell metadata, and every non-first shared-formula cell matches Excel's own `Range.Formula`. The other one is a deleted column on a sheet named like a cell (`'S3'!#REF!`). That is #1330's: it prints the sheet before `#REF!`, which this PR doesn't. The spacing and number workbooks match in full. Two inline LAMBDA calls, an add-in call and a data table keep their saved results, as described above.

## Ran locally (own target directory)

- `cargo fmt -p craft-xlsx --check`: clean
- `cargo clippy -p craft-xlsx --all-targets --all-features -- -D warnings`: clean (and `cargo clippy --workspace --all-targets -- -D warnings` after commit 9)
- `cargo test -p craft-xlsx --all-features`: 74 pass, 0 fail (after commit 7); 76 after commit 9. No other workspace crate depends on craft-xlsx.
- The new tests fail without their commit's code: 5 of 6 for commit 3 (the sixth checks that unreadable shared formulas keep their values, which commit 2 already does), and 1 for commit 4. The tests for commits 8 and 9 fail without them.

## For the coordinator

Once this is merged, gridcraft#3810, #3818 and #3821 can be closed, each with a note like "Ported to craft-xlsx in storytold/craft-libs#1166; GridCraft gets it when it switches to craft-xlsx." GridCraft's remainder is the mapping. That means `from_file` and the implicit `@` (as for `.xlsx`), Python cells, clearing array cells, spilling dynamic arrays, the `xlsb-import.md` / `gaps.md` / `file-format-parity.md` notes and the engine's use of `read_xlsb_with`. I'll redo those against the switch.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

* **New Features**
  * Binary Excel workbooks now display decoded formula text in familiar spreadsheet notation, including shared and array formulas.
  * Shared-formula cells show formula text adjusted to their own cell references. Array-formula follower cells do not display separate formula text.
* **Bug Fixes**
  * When formulas cannot be decoded or exceed supported limits, cells retain their saved results; a warning may be provided.
* **Documentation**
  * Clarified formula display behavior and documented limits on formula text, token length, nesting depth, and array constants.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-15 12:11:39 +01:00
Dylan M. Taylor 2c7891db0e Adapt camera EXIF to shared TIFF and add bounded parity tests
Retain original contributor imports as authored commits. Reuse shared spans and writer, preserve field semantics, add text budget and differential controls.
2026-11-15 06:05:32 -05:00
Fabio Lussi ef3e23916f Carry PhotoCraft camera date and checked-bounds changes
Source: storytold/photocraft commit 1ccdac4c940ce43e2647f4c07fac1d61725e1f74
2026-11-15 06:05:32 -05:00
Brandon Thomas ec2b4976f5 Import original PhotoCraft camera EXIF module
Source: storytold/photocraft commit aa87012aab6d710b1e1d2ff1654a45e58c09d955
2026-11-15 06:05:32 -05:00
Dylan M. Taylor f29e82f1a7 Merge pull request #1467 from keithcsqueensu/rtf-writer-lookup-1466
craft-rtf: look up fonts, colours and authors through an index (#1466)
2026-11-15 06:03:00 -05:00
Dylan M. Taylor 390fe98b0d Merge pull request #1508 from storytold/codex/materialx-displacement-tiling
Preserve UV tiling from MaterialX displacement images
2026-11-15 06:02:13 -05:00
Dylan M. Taylor dd866b846c pcraft-format: validate tip inputs and zlib completion with parity tests 2026-11-15 05:58:40 -05:00
3401d0af8e pcraft-format: extract PhotoCraft brush tip envelope and samples
Port PhotoCraft preset_store.rs at 19aec5eb44131de44361189d64910e7b3e19e57c. Preserve original primary author and later codec contributions.

Co-authored-by: David Mann <daaavid.mann@gmail.com>

Co-authored-by: FinnNk <finnnk@gmail.com>
2026-11-15 05:58:00 -05:00
Dylan M. Taylor a8f2612c7c Test decoded HLG AVIF reference levels 2026-11-15 10:46:51 +00:00
Dylan M. Taylor 30b78d048a Preserve requested AVIF alpha quality 2026-11-15 10:41:10 +00:00