Commit Graph
658 Commits
Author SHA1 Message Date
Michael Higgins f51ce3cfcc fix(render): bound the objects lopdf's fallback parses from object streams and arrays (#5083)
## What this changes

lopdf's fallback loader (used by `inspect` when the lazy reader fails, and always by `attachment_data`) parsed every object stream eagerly, capping only decoded bytes. A stream of `0 0 0 …` inside the 256 MiB limit built ~120M `Object`s (10+ GB) and aborted. This bounds what it builds:

- **Per top-level object:** at most 4,194,304 objects (`parser::MAX_OBJECT_NODES`, the same as cos's `MAX_NODES`), counting every array element and dictionary value at any depth. One budget is threaded through `array` / `_dictionary` / `_direct_object`. An object is charged once it is built, so failed `alt` branches cost nothing, and an exhausted budget is a nom `Failure`.
- **Per object stream:** at most 1,000,000 index entries (cos's `/N` clamp), and the index list is no longer allocated from the whole block.
- **Per load:** at most 8,388,608 objects across all object streams, in an atomic `NodeBudget` shared by rayon workers. A member reserves up to its own byte length (every object takes at least one byte) and refunds what it doesn't use, so parallel members can't hold the budget between them. Once the budget is spent, the remaining members are skipped with a warning.
- **`convert` in `crates/render/src/structure.rs`:** the depth limit is now 128, one level per array or dictionary, matching cos's parser. Before, it was 100 with two levels per dictionary, so ~51 nested dictionaries alone forced the lopdf fallback.

Logged as lopdf patch (7) in `vendor/README.md`.

Fixes #5083

## How you verified it

`cargo fmt --check` on `pdfcraft-render` and `cargo clippy -p pdfcraft-render --all-targets -- -D warnings` (which compiles the vendored lopdf). Regression tests in `crates/render/src/inspect/tests.rs`, where lopdf patches are tested in-tree (the vendored lopdf's own tests need its missing `assets/example.pdf`):

- `object_stream_members_over_the_node_cap_are_skipped` (croquete1): a member that is one array of 4.3M elements is skipped at the per-object cap; a small member is kept.
- `object_stream_index_entries_past_the_clamp_are_ignored`: of 1,000,001 distinct members, the one past the 1,000,000-entry clamp is never built.
- `object_streams_of_one_load_share_their_node_budget`, `object_streams_loaded_in_parallel_share_their_node_budget` and `a_malformed_object_stream_member_refunds_its_reservation` (Harrison-Booth): a small shared `NodeBudget` bounds the members of several streams, serially and across threads, and a malformed member refunds what it didn't build. `NodeBudget` and `ObjectStream::new_within` are now public for this.

- [x] `cargo fmt --check`
- [x] clippy with `-D warnings` on the crates I changed
- [x] tests pass on the crates I changed (`cargo test -p pdfcraft-render --lib object_stream`: 9 passed, including the five new tests)

## Never-crash

- [x] No new `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!` outside tests
- [x] No indexing or slicing with input-derived positions; the budget uses `checked_sub` and a `min`-bounded reservation; index entries and objects built are capped; nesting stays bounded (128 in `convert`)
- [x] No `unsafe`
- [x] The abort this fixes has synthetic regression tests (above)

## Provenance

- [x] No assets, fonts or copied code; the limits follow `pdfcraft_cos`'s own caps

## Security

- [x] No change to `--root`, MCP, the control channel or network access

## Parity and reach

- Not applicable: an internal robustness fix to the fallback loader, with no user-facing feature or parity entry. The vendored-patch log (`vendor/README.md`, lopdf (7)) is updated.

## Anything a reviewer should look at first

- `vendor/README.md` now lists the tests above for patch (7).
- Ordinary (not compressed) indirect objects are capped at 4M objects each but aren't charged to the per-load budget; their total is bounded only by file size.
- `get_compressed_object` still decodes and re-parses the whole stream per lookup (pre-existing); it now gets one object's budget.
- Objects nested 101–128 deep now reach the Inspector through the lazy reader rather than being skipped by the fallback. The Inspector's own walkers cap depth at 32 or less.


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
  * Added limits to PDF object parsing and object-stream processing. Loading now parses only selected members, while lookups parse only the requested member. Members that exceed available limits are skipped in a deterministic order.
  * Increased the supported nesting depth for dictionary and object conversion, allowing more deeply nested content to be processed.
  * Documented that conversion failures can trigger eager loading as a fallback.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 07:48:14 +01:00
FredApps e03d5a1c29 Render: an isolated CMYK group of any size blends in CMYK, in bands (#6885)
* Render: an isolated CMYK group of any size blends in CMYK, in bands

hayro patch (27) blended an isolated DeviceCMYK transparency group in CMYK only up to 2^24
device pixels (a 4096 x 4096 square); a larger one was blended in RGB. Print as image renders a
whole page at once, up to 8192 px on the long side, so a page enlarged to 11 x 17 in at 600 dpi
(about 43 MP) blended any group covering most of it in RGB, and a print test file's CMYK blend
patches drew their X's. The same applied to exporting a large page as an image.

A larger group is now blended in bands of whole rows, each within the old limit: both ink passes
and the tint conversion run per band, so their memory stays as before, and only the finished
group (4 bytes a pixel) is kept whole. Every band is blended before any is drawn, so a group a
pass has no tints for is still drawn as before, whole.

The regression test lowers the band to 7 rows (set_cmyk_band_pixels, hidden, per thread like
set_skip_offscreen_paths) and checks the banded group matches the whole one across an
anti-aliased edge, and is CMYK's green where magenta is under black with Difference.

* Test: CMYK green through the profile is (0, 166, 81); check green against RGB's magenta
2026-11-10 18:11:33 +01:00
regendorf-ux b7cfbd3980 Find and replace text in Edit text mode: a Replace row in the Find box, Replace all as one undo step (#2467) 2026-11-10 15:19:55 +07:00
Hazel Granados 45d51d8c46 M3: a file named with open parameters opens at their destination (#3425) (#5049) 2026-11-10 14:23:54 +07:00
Piohrun e2464e1440 M2: a JBIG2 mask with a large symbol dictionary draws, and one a decoder limit refuses takes its picture with it (#6722) (#6784) 2026-11-10 10:43:13 +07:00
bflatastic 36e7121ab0 Merge main into merge/ui-print-optimize2
# Conflicts:
#	crates/render/src/inspect.rs
#	crates/render/src/lib.rs
2026-11-09 21:16:23 +07:00
bflatastic 2ce2d3cf81 Merge main into merge/ui-print-optimize2
# Conflicts:
#	crates/automation/src/redact.rs
#	crates/automation/src/tools.rs
#	crates/print/src/lib.rs
#	crates/render/src/inspect.rs
#	crates/render/src/lib.rs
2026-11-09 20:56:15 +07:00
bflatastic 856d3fb0ac Merge main into merge/comments-annot2
# Conflicts:
#	crates/render/src/inspect.rs
#	crates/render/src/lib.rs
#	crates/ui-egui/src/canvas/page_pointer/tests.rs
#	crates/ui-egui/tests/it/comments.rs
2026-11-09 20:56:09 +07:00
BFlat b6cbf9bb30 Merge pull request #6710 from storytold/merge/forms-links-text2
Land forms, links, find and redaction-save fixes (#4496, #4532, #3567, #3773, #4470, #4474, #4497, #5658)
2026-11-09 19:45:39 +08:00
bflatastic c994a0a4ac Fit the landed changes together
Comment literals in tests gain the callout text box and pop-up fields, and the review-state map is formatted.
2026-11-09 18:29:25 +07:00
bflatastic 3ec0a629d4 Merge #5508: M12: edit attachment descriptions
# Conflicts:
#	crates/automation/src/dispatch.rs
#	crates/automation/src/json.rs
#	crates/automation/src/tools.rs
#	crates/engine/src/edit.rs
#	crates/engine/src/edit/labels.rs
#	crates/engine/src/edit/run.rs
#	crates/organize/src/import/carry.rs
#	crates/ui-egui/src/dialogs.rs
2026-11-09 18:25:40 +07:00
bflatastic d17159ef1e Merge #5259: M12: a PDF portfolio opens on its initial document and shows its files (#1652)
# Conflicts:
#	crates/render/src/inspect.rs
#	crates/render/src/lib.rs
2026-11-09 18:24:28 +07:00
bflatastic c638edf9c7 Merge #2137: M5: comment pop-up windows on the page, with connector lines (comment.popups, comment.connector-lines)
# Conflicts:
#	ROADMAP.md
#	crates/annot/src/info.rs
#	crates/annot/src/lib.rs
#	crates/annot/src/modify.rs
#	crates/annot/src/tests.rs
#	crates/automation/src/comments.rs
#	crates/automation/tests/it/automation.rs
#	crates/engine/src/comments.rs
#	crates/render/src/inspect.rs
#	crates/render/src/inspect/annotations.rs
#	crates/render/src/inspect/model.rs
#	crates/render/src/inspect/tests.rs
#	crates/render/src/lib.rs
#	crates/ui-egui/src/tools/comments/input.rs
#	crates/ui-egui/src/tools/comments/paint.rs
#	crates/ui-egui/src/tools/comments/tests.rs
#	crates/ui-egui/tests/it/comments.rs
2026-11-09 18:22:50 +07:00
bflatastic d8dad6f9ef Merge #4474: M3: relative links resolve against the document's base URI, and a link is active only in its QuadPoints (view.follow-links) 2026-11-09 18:20:36 +07:00
bflatastic 4494642b05 Search reads ligatures through the page text's ligature_letters 2026-11-09 18:20:36 +07:00
bflatastic 88a2ab97ec Merge #4094: M5: a callout's arrow point, knee and text box change on their own (#1152)
# Conflicts:
#	crates/annot/src/info.rs
#	crates/annot/src/modify.rs
#	crates/annot/src/tests.rs
#	crates/automation/src/tools.rs
#	crates/automation/tests/it/automation.rs
#	crates/ui-egui/tests/it/comments.rs
2026-11-09 18:19:26 +07:00
bflatastic 0dda66820c Merge #4396: M2: layers open in the state the view auto state sets, and the Layers panel follows /Order (view.layers-panel)
# Conflicts:
#	crates/render/src/inspect.rs
#	crates/render/src/lib.rs
#	vendor/README.md
2026-11-09 18:19:07 +07:00
bflatastic 15a264ff04 Merge #4393: M2: an Indexed image with a short lookup table or an out-of-range hival is drawn as most readers draw it, and one with no table is skipped with a warning (view.render-pages)
# Conflicts:
#	vendor/README.md
2026-11-09 18:17:39 +07:00
bflatastic 5af131db8e Merge #4358: M5: comments in an /RT /Group are one comment, with the primary's attributes (#4357)
# Conflicts:
#	crates/automation/src/comments.rs
#	crates/automation/tests/it/automation.rs
#	crates/render/src/inspect.rs
#	crates/render/src/inspect/annotations.rs
#	crates/render/src/lib.rs
#	crates/ui-egui/src/panels/comments_panel.rs
2026-11-09 18:17:31 +07:00
bflatastic 5257a6a34a Merge #3969: M10: a symbolic TrueType font without an encoding reads its glyph names (#3962)
# Conflicts:
#	vendor/README.md
2026-11-09 18:17:22 +07:00
mtlcl410 ad0fab3339 Merge remote-tracking branch 'upstream/main' into lane4-4474b
# Conflicts:
#	crates/automation/src/tools.rs
#	crates/automation/tests/it/automation.rs
#	crates/ui-egui/tests/it/document_links.rs
2026-11-09 18:52:46 +08:00
mtlcl410 ad42d5c276 Merge upstream/main into fix/indexed-short-lookup 2026-11-09 18:48:21 +08:00
mtlcl410 60de09e1a5 Merge remote-tracking branch 'upstream/main' into lane4-4470b
# Conflicts:
#	crates/render/src/text/search.rs
#	crates/render/tests/it/search.rs
2026-11-09 18:43:43 +08:00
mtlcl410 0bb77ce5e6 M5: review statuses are hidden, chain per reviewer and show per reviewer, by /IRT reference (#4122) (#4597) 2026-11-09 18:24:59 +08:00
Dylan M. Taylor 910b326774 fix(render): the named-destination index spends key bytes from one allowance (#6097) 2026-11-09 18:21:13 +08:00
mtlcl410 8d9f8d5f30 M5: a reply is not drawn on the page or flattened into it (comment.reply) (#5632) 2026-11-09 18:15:44 +08:00
jonathanwxh-cellandjonathanwxh-cell ab502255cb edit: preserve gap-drawn spaces and indents in subset fonts via /ActualText (#4229) (#4514)
Co-authored-by: jonathanwxh-cell <jonathanwxh-cell@users.noreply.github.com>
2026-11-09 18:13:34 +08:00
mtlcl410 7eea609c5d M3: copied text keeps one line per baseline, reading order for turned lines, letters for ligatures (#4467) (#4468) 2026-11-09 18:11:40 +08:00
mtlcl410 bd5db5dc35 M2: a page whose content was cut at the renderer's limits says so (#623) (#2113) 2026-11-09 18:09:37 +08:00
mtlcl410 cc1d76f3fd M3: whole-word Find matches inside Chinese, Japanese and Thai text (view.find-options) (#2075) 2026-11-09 18:09:20 +08:00
mtlcl410 9e29af7849 fix(render): a stream ending in endstrea is not read into a later object's stream (hayro-syntax patch 36) (#6669) 2026-11-09 18:03:41 +08:00
mtlcl410 375398cfac Merge main into fix/link-uri-base-and-quadpoints
crates/render/Cargo.toml and Cargo.lock: pdfcraft-render takes both url (the
/URI /Base resolution) and main's unicode-bidi.
2026-11-09 14:22:00 +08:00
Eric Kaden 6d126db6a0 Merge remote-tracking branch 'upstream/main' into feat/pdf-portfolio 2026-11-08 23:45:35 -05:00
mtlcl410 6c4f1fb630 Merge upstream/main into fix/search-ligature-accent-hyphen
The view.find parity entry keeps main's new notes and evidence (combining
marks and Thai or Indic word gaps, #6517) with this branch's ligature and
line-end hyphen sentence and tests added.
2026-11-09 12:02:34 +08:00
mtlcl410 f51eaa86e3 Merge main into fix/ocg-auto-state-view; renumber its patch to (103)
Main took hayro-interpret patch (76) for non-embedded Arial and Times New Roman
CIDs, so the View auto state becomes patch (103) in the ledger and the code
comments. The view.layers-panel note no longer lists the Print event's auto
state as missing: main applies it when printing (print.print-ready-pdf).
2026-11-09 11:50:10 +08:00
mtlcl410 c13acf6d0f Merge upstream/main into feat/callout-reshape-1152
Keeps main's Callout font size preference with the shared callout_knee(),
main's on_text() beside the callout box resizing, and both new inspector
helpers.
2026-11-09 11:43:24 +08:00
mtlcl410andClaude Opus 5.5 98a8fbbd9b Merge upstream/main into fix/search-ligature-accent-hyphen
Clean merge; brings in main's fix for the ui-egui dialog and security-notice
tests that failed this branch's gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 11:34:34 +08:00
mtlcl410andClaude Opus 5.5 8957eef644 Merge upstream/main into fix/link-uri-base-and-quadpoints
Main moved a link's action reading into Inspector::action_target, shared
with bookmarks, and made the link/attachment text budget start with the
outline. Resolution:
- annotations.rs: link() calls action_target and then resolves a URI
  target against /URI /Base, spending what the base adds from the same
  budget (or keeping the raw address); QuadPoints reading unchanged.
  The budget doc comment covers bookmarks, links, quads and attachments.
- core.rs: keeps the branch's uri_base read; the outline goes through
  main's budgeted outline_window (the branch's old call is dropped).
- tools.rs: link_list keeps main's "to_page with its zoom" and the
  branch's resolved_url / active_areas text.
relative_uri_actions_resolve_against_the_catalog_base re-proven red with
the resolution reverted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 11:23:54 +08:00
mtlcl410andClaude Opus 5.5 02db83e15d Merge upstream/main into fix/indexed-short-lookup
Main's #6663 added InterpreterWarning/RenderWarning::Type3WidthsMissing (patch
90) and a decoded-image cache that records warnings as flags (patch 91). Both
warning variants are kept in every match (renderer, automation, CLI), and
MalformedIndexedImage takes flag 32 in warning_bit/warnings_in. The ledger
keeps main's rows with (80) added; both sides' raster test modules are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 11:23:00 +08:00
mtlcl410andClaude Opus 5.5 b3557f7ebf Merge upstream/main into feat/comment-groups
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 11:18:11 +08:00
mtlcl410andClaude Opus 5.5 d130c4f4de Merge upstream/main into fix/symbolic-truetype-glyph-names-3962
Main's #6569 (hayro-interpret patch 101) reads a symbolic TrueType font that
names no glyphs and keeps the Macintosh order as Mac OS Roman; this branch's
patch (79) reads one that names its glyphs by those names. Both are kept: the
renderer struct holds both fields, and the editor's unicode_map applies the
glyph-name reading only to a font whose post table names glyphs, then the Mac
OS Roman one, so each font takes the reading the renderer gives it. The ledger
keeps main's rows with (79) added; the editor reuses descriptor_flags and
MAX_SFNT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 11:15:40 +08:00
bflatastic 6bfc0af05d Merge remote-tracking branch 'origin/main' into tmpup 2026-11-09 10:12:20 +07:00
BFlat d8c2b9ffe8 Revert "fix(ui): show creation and modification dates in Document Properties (#5534) (#5536)" (#6671) 2026-11-09 11:09:22 +08:00
Eric Kaden 1247099d0b Merge remote-tracking branch 'upstream/main' into feat/pdf-portfolio
# Conflicts:
#	crates/render/src/inspect/core.rs
2026-11-08 21:58:44 -05:00
mtlcl410andClaude Opus 5.5 09e965ff55 Merge upstream/main into feat/comment-groups
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 10:57:18 +08:00
mtlcl410andClaude Opus 5.5 cc1de556be Merge upstream/main into feat/comment-groups
Main's Replace Text (#5689) makes the caret the group's primary, as this
branch does. Kept: this branch's member-by-member deletion and release of
orphaned subordinates, with main's seed() (the strikeout also seeds its
caret) and its pair restyle; the writer returns the caret (the group's
comment) and leaves the strikeout without /Contents. Main's tests read the
strikeout from the caret's group instead of its replies, and the automation
test reads the strikeout's own colour from an XFDF export, since
comment_list gives a group its primary's attributes only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 10:57:18 +08:00
bflatastic 08c0a59212 Merge main into merge/redact-forms-misc
# Conflicts:
#	ROADMAP.md
#	crates/render/src/text/tests.rs
2026-11-09 09:56:59 +07:00
BFlat cde331fb7e Merge pull request #6663 from storytold/merge/render-interpret-patches
Land render and font patches (#6292, #6287, #6314, #6353, #6359, #6368, #6409, #6491, #6545, #6516, #6521, #6528, #6533, #6529, #6542, #6543, #6569, #6572)
2026-11-09 10:54:50 +08:00
mtlcl410andClaude Opus 5.5 f49ea0f70c Merge upstream/main into fix/indexed-short-lookup
Keeps main's vendor ledger with hayro-interpret patch (80) added, and both
sides' test modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 10:42:14 +08:00
mtlcl410andClaude Opus 5.5 45c7222704 Merge upstream/main into fix/symbolic-truetype-glyph-names-3962
Keeps main's vendor ledger and adds hayro-interpret patch (79) to it; the
symbolic TrueType program is read through doc.with_direct_parms, as main now
reads every font stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
2026-11-09 10:34:14 +08:00