mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-11-11 15:22:22 +00:00
Michael Higgins
f51ce3cfcc
## What this changes lopdf's fallback loader (used by `inspect` when the lazy reader fails, and always by `attachment_data`) parsed every object stream eagerly, capping only decoded bytes. A stream of `0 0 0 …` inside the 256 MiB limit built ~120M `Object`s (10+ GB) and aborted. This bounds what it builds: - **Per top-level object:** at most 4,194,304 objects (`parser::MAX_OBJECT_NODES`, the same as cos's `MAX_NODES`), counting every array element and dictionary value at any depth. One budget is threaded through `array` / `_dictionary` / `_direct_object`. An object is charged once it is built, so failed `alt` branches cost nothing, and an exhausted budget is a nom `Failure`. - **Per object stream:** at most 1,000,000 index entries (cos's `/N` clamp), and the index list is no longer allocated from the whole block. - **Per load:** at most 8,388,608 objects across all object streams, in an atomic `NodeBudget` shared by rayon workers. A member reserves up to its own byte length (every object takes at least one byte) and refunds what it doesn't use, so parallel members can't hold the budget between them. Once the budget is spent, the remaining members are skipped with a warning. - **`convert` in `crates/render/src/structure.rs`:** the depth limit is now 128, one level per array or dictionary, matching cos's parser. Before, it was 100 with two levels per dictionary, so ~51 nested dictionaries alone forced the lopdf fallback. Logged as lopdf patch (7) in `vendor/README.md`. Fixes #5083 ## How you verified it `cargo fmt --check` on `pdfcraft-render` and `cargo clippy -p pdfcraft-render --all-targets -- -D warnings` (which compiles the vendored lopdf). Regression tests in `crates/render/src/inspect/tests.rs`, where lopdf patches are tested in-tree (the vendored lopdf's own tests need its missing `assets/example.pdf`): - `object_stream_members_over_the_node_cap_are_skipped` (croquete1): a member that is one array of 4.3M elements is skipped at the per-object cap; a small member is kept. - `object_stream_index_entries_past_the_clamp_are_ignored`: of 1,000,001 distinct members, the one past the 1,000,000-entry clamp is never built. - `object_streams_of_one_load_share_their_node_budget`, `object_streams_loaded_in_parallel_share_their_node_budget` and `a_malformed_object_stream_member_refunds_its_reservation` (Harrison-Booth): a small shared `NodeBudget` bounds the members of several streams, serially and across threads, and a malformed member refunds what it didn't build. `NodeBudget` and `ObjectStream::new_within` are now public for this. - [x] `cargo fmt --check` - [x] clippy with `-D warnings` on the crates I changed - [x] tests pass on the crates I changed (`cargo test -p pdfcraft-render --lib object_stream`: 9 passed, including the five new tests) ## Never-crash - [x] No new `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!` outside tests - [x] No indexing or slicing with input-derived positions; the budget uses `checked_sub` and a `min`-bounded reservation; index entries and objects built are capped; nesting stays bounded (128 in `convert`) - [x] No `unsafe` - [x] The abort this fixes has synthetic regression tests (above) ## Provenance - [x] No assets, fonts or copied code; the limits follow `pdfcraft_cos`'s own caps ## Security - [x] No change to `--root`, MCP, the control channel or network access ## Parity and reach - Not applicable: an internal robustness fix to the fallback loader, with no user-facing feature or parity entry. The vendored-patch log (`vendor/README.md`, lopdf (7)) is updated. ## Anything a reviewer should look at first - `vendor/README.md` now lists the tests above for patch (7). - Ordinary (not compressed) indirect objects are capped at 4M objects each but aren't charged to the per-load budget; their total is bounded only by file size. - `get_compressed_object` still decodes and re-parses the whole stream per lookup (pre-existing); it now gets one object's budget. - Objects nested 101–128 deep now reach the Inspector through the lazy reader rather than being skipped by the fallback. The Inspector's own walkers cap depth at 32 or less. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Bug Fixes** * Added limits to PDF object parsing and object-stream processing. Loading now parses only selected members, while lookups parse only the requested member. Members that exceed available limits are skipped in a deterministic order. * Increased the supported nesting depth for dictionary and object conversion, allowing more deeply nested content to be processed. * Documented that conversion failures can trigger eager loading as a fallback. <!-- end of auto-generated comment: release notes by coderabbit.ai -->