From 97cef90e016fc81d6683996e88c6eed13c1b8be7 Mon Sep 17 00:00:00 2001 From: Hamza Naveed <45464935+Hamza0101@users.noreply.github.com> Date: Fri, 6 Nov 2026 14:01:42 +0500 Subject: [PATCH] Redaction: a pattern nothing paints with any more leaves the page (#3680) * Redaction: a pattern nothing paints with any more leaves the page A tiling pattern's cell is a content stream of its own and can draw anything, text included. Redacting a shape filled with such a pattern removed the fill from the page's content -- the page looked right, a black box -- but left the pattern in /Resources /Pattern, and its cell, with the text that was under the mark, was written into the saved file unchanged. The pass does not enter pattern cells, so it cannot redact inside one. What it can do is notice that nothing paints with the pattern any more. The graphics state now carries the fill and stroke pattern a `scn` or `SCN` selected (and `cs`, `g`, `rg`, `k` and their stroke forms clear it), every painting operator that survives the rewrite records the pattern it drew with, and a pattern the page selected but no surviving paint used leaves the page's resources. Measured on the report's file: before applied: 1, (SECRET) in the saved bytes after applied: 1, no /Pattern on the page, (SECRET) gone A pattern some surviving paint still uses is kept, and so is its cell: with two boxes filled from one pattern and only the first marked, patterns_removed is 0 and the resources keep the entry -- the second box still shows the word, so it is not a leak. Report gains patterns_removed. Fixes #3647. * Keep a pattern every operator that can paint with it still uses Two defects CodeRabbit found in the first version of this change, both of which could remove a pattern the page still draws with. Pattern use was recorded for path painting only. Text shows with the fill pattern too -- and with the stroke pattern in the stroking render modes -- as do image masks and anything a form draws, and a form's content is interpreted with a graphics state of its own, so a pattern it inherits is never seen inside it. Content like /Pattern cs /P0 scn 20 140 100 30 re f BT (KEEP) Tj ET with the box redacted therefore lost the fill of text still on the page. Every operator that can paint with the current pattern now records it: text (except mode 3, which paints nothing, and mode 7, which only clips), inline images, and image or form XObjects. Only path painting knows exactly what it drew and whether the paint survived, so the others are recorded as they are met. That can keep a pattern whose paint was in fact all redacted, which costs an unused resource entry, while dropping one something still draws with takes paint off the page. /Resources /Pattern may be an indirect object, like any value. Reading the reference itself as a dictionary found none and pruned nothing, so on such a file the cell -- with the text that was under the mark -- stayed in the saved bytes, which is the leak this change is for. It is resolved now, and what is written back is a dictionary of this page's own, so pruning never reaches through a /Pattern dictionary another page shares. The controls in the test: without the first fix the surviving text's pattern is removed (left: 1, right: 0), and without the second the indirect file prunes nothing (left: 0, right: 1). * M8: refuse retained unsafe tiling patterns * M8: guard shared appearances and narrow pattern scans * M8: update TLS dependencies for new advisories * Redaction: the comment on prune_patterns said more than the code does `prune_patterns` drops the `/Pattern` resource entry; the `scn` that selected it stays in the content, so the page names a pattern its resources no longer hold. The comment claimed the selector went too (noted in review). ISO 32000-1 8.6.6.2 makes that operand a key of the `/Pattern` subdictionary, so the open question is whether a reader stops reading the stream at the stale name. `a_page_whose_pruned_pattern_is_still_selected_renders` answers it with the renderer rather than with prose: the redacted page is saved, reopened and rasterized, and * it renders with no error and the same 300x300 page box, * the text written *after* the stale selector is still painted -- which is exactly what a reader would drop if it gave up at the bad operand, * the redaction overlay is black over the box the pattern filled. Control: with the drawing after the selector removed, the pixel assertion fails ("the text after the stale selector is still painted"), so it is reading that text and not a stray pixel. The selector is left in place rather than cut out: removing operators is a rewrite of the content this pass does not otherwise do, and nothing surviving paints with that colour. `pdfcraft-render` is a dev dependency of this crate only -- layer 4 on layer 3, which the layering allows. * WIP: pattern guard depth bound sized from the interpreter's * Pattern tests: keep the guard's cases reaching the guard now a hit path is removed whole * Pattern automation tests: keep the pattern in use where they test the guard, since a hit path is now removed whole * Redaction: a form without resources keeps the page pattern it still paints A form with no /Resources of its own paints with its caller's patterns, but its surviving pattern paint was recorded only in the form's own output and never reached the page. With the page's own fill under the mark removed whole, prune_patterns then dropped /P0 from the page while the form outside the mark still painted with it, so that fill lost its paint in the saved file. Pass the painted names up from a form that inherits its caller's resources. Only the painted side goes up, so this can keep a pattern, never prune one. * Redaction: a pruned pattern does not stay named by the page tree prune_patterns writes only the page's own /Resources, and the guard walked only pages' effective resources. A /Pages node that still named the pruned pattern kept its cell reachable from Root, so a full save wrote the cell, text included, while apply returned Ok. Three shapes did it: a page with no /Parent inheriting from the root, a /Parent that disagrees with the /Kids tree, and a root /Resources shadowed by a nearer holder that was retired. Patterns pruned from a page are now unlinked from page-tree holders no page inherits from, as unlink_doomed does for XObjects, when every /Parent agrees with the tree. Whenever a pattern was pruned (or a selected page had any), the guard also inspects the /Resources of every /Pages node Root reaches down /Kids and up each page's /Parent chain, so whatever stays reachable is proven safe or the redaction is refused, leaving the document unchanged. * Redaction: a Type 3 glyph without resources keeps the page pattern it paints A Type 3 font with no /Resources looks the names in its glyph descriptions up in the resources of the page or form showing it, so a glyph outside the mark can paint with the page's pattern. Text recorded only the fill and stroke pattern of the graphics state, so with the page's own fill under the mark removed whole, prune_patterns dropped the pattern and the glyph lost its paint. Showing such a font now keeps every pattern of the scope, as a form without /Resources keeps what it paints. * Redaction: say what text-recorded pattern use keeps Text, inline images and XObjects record their pattern before the hit test, so a pattern painted only by redacted text keeps its cell. The guard refuses a cell with text or images; a vector-only cell stays in the file. The comment said this cost only an unused resource entry. Pin the behaviour in a test. * Redaction: the page tree is followed up every /Parent link A full save writes every node a /Parent link reaches, but the guard followed /Parent only from pages and stopped at the first node it had already seen, and unlinking never followed it. A /Pages node reached only through the /Parent of a /Kids node -- a root /Pages with a /Parent, or an intermediate node whose /Parent leads out of the tree -- kept naming the pruned pattern, and its cell, text included, stayed in the saved file while apply returned Ok. The guard now follows /Parent from every node it visits, page or /Pages, with the same seen-set against cycles. Unlinking treats a root with a /Parent as an inconsistent tree, like a disagreeing link, and leaves the decision to the guard, which inspects every node those links reach and refuses an unsafe cell. * Redaction: a long page tree does not exhaust the pattern guard's budget The page-tree walk counted every node, pages included, against the guard's 16,384-object budget, so redacting a pattern page of a 17,000-page document was refused with a limit error where it had applied before. Only the holder nodes the guard inspects count against that budget now; the walk itself is bounded at a million nodes, as unlink_doomed bounds the same tree. * Redaction: a Type 3 font set by an ExtGState or shown in a form keeps the page patterns The resourceless Type 3 check covered only a font set by Tf in the scope that paints. A Type 3 font without /Resources set through an ExtGState /Font entry was not flagged, and a glyph shown inside a form with /Resources of its own recorded its patterns in the form's scope, which the page never saw. In both cases the page's own fill under the mark was removed whole, prune_patterns dropped the pattern, and the glyph outside the mark lost its paint. An ExtGState font is now checked the same way, and a glyph shown in any nested form marks the page as keeping every pattern it selected. This can only keep patterns; whatever is kept is checked by the guard. * Redaction: a pruned pattern a full save would still write is proven safe or refused The walks that check retained pattern cells follow the references they know -- resource categories, the page tree, appearances -- but a full save writes whatever the trailer reaches by any reference. A direct /Parent dictionary, a /Kids array on a page, or a stray resource entry (/Shading naming the pattern stream) kept a pruned cell, text included, in the saved file while apply returned Ok. pdfcraft_cos gains reachable_objects, which runs the same breadth-first walk write_full uses (factored out as full_save_order, so the two cannot drift). Before apply succeeds, every pruned pattern that walk still reaches is inspected like a retained cell: vector-only passes, anything else refuses with the existing pattern error. It runs only when a pattern was pruned and costs one walk of the reachable objects. * Redaction: a direct /Parent dictionary is part of the page tree too A /Parent given as a dictionary rather than a reference is written inline by a full save, but the page-tree walk followed /Parent only as a reference, and unlinking took such a tree for consistent. A root /Pages whose direct /Parent held /Resources naming the pruned pattern kept its cell in the saved file. The guard now walks a direct /Parent like any node (its /Resources, /Parent and /Kids), and unlinking counts any /Parent that is not the reference /Kids came from as an inconsistent tree, leaving the decision to the guard. * Redaction: page-tree walks queue each node once and borrow /Kids On a hostile tree whose /Pages nodes share one long /Kids array, the guard's page-tree walk and pattern unlinking pushed every /Kids entry of every node before checking whether it had been seen, and copied the array on each visit. Both now queue only nodes not seen yet and read /Kids in place; unlinking also stops at the first node reached twice, since its /Parent can agree with at most one of the nodes above it, and leaves that tree to the guard. * Redaction: a Type 3 font with /Resources null is resourceless too /Resources null is the same as no /Resources, but the check counted any entry, so such a font's glyph outside the mark lost the page pattern it painted with. A Type 3 glyph shown by a kept annotation appearance is a known limit, pinned as an ignored test: appearances are not interpreted for pattern use, and under ISO 32000 the glyph resolves in the appearance's own resources. * Redaction: a pruned pattern is recorded along its whole reference chain A pattern name can lead to a reference-valued object whose value is the cell (/P0 7 0 R, object 7 being 6 0 R). Pruning recorded only object 7, so the reachability backstop checked an object nothing reached while a stray reference kept the cell, object 6, in the saved file. Every object along the chain is recorded now, bounded like the other walks. * Pattern tests: pin the file identifier where two saves are compared main now gives a redacted rewrite a new identifier from fresh entropy, so an incremental and a full save after Apply differ in /ID alone. The test that checks they are the same save passes fixed entropy to both. * Redaction: replies and pop-ups removed on other pages feed the shared-appearance check Apply checks every page for retained unsafe pattern cells when a removed annotation's appearance is shared with one that stays on an unselected page. Only annotations removed under a mark fed that check. Replies and pop-ups that go with them in step 2b (removed_everywhere), often listed on another page, did not, so a text-bearing cell in an appearance a kept annotation shares with them escaped the all-pages check and Apply returned Ok. Their appearance references now go into removed_appearances too, with the same limit. Raised by FredApps' review of the rebase. --------- Co-authored-by: skylar taylor-barrick --- Cargo.lock | 1 + ROADMAP.md | 2 + crates/automation/tests/pattern_redaction.rs | 146 ++++ crates/cos/src/lib.rs | 2 +- crates/cos/src/writer.rs | 40 +- crates/redact/Cargo.toml | 3 + crates/redact/README.md | 9 + crates/redact/src/interp.rs | 127 ++- crates/redact/src/interp/prune.rs | 136 ++++ crates/redact/src/lib.rs | 137 +++- crates/redact/src/pattern_guard.rs | 496 ++++++++++++ crates/redact/src/tests.rs | 194 +++++ crates/redact/tests/pattern_page_tree.rs | 221 +++++ .../redact/tests/pattern_page_tree_shapes.rs | 275 +++++++ crates/redact/tests/pattern_reachability.rs | 304 +++++++ crates/redact/tests/pattern_redaction.rs | 758 ++++++++++++++++++ docs/roadmap.md | 2 +- 17 files changed, 2837 insertions(+), 16 deletions(-) create mode 100644 crates/automation/tests/pattern_redaction.rs create mode 100644 crates/redact/src/pattern_guard.rs create mode 100644 crates/redact/tests/pattern_page_tree.rs create mode 100644 crates/redact/tests/pattern_page_tree_shapes.rs create mode 100644 crates/redact/tests/pattern_reachability.rs create mode 100644 crates/redact/tests/pattern_redaction.rs diff --git a/Cargo.lock b/Cargo.lock index e1f271ed12..8617c45522 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4628,6 +4628,7 @@ dependencies = [ "pdfcraft-fonts", "pdfcraft-forms", "pdfcraft-model", + "pdfcraft-render", "pdfcraft-testkit", "pdfcraft-xfa", "sha2 0.11.1", diff --git a/ROADMAP.md b/ROADMAP.md index 4fcdadaee9..820ceb0179 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -357,6 +357,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-20 (M5, arrow keys move comments):** With a comment selected, the arrow keys move it a screen pixel and ⌘/Ctrl or Shift and an arrow ten, instead of scrolling the view or turning the page; up is up on screen at any zoom and rotation, and each press is one undo step. Text markup and locked comments stay put. Estimates unchanged. +- **2026-10-20 (M8, #3647):** Apply prunes fully removed tiling patterns, then checks retained pattern resources for text, images and unreadable content before reporting success. Shared patterns, Form-owned and inherited patterns, and cells kept by another page cause an atomic refusal when safe removal cannot be shown. Synthetic saved-file and automation regressions cover refusal, preserved marks and undo, and vector-only controls. Estimates unchanged. + - **2026-10-20 (M1, PR #1143 review):** Repair-aware dictionary/preflight edits and snapshot restoration preserve both accepted generations of a damaged live slot. Such edits use the existing full-save fallback; classic and packed rewrites coalesce aliases so page-tree parent links retain one object identity. Explicit generation replacement remains single-generation. Materialized xref diagnostics inspect retained primary and hybrid rows before folding, retaining the 256-object cap plus summary and discarding abandoned-index notes. Synthetic red/green controls cover preflight, undo restoration, equal values, explicit narrowing, encryption packing and four save variants. Estimates unchanged. - **2026-10-19 (M14, #3373):** The main menu and its long submenus scroll in short windows, keeping their commands reachable. Estimates unchanged. diff --git a/crates/automation/tests/pattern_redaction.rs b/crates/automation/tests/pattern_redaction.rs new file mode 100644 index 0000000000..667779cef6 --- /dev/null +++ b/crates/automation/tests/pattern_redaction.rs @@ -0,0 +1,146 @@ +//! Rectangle redaction must not report a pattern-cell secret safely removed (#3647). + +use std::sync::Arc; + +use pdfcraft_automation::{Automation, Content}; +use pdfcraft_cos::{Dict, Document, Object, Stream}; +use pdfcraft_engine::DocId; +use serde_json::{Value, json}; + +fn tool(automation: &mut Automation, name: &str, args: Value) -> Value { + match automation.call(name, &args).unwrap().remove(0) { + Content::Json(value) => value, + other => panic!("expected JSON: {other:?}"), + } +} + +/// `retained`: paint with the pattern once more, away from the mark (x 290..340 lies right of it). +/// A path the mark hits is removed whole (F-008) and a pattern nothing surviving paints with is +/// pruned, so only a pattern that some surviving paint still uses reaches the guard. +fn fixture(cell: &[u8], partial: bool, compressed: bool, retained: bool) -> Vec { + let mut pattern = Dict::from_iter([ + (b"Type".to_vec(), Object::name("Pattern")), + (b"PatternType".to_vec(), Object::Int(1)), + (b"PaintType".to_vec(), Object::Int(1)), + (b"TilingType".to_vec(), Object::Int(1)), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + (b"XStep".to_vec(), Object::Int(500)), + (b"YStep".to_vec(), Object::Int(300)), + ]); + pattern.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([( + b"Font".to_vec(), + Object::Dict(Dict::from_iter([(b"F1".to_vec(), Object::Ref(pdfcraft_cos::ObjRef::new(4, 0)))])), + )])), + ); + let width = if partial { 400 } else { 170 }; + let survivor = if retained { " 290 115 50 30 re f" } else { "" }; + let content = format!("BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET /Pattern cs /P0 scn 35 110 {width} 35 re f{survivor}"); + let mut page_stream = Vec::new(); + pdfcraft_cos::serialize(&Object::Stream(Stream::from_raw(Dict::new(), content.into_bytes())), &mut page_stream); + let mut pattern_stream = Vec::new(); + let stream = if compressed { Stream::flate(pattern, cell) } else { Stream::from_raw(pattern, cell.to_vec()) }; + pdfcraft_cos::serialize(&Object::Stream(stream), &mut pattern_stream); + let objects = [ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> >> >>" + .to_vec(), + b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(), + page_stream, + pattern_stream, + ]; + let mut bytes = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (index, object) in objects.iter().enumerate() { + offsets.push(bytes.len()); + bytes.extend(format!("{} 0 obj\n", index + 1).as_bytes()); + bytes.extend(object); + bytes.extend(b"\nendobj\n"); + } + let xref = bytes.len(); + bytes.extend(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes()); + for offset in offsets { + bytes.extend(format!("{offset:010} 00000 n \n").as_bytes()); + } + bytes.extend(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objects.len() + 1).as_bytes()); + bytes +} + +fn saved_has_secret(bytes: Vec) -> bool { + let document = Document::open(Arc::new(bytes)).unwrap(); + document.object_numbers().into_iter().any(|num| { + let object = document.try_get(num).unwrap(); + matches!(&*object, Object::Stream(stream) if stream.decoded().unwrap().windows(10).any(|part| part == b"CELLSECRET")) + }) +} + +#[test] +fn pattern_redaction_tools_remove_or_refuse_without_leaking_or_changing_undo() { + let dir = tempfile::tempdir().unwrap(); + for (partial, retained) in [(false, false), (true, false), (true, true)] { + for compressed in [false, true] { + let cell = b"BT /F1 20 Tf 40 120 Td (CELLSECRET) Tj ET BT /F1 20 Tf 300 120 Td (SAFE) Tj ET"; + let bytes = fixture(cell, partial, compressed, retained); + assert!(saved_has_secret(bytes.clone())); + std::fs::write(dir.path().join("input.pdf"), bytes).unwrap(); + let mut automation = Automation::new().with_root(dir.path()).unwrap().with_clock(|| 1_700_000_000); + let id = tool(&mut automation, "doc_open", json!({"path": "input.pdf"}))["doc"].as_u64().unwrap(); + tool(&mut automation, "redact_mark", json!({"doc": id, "page": 1, "rect": [30, 150, 215, 195]})); + let before = automation.session().get(DocId(id)).unwrap().bytes.clone(); + let pixels = automation.render_pixels(id, 1, 1.0).unwrap(); + match automation.call("redact_apply", &json!({"doc": id})) { + Ok(_) => { + assert!(!retained, "a text-bearing cell still in use must not be reported safe (partial: {partial})"); + tool(&mut automation, "doc_save", json!({"doc": id, "path": "redacted.pdf"})); + assert!( + !saved_has_secret(std::fs::read(dir.path().join("redacted.pdf")).unwrap()), + "successful apply/save left the pattern-cell text recoverable" + ); + } + Err(error) => { + assert!(retained, "a pattern nothing surviving paints with should apply (partial: {partial}): {error}"); + assert!(error.to_string().contains("tiling pattern"), "{error}"); + assert!(!error.to_string().contains("CELLSECRET")); + let document = automation.session().get(DocId(id)).unwrap(); + assert_eq!(document.bytes, before); + assert_eq!(document.redaction_marks(), 1); + assert_eq!(document.can_undo(), Some("Add redaction mark")); + assert_eq!(automation.render_pixels(id, 1, 1.0).unwrap(), pixels); + tool(&mut automation, "edit_undo", json!({"doc": id})); + assert_eq!(automation.session().get(DocId(id)).unwrap().redaction_marks(), 0); + } + } + } + } +} + +#[test] +fn pattern_redaction_vector_control_saves_reopens_and_preserves_outside_pixels() { + let dir = tempfile::tempdir().unwrap(); + // The guard's control: the cell is plain vectors and a paint outside the mark still uses it, + // so it is inspected and kept, not refused, and what that paint draws is unchanged. + let bytes = fixture(b"0.2 0.4 0.6 rg 40 120 15 15 re f 300 120 30 20 re f", true, true, true); + std::fs::write(dir.path().join("vectors.pdf"), bytes).unwrap(); + let mut automation = Automation::new().with_root(dir.path()).unwrap().with_clock(|| 1_700_000_000); + let id = tool(&mut automation, "doc_open", json!({"path": "vectors.pdf"}))["doc"].as_u64().unwrap(); + let (width, height, original) = automation.render_pixels(id, 1, 1.0).unwrap(); + tool(&mut automation, "redact_mark", json!({"doc": id, "page": 1, "rect": [30, 150, 215, 195]})); + let applied = tool(&mut automation, "redact_apply", json!({"doc": id})); + assert_eq!(applied["applied"], 1); + let save = tool(&mut automation, "doc_save", json!({"doc": id, "path": "vectors-redacted.pdf"})); + assert_eq!(save["incremental"], false); + let reopened = tool(&mut automation, "doc_open", json!({"path": "vectors-redacted.pdf"}))["doc"].as_u64().unwrap(); + let (new_width, new_height, actual) = automation.render_pixels(reopened, 1, 1.0).unwrap(); + assert_eq!((new_width, new_height), (width, height)); + for (index, (before, after)) in original.as_chunks::<4>().0.iter().zip(actual.as_chunks::<4>().0).enumerate() { + let (x, y) = (index % width as usize, index / width as usize); + if x >= 280 { + assert_eq!(before, after, "unmarked pattern pixels changed at {x},{y}"); + } + if (35..205).contains(&x) && (155..190).contains(&y) { + assert_eq!(after, &[0, 0, 0, 255], "marked interior must be black"); + } + } +} diff --git a/crates/cos/src/lib.rs b/crates/cos/src/lib.rs index 8e051e4578..32a6f6d8b3 100644 --- a/crates/cos/src/lib.rs +++ b/crates/cos/src/lib.rs @@ -30,7 +30,7 @@ pub use parser::{Lexer, is_whitespace, parse_indirect}; pub use pdfcraft_crypt::{ Algorithm, Auth, Method as CryptMethod, NewEncryption, Permissions, SecurityHandler, effective_password, pdfdoc_byte, pdfdoc_char, }; -pub use writer::{SaveOptions, pdf_date, serialize, write_full, write_incremental}; +pub use writer::{SaveOptions, pdf_date, reachable_objects, serialize, write_full, write_incremental}; #[derive(Debug, thiserror::Error, Clone, PartialEq)] pub enum CosError { diff --git a/crates/cos/src/writer.rs b/crates/cos/src/writer.rs index e0762612dc..0af74f7bab 100644 --- a/crates/cos/src/writer.rs +++ b/crates/cos/src/writer.rs @@ -273,16 +273,12 @@ pub fn write_incremental(doc: &Document, opts: &SaveOptions) -> Result, Ok(file) } -/// Write only reachable objects, renumbered from 1, with a table or cross-reference stream. -pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result, CosError> { - if doc.stream_limited() { - return Err(CosError::ReadOnlyLimit); - } - // A full traversal must not fill the editor's shared caches (which undo snapshots also - // own). Parsed objects and decoded object streams are temporary working data here. - let mut reader = doc.object_reader(); - stamp_mod_date(&mut reader.document, opts); - let trailer_in = reader.document.trailer().clone(); +/// The objects [`write_full`] writes: everything reachable from the trailer's /Root, /Info and +/// /Encrypt and from its unknown entries, breadth-first, with the new number each gets +/// (aliases of one slot included) and, in visit order, whether it is a stream. +type SaveOrder = (HashMap, Vec<(ObjRef, bool)>); + +fn full_save_order(reader: &mut ObjectReader, trailer_in: &Dict) -> Result { // Breadth-first walk from the trailer, assigning new numbers in visit order. let mut map: HashMap = HashMap::new(); // Record the type while traversing so classifying packed objects needs no second parse. @@ -326,6 +322,30 @@ pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result, CosErro order.push((canonical, matches!(*o, Object::Stream(_)))); visit_refs(&o, &mut queue); } + Ok((map, order)) +} + +/// The object numbers a full save of `doc` would write: exactly the objects [`write_full`] +/// reaches from the trailer, object streams included. For a check that something removed from a +/// document cannot reach the saved file through a reference nobody looked for. +pub fn reachable_objects(doc: &Document) -> Result, CosError> { + let mut reader = doc.object_reader(); + let trailer_in = reader.document.trailer().clone(); + let (_, order) = full_save_order(&mut reader, &trailer_in)?; + Ok(order.into_iter().map(|(r, _)| r.num).collect()) +} + +/// Write only reachable objects, renumbered from 1, with a table or cross-reference stream. +pub fn write_full(doc: &Document, opts: &SaveOptions) -> Result, CosError> { + if doc.stream_limited() { + return Err(CosError::ReadOnlyLimit); + } + // A full traversal must not fill the editor's shared caches (which undo snapshots also + // own). Parsed objects and decoded object streams are temporary working data here. + let mut reader = doc.object_reader(); + stamp_mod_date(&mut reader.document, opts); + let trailer_in = reader.document.trailer().clone(); + let (map, order) = full_save_order(&mut reader, &trailer_in)?; let version = reader.document.version().chars().take(3).collect::(); let version = if opts.object_streams && version.as_str() < "1.5" { "1.5".to_string() } else { version }; let mut out = Vec::new(); diff --git a/crates/redact/Cargo.toml b/crates/redact/Cargo.toml index c02335e3c4..149d8d8295 100644 --- a/crates/redact/Cargo.toml +++ b/crates/redact/Cargo.toml @@ -25,3 +25,6 @@ sha2 = "0.11" [dev-dependencies] pdfcraft-testkit = { workspace = true } +# The redacted page is rendered in `tests::a_page_whose_pruned_pattern_is_still_selected_renders`: +# layer 3 from layer 4, a dev dependency only. +pdfcraft-render = { workspace = true } diff --git a/crates/redact/README.md b/crates/redact/README.md index db9f46b8c9..f78f819c03 100644 --- a/crates/redact/README.md +++ b/crates/redact/README.md @@ -60,6 +60,15 @@ resume after it would be a guess. A Redact annotation with neither a usable `/Qu `/Rect` makes it fail with `RedactError::UnreadableMark` before anything changes, instead of skipping that mark. The count covers the marks on the pages being applied. +Tiling-pattern cells are separate streams. Applying a mark removes a pattern resource when no +surviving paint uses it. If a cell remains reachable after rewriting the selected pages and may +contain text, images, replacement descriptions, or unreadable content, Apply refuses with a +`RedactError::Pattern` and restores the document and pending marks. This includes patterns owned +by Forms and patterns inherited by Forms. Vector-only retained cells remain supported. The +bounded check is conservative: an unrelated retained pattern on a selected page can also cause +refusal when its content cannot be proven safe. If a selected page had a pattern resource, the +check covers all pages after rewriting so another page cannot keep the same cell in the saved PDF. + Before changing any page, `apply` prepares the redaction appearances with a shared 8 MiB budget for their uncompressed PDF content across the selected pages. It bounds repeated-text counts and working line buffers before allocating them, and measures and encodes each repeated diff --git a/crates/redact/src/interp.rs b/crates/redact/src/interp.rs index 7a2cd26eca..b0a7257cb3 100644 --- a/crates/redact/src/interp.rs +++ b/crates/redact/src/interp.rs @@ -31,7 +31,7 @@ mod prune; use layers::strip_hidden_layers; use prune::{clip_out, extgstate_font, prune_form_resources}; -pub(crate) use prune::{prune, prune_properties, resource_ops, unlink_doomed}; +pub(crate) use prune::{prune, prune_patterns, prune_properties, resource_ops, unlink_doomed, unlink_pruned_patterns}; /// How deep this pass follows form XObjects. A form past it is removed rather than walked, which /// is the fail-safe choice for a redaction -- marked content inside a form we cannot read would @@ -85,6 +85,17 @@ struct Gs { scale: f64, leading: f64, rise: f64, + /// The tiling or shading pattern selected as the fill colour (`/Name scn`), and as the stroke + /// colour (`/Name SCN`). Part of the graphics state, so `q`/`Q` save and restore them with + /// everything else. A pattern's cell is a content stream of its own and this pass does not + /// enter it, so a pattern whose only painting was removed has to leave the page's resources + /// or its drawing stays in the file (#3647). + fill_pattern: Option>, + stroke_pattern: Option>, + /// The current font is a Type 3 font without `/Resources`: its glyph descriptions look their + /// names up in this scope's resources (ISO 32000-2 §9.6.4), so a shown glyph may paint with + /// any of this scope's patterns. + type3_inherits: bool, } /// What processing a scope produced. @@ -102,6 +113,16 @@ pub(crate) struct Output { /// dictionaries may name them too (an inherited or shared /Resources, or the caller's when a /// form has none of its own): [`unlink_doomed`] removes those entries. pub doomed: Vec, + /// Pattern resource names a painting operator that survived the rewrite still uses. A name + /// this scope saw selected but never painted with afterwards is pruned from the page's + /// `/Resources /Pattern`, so a cell whose only drawing was redacted does not stay in the + /// saved file (#3647). + pub patterns_seen: Vec>, + pub patterns_painted: Vec>, + /// A Type 3 glyph without resources of its own was shown in this scope or a nested form, + /// through a font set by `Tf` or an ExtGState: which of the page's patterns it paints with is + /// not tracked, so the page keeps them all. + pub keeps_all_patterns: bool, /// Original property values retired by this scope or a nested form. Other resource holders /// may still name them; unlink only entries no surviving content uses. pub retired_properties: Vec, @@ -264,6 +285,22 @@ struct Glyph { rects: Vec, } +/// Record the patterns an operator paints with, so [`prune_patterns`] keeps them. Only path +/// painting knows exactly what it draws and whether the paint survived; text, images and forms +/// are recorded as they are met, which can keep a pattern whose paint was in fact all redacted, +/// and with it the pattern's cell. A cell that carries text or images is then refused by +/// `pattern_guard`; a vector-only cell stays in the file though nothing shows it +/// (`a_pattern_painted_only_by_redacted_text_keeps_its_vector_cell`). Dropping a pattern something +/// still draws with would lose paint from the page instead. +fn uses_patterns(out: &mut Output, gs: &Gs, fills: bool, strokes: bool) { + if fills && let Some(n) = &gs.fill_pattern { + out.patterns_painted.push(n.clone()); + } + if strokes && let Some(n) = &gs.stroke_pattern { + out.patterns_painted.push(n.clone()); + } +} + /// Process a scope's content streams (state flows from one stream to the next). pub(crate) fn process(doc: &mut Document, scope: &mut Scope<'_>, streams: &[Vec], resources: &Dict, ctm: Matrix) -> Output { let gs = Gs { @@ -277,6 +314,9 @@ pub(crate) fn process(doc: &mut Document, scope: &mut Scope<'_>, streams: &[Vec< scale: 1.0, leading: 0.0, rise: 0.0, + fill_pattern: None, + stroke_pattern: None, + type3_inherits: false, }; process_from(doc, scope, streams, resources, gs) } @@ -377,6 +417,8 @@ struct Walk { fonts_res: Dict, ext_gstates: Dict, xobjects: Dict, + /// This scope's pattern names, which a Type 3 glyph without resources may paint with. + patterns: Vec>, fallback: Rc, /// The current path: its operators, its bounding box in user space, and whether it clips. path: Vec, @@ -395,6 +437,7 @@ impl Walk { fonts_res: res_dict(doc, resources, b"Font"), ext_gstates: res_dict(doc, resources, b"ExtGState"), xobjects: res_dict(doc, resources, b"XObject"), + patterns: res_dict(doc, resources, b"Pattern").iter().map(|(n, _)| n.to_vec()).collect(), fallback: Rc::new(Metrics::fallback()), path: Vec::new(), path_box: None, @@ -433,6 +476,39 @@ impl Walk { b"BI" => Self::inline_image(&self.gs, scope, &mut self.out, op, piece), b"sh" if scope.mode == Mode::Apply => Self::shading(&self.gs, scope, op, piece), b"Do" => self.draw_xobject(doc, scope, resources, op, piece), + // `/Name scn` selects a pattern as the fill colour; numbers select a plain colour + // and clear it. `cs` with a space other than /Pattern clears it too. + b"scn" | b"SCN" => { + let fill = o == b"scn"; + let picked = op.operands.last().and_then(Object::as_name).map(<[u8]>::to_vec); + if let Some(name) = &picked { + self.out.patterns_seen.push(name.clone()); + } + if fill { + self.gs.fill_pattern = picked; + } else { + self.gs.stroke_pattern = picked; + } + Some(op) + } + b"cs" | b"CS" => { + if op.name(0) != Some(b"Pattern") { + if o == b"cs" { + self.gs.fill_pattern = None; + } else { + self.gs.stroke_pattern = None; + } + } + Some(op) + } + b"g" | b"rg" | b"k" => { + self.gs.fill_pattern = None; + Some(op) + } + b"G" | b"RG" | b"K" => { + self.gs.stroke_pattern = None; + Some(op) + } _ => Some(op), }; piece.ops.extend(kept); @@ -464,6 +540,12 @@ impl Walk { let clips = std::mem::replace(&mut self.clip, false); let hit = scope.mode == Mode::Apply && op.op.as_slice() != b"n" && bbox.is_some_and(|b| scope.hits(b)); if !hit { + // The paint survives, so the pattern it draws with is still in use. `n` paints + // nothing, so it uses none. A hit path is removed whole (F-008), pattern and all. + let o = op.op.as_slice(); + let strokes = matches!(o, b"S" | b"s" | b"B" | b"B*" | b"b" | b"b*"); + let fills = matches!(o, b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*"); + uses_patterns(&mut self.out, &self.gs, fills, strokes); piece.ops.extend(body); piece.ops.push(op); return; @@ -497,6 +579,8 @@ impl Walk { _ => { if let Some((font, vertical, size)) = op.name(0).and_then(|n| extgstate_font(doc, scope, &self.ext_gstates, n, &self.fallback)) { (self.gs.font, self.gs.vertical, self.gs.size) = (font, vertical, size); + // As for `Tf`: a Type 3 font without /Resources paints through this scope's. + self.gs.type3_inherits = op.name(0).is_some_and(|n| extgstate_type3_inherits(doc, &self.ext_gstates, n)); } } } @@ -507,6 +591,7 @@ impl Walk { let Some(name) = op.name(0) else { return }; let font_dict = self.fonts_res.get(name).and_then(|f| doc.resolve(f).as_dict().cloned()); self.gs.vertical = font_dict.as_ref().is_some_and(|d| is_vertical(doc, d)); + self.gs.type3_inherits = font_dict.as_ref().is_some_and(|d| type3_without_resources(doc, d)); self.gs.font = match scope.fonts.get(name) { Some(f) => f.clone(), None => { @@ -555,6 +640,16 @@ impl Walk { /// `Tj`, `TJ`, `'` and `"`: glyphs under a region are cut out (or counted, when verifying). fn show_text(&mut self, scope: &mut Scope<'_>, op: Op, piece: &mut Piece) -> Option { + // Text paints with the fill pattern, and with the stroke pattern in the stroking render + // modes. Mode 3 paints nothing and mode 7 only clips. + let mode = self.gs.render_mode; + uses_patterns(&mut self.out, &self.gs, matches!(mode, 0 | 2 | 4 | 6), matches!(mode, 1 | 2 | 5 | 6)); + // A Type 3 glyph without resources of its own paints through this scope's: keep every + // pattern it could select, as with a form that has no /Resources. + if self.gs.type3_inherits { + self.out.patterns_painted.extend(self.patterns.iter().cloned()); + self.out.keeps_all_patterns = true; + } let o = op.op.as_slice(); if o == b"\"" { self.gs.word_spacing = op.num(0).unwrap_or(self.gs.word_spacing); @@ -608,6 +703,8 @@ impl Walk { /// An inline image under a region is removed (or counted, when verifying). fn inline_image(gs: &Gs, scope: &mut Scope<'_>, out: &mut Output, op: Op, piece: &mut Piece) -> Option { + // An image mask paints with the fill colour, pattern included. + uses_patterns(out, gs, true, false); if !scope.hits(gs.ctm.bbox([0.0, 0.0, 1.0, 1.0])) { return Some(op); } @@ -632,6 +729,9 @@ impl Walk { } fn draw_xobject(&mut self, doc: &mut Document, scope: &mut Scope<'_>, resources: &Dict, op: Op, piece: &mut Piece) -> Option { + // An image mask paints with the fill colour, and a form's content is interpreted with a + // graphics state of its own, so a pattern it inherits from here is never recorded inside it. + uses_patterns(&mut self.out, &self.gs, true, true); let Some(new) = xobject(doc, scope, &op, &self.gs, resources, &mut self.xobjects, &mut self.out) else { return Some(op) }; piece.changed = true; piece.ops.extend(new); @@ -924,8 +1024,18 @@ fn xobject( out.ambiguous |= inner.ambiguous; out.doomed.extend_from_slice(&inner.doomed); out.retired_properties.extend(inner.retired_properties); + // A Type 3 glyph without resources shown inside the form may paint with the page's + // patterns (the renderer resolves its names there), whether or not the form has + // /Resources of its own. + out.keeps_all_patterns |= inner.keeps_all_patterns; out.tag_changes.extend(inner.tag_changes); out.error = inner.error.or(out.error.take()); + // A form without /Resources of its own paints with its caller's patterns, so what it + // still paints keeps the caller's entry. Only the painted side is passed up: a name + // the form selected but no longer paints with is left for the caller to keep. + if own.is_none() { + out.patterns_painted.extend(inner.patterns_painted); + } if scope.mode == Mode::Verify { return None; } @@ -992,3 +1102,18 @@ fn xobject( _ => None, } } + +/// Whether the ExtGState `name` sets a Type 3 font that has no `/Resources` of its own. +fn extgstate_type3_inherits(doc: &Document, ext_gstates: &Dict, name: &[u8]) -> bool { + let Some(state) = ext_gstates.get(name).map(|s| doc.resolve(s)) else { return false }; + let Some(font) = state.as_dict().and_then(|d| d.get(b"Font")).map(|f| doc.resolve(f)) else { return false }; + let Some(reference) = font.as_array().and_then(|a| a.first()) else { return false }; + let dict = doc.resolve(reference); + dict.as_dict().is_some_and(|d| type3_without_resources(doc, d)) +} + +/// A Type 3 font whose glyph descriptions use the resources of whatever shows them: no +/// `/Resources`, or `/Resources null`, which is the same thing. +fn type3_without_resources(doc: &Document, font: &Dict) -> bool { + font.name(b"Subtype") == Some(b"Type3") && font.get(b"Resources").is_none_or(|r| matches!(&*doc.resolve(r), Object::Null)) +} diff --git a/crates/redact/src/interp/prune.rs b/crates/redact/src/interp/prune.rs index 6ca5ea2803..188af03abe 100644 --- a/crates/redact/src/interp/prune.rs +++ b/crates/redact/src/interp/prune.rs @@ -355,3 +355,139 @@ pub(super) fn extgstate_font( }; Some((metrics, vertical, size)) } + +/// Drop every `/Pattern` entry this scope saw selected but no surviving painting operator drew +/// with. The `scn` that selected it stays in the content, so the page names a pattern its +/// resources no longer hold; nothing surviving paints with that colour, and +/// `a_page_whose_pruned_pattern_is_still_selected_renders` pins that the page still renders and +/// that operators after the stale selector still draw. A pattern's cell is a content stream of its own which +/// this pass does not enter, so a pattern whose only painting was redacted would otherwise keep +/// its drawing -- text included -- in the saved file (#3647). A pattern some surviving paint +/// still uses, or one the scope never saw, is left alone. +/// `/Pattern` may be an indirect object like any other value, so it is resolved: reading the +/// reference itself as a dictionary found none and pruned nothing, leaving the cell in the file. +/// What is written back is a dictionary of this page's own, so pruning never reaches through a +/// `/Pattern` dictionary another page shares. The pattern objects dropped are added to `pruned`, +/// for [`unlink_pruned_patterns`]. +pub(crate) fn prune_patterns(doc: &Document, res: &mut Dict, seen: &[Vec], painted: &[Vec], pruned: &mut Vec) -> usize { + let Some(mut pat) = res.get(b"Pattern").map(|p| doc.resolve(p)).and_then(|p| p.as_dict().cloned()) else { return 0 }; + let mut dropped = 0; + for n in seen { + if !painted.contains(n) + && let Some(value) = pat.get(n) + { + // Every object along the reference chain: a name can lead to a reference-valued + // object whose value is the cell, and the cell is what must not stay reachable. + let mut next = value.as_ref(); + for _ in 0..MAX_DEPTH { + let Some(reference) = next else { break }; + pruned.push(reference); + next = match &*doc.get(reference) { + Object::Ref(r) => Some(*r), + _ => None, + }; + } + pat.remove(n); + dropped += 1; + } + } + if dropped > 0 { + if pat.is_empty() { + res.remove(b"Pattern"); + } else { + res.set(b"Pattern".to_vec(), Object::Dict(pat)); + } + } + dropped +} + +/// Unlink the patterns a page's rewrite pruned (`pruned`) from page-tree nodes whose +/// `/Resources` no page inherits any more: a full save keeps whatever Root still reaches, so a +/// `/Pages` node above the page that still names the cell would keep it in the file. That happens +/// when the page's nearest holder was retired but a holder further up, which it shadowed, still +/// names the pattern. A holder some page still inherits from is left alone; whatever stays +/// reachable is checked by `pattern_guard`, which refuses a retained cell it cannot prove safe. +/// When any node's `/Parent` disagrees with where the `/Kids` tree puts it (or the root has +/// one), which holder a page inherits from is ambiguous and a node may be reached only through +/// `/Parent`, so nothing is unlinked and the guard, which follows `/Parent` too, decides. +pub(crate) fn unlink_pruned_patterns(doc: &mut Document, pruned: &[ObjRef]) -> Result<(), RedactError> { + if pruned.is_empty() { + return Ok(()); + } + let pruned: HashSet = pruned.iter().copied().collect(); + let Some(top) = doc.root().and_then(|root| doc.get(root).as_dict().and_then(|d| d.reference(b"Pages"))) else { return Ok(()) }; + // Every node with a /Resources of its own, and whether any page inherits from it (it is the + // nearest holder above a page, or the page itself). Cycles are cut as in `unlink_doomed`. + let mut holders: Vec<(ObjRef, bool)> = Vec::new(); + let mut seen = HashSet::new(); + let mut consistent = true; + let mut stack: Vec<(ObjRef, Option, Option)> = vec![(top, None, None)]; + seen.insert(top); + while let Some((node, parent, mut holder)) = stack.pop() { + let obj = doc.get(node); + let Some(d) = obj.as_dict() else { continue }; + // Every node's /Parent must be the node /Kids reached it from, and the root has none. + // A node reached only through a /Parent link is then impossible here; when the links + // disagree, the guard inspects every node they reach instead. A /Parent that is not a + // reference (a direct dictionary, written inline by a full save) never agrees. + let agrees = match d.get(b"Parent") { + None => parent.is_none(), + Some(Object::Ref(r)) => Some(*r) == parent, + Some(_) => false, + }; + if !agrees { + consistent = false; + break; + } + if d.contains(b"Resources") { + holders.push((node, false)); + holder = Some(holders.len() - 1); + } + let is_pages = d.name(b"Type") == Some(b"Pages") || (d.contains(b"Kids") && d.name(b"Type") != Some(b"Page")); + if is_pages { + // Borrowed, not copied: a /Kids array can be shared by many hostile nodes. + let kids_object = d.get(b"Kids").map(|k| doc.resolve(k)); + for kid in kids_object.as_deref().and_then(Object::as_array).into_iter().flatten().filter_map(Object::as_ref) { + // A node reached twice (a shared /Kids entry or a cycle) has a /Parent that agrees + // with at most one of the nodes above it. + if !seen.insert(kid) || seen.len() > 1_000_000 { + consistent = false; + break; + } + stack.push((kid, Some(node), holder)); + } + if !consistent { + break; + } + } else if let Some(h) = holder.and_then(|i| holders.get_mut(i)) { + h.1 = true; + } + } + if !consistent { + return Ok(()); + } + for (node, inherited) in holders { + if inherited { + continue; + } + let Some(mut res) = doc.get(node).as_dict().and_then(|d| d.get(b"Resources")).and_then(|r| doc.resolve(r).as_dict().cloned()) else { + continue; + }; + let mut pat = res_dict(doc, &res, b"Pattern"); + let names: Vec> = pat.iter().filter(|(_, v)| v.as_ref().is_some_and(|r| pruned.contains(&r))).map(|(n, _)| n.to_vec()).collect(); + if names.is_empty() { + continue; + } + for n in &names { + pat.remove(n); + } + if pat.is_empty() { + res.remove(b"Pattern"); + } else { + res.set(b"Pattern".to_vec(), Object::Dict(pat)); + } + // A private copy on this node: the dictionary object may be shared with other nodes. + doc.update_dict(node, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + } + Ok(()) +} diff --git a/crates/redact/src/lib.rs b/crates/redact/src/lib.rs index 6bb8e5cc06..6d323004ad 100644 --- a/crates/redact/src/lib.rs +++ b/crates/redact/src/lib.rs @@ -22,6 +22,7 @@ pub mod codes; mod image; mod interp; mod overlay; +mod pattern_guard; pub mod patterns; pub mod sanitize; mod tags; @@ -34,6 +35,7 @@ pub(crate) mod type3_tables; pub mod verify; use interp::{Mode, Scope, process}; +pub use pattern_guard::PatternProblem; pub use verify::{Proof, ProofOptions, Snapshot}; pub type Rgb = [f64; 3]; @@ -53,6 +55,8 @@ pub enum RedactError { #[error("redaction could not be verified: {0} item(s) were still found under the marks, so nothing was changed")] Residue(usize), /// A mark's overlay text has a character Helvetica can't draw; nothing was redacted. + #[error("page {page} cannot be redacted safely: {reason}; nothing was changed")] + Pattern { page: usize, reason: PatternProblem }, #[error("{0}")] Undrawable(String), #[error( @@ -146,6 +150,8 @@ fn overlay_look(doc: &Document, d: &Dict) -> pdfcraft_annot::OverlayLook { /// What [`apply`] removed. #[derive(Clone, Debug, Default, PartialEq)] pub struct Report { + /// Patterns dropped from a page's resources because nothing painted with them any more. + pub patterns_removed: usize, pub marks: usize, pub pages: usize, pub glyphs: usize, @@ -397,6 +403,38 @@ pub fn apply(doc: &mut Document, pages: Option<&[usize]>) -> Result Option> { + const MAX_REFERENCES: usize = 16_384; + let mut references = HashSet::new(); + let Some(ap) = annotation.get(b"AP") else { return Some(references) }; + if let Some(reference) = ap.as_ref() { + references.insert(reference); + } + let resolved_ap = doc.resolve(ap); + let Some(appearances) = resolved_ap.as_dict() else { return Some(references) }; + for key in [b"N".as_slice(), b"R", b"D"] { + let Some(value) = appearances.get(key) else { continue }; + if let Some(reference) = value.as_ref() { + references.insert(reference); + } + let resolved = doc.resolve(value); + if let Object::Dict(states) = &*resolved { + for (_, state) in states.iter() { + if let Some(reference) = state.as_ref() { + references.insert(reference); + if references.len() > MAX_REFERENCES { + return None; + } + } + } + } + } + Some(references) +} + /// [`apply`] without the copy: an error can leave `doc` part redacted. fn apply_to(doc: &mut Document, pages: Option<&[usize]>) -> Result { let images = pdfcraft_cos::ImageBudget::new(doc); @@ -429,6 +467,16 @@ fn apply_within(doc: &mut Document, pages: Option<&[usize]>, images: pdfcraft_co // content rewriting and verification instead of scanning all marks for every page. let groups: Vec<_> = chosen.chunk_by(|a, b| a.page == b.page).filter_map(|group| group.first().map(|mark| (mark.page, group))).collect(); let by_page: Vec<_> = groups.iter().map(|(page, _)| *page).collect(); + // A pattern pruned from a selected page can remain reachable through another page's + // resources. When a selected page had patterns, inspect every page after the rewrite. + let source_pages = pdfcraft_model::pages(doc); + let shared_pattern_possible = by_page.iter().any(|&pi| { + source_pages + .get(pi) + .and_then(|page| page.dict.get(b"Resources")) + .map(|resources| doc.resolve(resources)) + .is_some_and(|resources| resources.as_dict().is_some_and(|dict| dict.get(b"Pattern").is_some())) + }); // Budget failures must leave even a direct library caller's document untouched. let mut remaining = overlay::MAX_BYTES; let overlays = groups @@ -448,6 +496,9 @@ fn apply_within(doc: &mut Document, pages: Option<&[usize]>, images: pdfcraft_co widget_owner, removed_everywhere: HashSet::new(), images, + removed_appearances: HashSet::new(), + appearance_scan_limit: false, + pruned_patterns: Vec::new(), }; for &(pi, page_marks) in &groups { let page = pdfcraft_model::pages(doc).swap_remove(pi); @@ -458,7 +509,18 @@ fn apply_within(doc: &mut Document, pages: Option<&[usize]>, images: pdfcraft_co if applying.images.refused() > 0 { return Err(RedactError::ImagesPastAllowance); } - let Applying { mut report, doomed_fields, doomed_xobjects, retired_properties, mut removed_everywhere, mut images, .. } = applying; + let Applying { + mut report, + doomed_fields, + doomed_xobjects, + retired_properties, + mut removed_everywhere, + mut images, + mut removed_appearances, + mut appearance_scan_limit, + pruned_patterns, + .. + } = applying; // 2b. Replies and pop-ups of a removed annotation can be listed on another page than their // parent: their `/IRT` or `/Parent` would keep the removed comment reachable. @@ -481,6 +543,19 @@ fn apply_within(doc: &mut Document, pages: Option<&[usize]>, images: pdfcraft_co d.get(b"IRT").and_then(Object::as_ref).is_some_and(|x| removed_everywhere.contains(&x)) && d.name(b"Subtype") != Some(b"Widget"); let is_popup = d.get(b"Parent").and_then(Object::as_ref).is_some_and(|x| removed_everywhere.contains(&x)); if removed_everywhere.contains(&r) || is_reply || is_popup { + // A reply or pop-up removed here can share its appearance with an annotation + // that stays on an unselected page, like one removed under a mark: the + // all-pages pattern check must see it. + if !appearance_scan_limit { + match appearance_refs(doc, d) { + Some(references) => removed_appearances.extend(references), + None => appearance_scan_limit = true, + } + if removed_appearances.len() > 16_384 { + appearance_scan_limit = true; + removed_appearances.clear(); + } + } if removed_everywhere.insert(r) { grew = true; if is_reply { @@ -541,6 +616,8 @@ fn apply_within(doc: &mut Document, pages: Option<&[usize]>, images: pdfcraft_co // A removed image or form must not stay reachable through another resource dictionary (an // inherited or shared /Resources): a full save keeps whatever is still referenced. interp::unlink_doomed(doc, &doomed_xobjects, &retired_properties)?; + // Likewise a pruned pattern: a /Pages node above the page can still name its cell. + interp::unlink_pruned_patterns(doc, &pruned_patterns)?; report.images_restored = restore_shared_jbig2(doc, &doomed_xobjects, &mut images)?; // 5. Verify: nothing readable may remain under a region. @@ -563,6 +640,29 @@ fn apply_within(doc: &mut Document, pages: Option<&[usize]>, images: pdfcraft_co if residue > 0 { return Err(RedactError::Residue(residue)); } + // The interpreter can remove a pattern's paint while another use keeps the cell reachable. + // Inspect the finished selected pages, including rewritten Forms, and all pages if a removed + // pattern or appearance may still be shared. Retained unsafe cells refuse the redaction; + // `apply` runs on a copy, so the refusal leaves the caller's document as it was. + let shared_appearance_possible = appearance_scan_limit + || !removed_appearances.is_empty() + && pdfcraft_model::pages(doc).iter().enumerate().any(|(index, page)| { + !by_page.contains(&index) + && annots_of(doc, &page.dict).iter().any(|annotation| { + doc.resolve(annotation).as_dict().is_some_and(|dict| { + appearance_refs(doc, dict).is_none_or(|refs| refs.iter().any(|reference| removed_appearances.contains(reference))) + }) + }) + }); + let first_page = by_page.first().map_or(1, |p| p + 1); + let check_pages = + if shared_pattern_possible || shared_appearance_possible { (0..pdfcraft_model::pages(doc).len()).collect::>() } else { by_page }; + // A pruned pattern may also stay named by a /Pages node no page inherits from, or by one a + // page with a broken /Parent link still reaches: then every page-tree node is inspected too. + pattern_guard::check(doc, &check_pages, shared_pattern_possible || !pruned_patterns.is_empty())?; + // Whatever reference keeps a pruned pattern in the file -- one no walk above follows -- the + // full save would write it: prove it safe or refuse. + pattern_guard::check_pruned_unreachable(doc, &pruned_patterns, first_page)?; // The previous revision still holds the removed content: the next save must rewrite, and the // rewritten file is a different file from its source, so it gets a new identifier. doc.require_full_save_with_new_id(); @@ -650,6 +750,13 @@ struct Applying { removed_everywhere: HashSet, /// What clearing images may spend, across the run's pages. images: pdfcraft_cos::ImageBudget, + /// Appearance streams of the annotations removed, on any page: one shared with a kept + /// annotation keeps a pruned pattern's cell reachable (see `pattern_guard`). + removed_appearances: HashSet, + /// Too many appearance references to track: the pattern guard inspects every page. + appearance_scan_limit: bool, + /// Pattern objects pruned from a page's resources, which page-tree nodes may still name. + pruned_patterns: Vec, } /// Step 1 for one page: rewrites its content so nothing readable stays under a mark, and cleans the tags it no longer shows. @@ -660,7 +767,7 @@ fn redact_page_content( page: &pdfcraft_model::Page, rects: &[[f64; 4]], ) -> Result<(), RedactError> { - let Applying { report, doomed_xobjects, retired_properties, images, .. } = applying; + let Applying { report, doomed_xobjects, retired_properties, images, pruned_patterns, .. } = applying; // 1. Content. let mut budget = pdfcraft_cos::DecodeBudget::page(); let (list, data) = page_streams_within(doc, &page.dict, pi, &mut budget)?; @@ -733,6 +840,11 @@ fn redact_page_content( }, pi, )?; + // A pattern this page selected but no surviving paint draws leaves its resources, so + // its cell -- a content stream this pass does not enter -- cannot stay in the file. + // A Type 3 glyph without resources paints with patterns nobody tracked: keep them all. + let painted = if out.keeps_all_patterns { &out.patterns_seen } else { &out.patterns_painted }; + report.patterns_removed += interp::prune_patterns(doc, &mut res, &out.patterns_seen, painted, pruned_patterns); retired_properties.extend(interp::prune_properties(doc, &mut res, &cleaned_data)?); type3::prune_type3_from_resources(doc, &mut res, &cleaned_data)?; doc.update_dict(page.obj, |d| { @@ -761,7 +873,7 @@ fn remove_page_annotations( rects: &[[f64; 4]], page_marks: &[&Mark], ) -> Result<(), RedactError> { - let Applying { report, doomed_fields, widget_owner, removed_everywhere, .. } = applying; + let Applying { report, doomed_fields, widget_owner, removed_everywhere, removed_appearances, appearance_scan_limit, .. } = applying; // 2. Annotations: the marks, and whatever lies under them (with their pop-ups). let annots = annots_of(doc, &doc.get(page.obj).as_dict().cloned().unwrap_or_default()); let mut removed: Vec = Vec::new(); @@ -819,6 +931,25 @@ fn remove_page_annotations( } } removed_everywhere.extend(removed.iter().copied()); + // A removed annotation's appearance streams can be shared with a kept one on another page, + // which would keep a pruned pattern's cell reachable: remember them for the pattern guard. + for reference in &removed { + if *appearance_scan_limit { + break; + } + if let Some(annotation) = doc.get(*reference).as_dict() { + match appearance_refs(doc, annotation) { + Some(references) => { + removed_appearances.extend(references); + if removed_appearances.len() > 16_384 { + *appearance_scan_limit = true; + removed_appearances.clear(); + } + } + None => *appearance_scan_limit = true, + } + } + } // Pop-ups dropped with their parent below count too: a reply on another page may answer one. for a in &annots { let Some(r) = a.as_ref() else { continue }; diff --git a/crates/redact/src/pattern_guard.rs b/crates/redact/src/pattern_guard.rs new file mode 100644 index 0000000000..96b865e3da --- /dev/null +++ b/crates/redact/src/pattern_guard.rs @@ -0,0 +1,496 @@ +//! Refuse pattern content that remains reachable after redaction (#3647). +//! +//! Pattern cells paint through a path, not `Do`, so clipping/removing the path does not remove +//! their bytes. Inspect the rewritten pages' resource graphs before committing the +//! working copy. This is deliberately conservative: an unused text/image-bearing pattern is +//! refused too. A plain vector pattern remains supported. The bounded read-only pass follows +//! the fail-closed approach proposed in PR #162. + +use std::collections::HashSet; + +use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream}; + +use crate::RedactError; + +/// How deep the walk may recurse. Every step down (a reference, an array, a dictionary, a stream, +/// a resources map) is one frame, and a nested form costs about four of them: the 50-deep chain +/// in `a_deeply_nested_form_keeps_the_text_nobody_marked` peaks at 200. The interpreter follows +/// forms as deep as the renderer and Edit text read (`interp::MAX_DEPTH`, #3645), so this must +/// reach at least that far, or the guard would refuse a document the redaction itself handles. +/// Total work is bounded separately by `MAX_OBJECTS`, and 512 frames stay far inside a thread's +/// stack. +const MAX_DEPTH: usize = 512; +const _: () = assert!(MAX_DEPTH >= 8 * crate::interp::MAX_DEPTH, "the guard must walk past the deepest form the interpreter follows"); +const MAX_OBJECTS: usize = 16_384; +const MAX_STREAM_BYTES: usize = 1 << 20; +const MAX_TOTAL_BYTES: usize = 8 << 20; +const MAX_FILTERS: usize = 16; + +#[derive(Clone, Copy, Debug, PartialEq, Eq, thiserror::Error)] +pub enum PatternProblem { + #[error("a tiling pattern carries text or images that cannot be removed safely")] + Content, + #[error("pattern resources or content cannot be read completely")] + Unreadable, + #[error("pattern inspection exceeded its resource, depth or decoded-content budget")] + Limit, +} + +struct Guard { + left: usize, + bytes_left: usize, + seen: HashSet<(ObjRef, bool)>, + active: HashSet<(ObjRef, bool)>, +} + +impl Guard { + fn new() -> Self { + Self { left: MAX_OBJECTS, bytes_left: MAX_TOTAL_BYTES, seen: HashSet::new(), active: HashSet::new() } + } + + fn charge(&mut self, depth: usize) -> Result<(), PatternProblem> { + self.left = self.left.checked_sub(1).ok_or(PatternProblem::Limit)?; + if depth > MAX_DEPTH { + return Err(PatternProblem::Limit); + } + Ok(()) + } + + fn walk(&mut self, doc: &Document, object: &Object, inside: bool, depth: usize) -> Result<(), PatternProblem> { + self.charge(depth)?; + match object { + Object::Ref(reference) => { + let key = (*reference, inside); + if inside && self.active.contains(&key) { + return Err(PatternProblem::Unreadable); + } + if !self.seen.insert(key) { + return Ok(()); + } + let value = doc.try_get(reference.num).map_err(|_| PatternProblem::Unreadable)?; + self.active.insert(key); + let result = self.walk(doc, &value, inside, depth + 1); + self.active.remove(&key); + result + } + Object::Array(values) => { + for value in values { + self.walk(doc, value, inside, depth + 1)?; + } + Ok(()) + } + Object::Dict(dict) => self.dictionary(doc, dict, inside, false, depth + 1), + Object::Stream(stream) => { + if inside { + let subtype = resolved(doc, stream.dict.get(b"Subtype"))?; + let pattern_type = resolved(doc, stream.dict.get(b"PatternType"))?; + if subtype.as_name() == Some(b"Image") { + return Err(PatternProblem::Content); + } + if subtype.as_name() == Some(b"Form") { + if resolved(doc, stream.dict.get(b"Subtype2"))? != Object::Null || resolved(doc, stream.dict.get(b"Ref"))? != Object::Null { + return Err(PatternProblem::Unreadable); + } + self.content(doc, stream)?; + } else if pattern_type.as_int() == Some(1) { + self.content(doc, stream)?; + } + } + self.dictionary(doc, &stream.dict, inside, true, depth + 1) + } + _ => Ok(()), + } + } + + fn dictionary(&mut self, doc: &Document, dict: &Dict, inside: bool, stream: bool, depth: usize) -> Result<(), PatternProblem> { + // Stream dictionaries include appearance and soft-mask streams whose consumers + // may infer Form semantics without /Subtype. Resource-map names, in contrast, + // are arbitrary and must not be mistaken for dictionary field names. + let subtype = resolved(doc, dict.get(b"Subtype"))?; + let pattern_type = resolved(doc, dict.get(b"PatternType"))?; + let owns_resources = stream || matches!(subtype.as_name(), Some(b"Form" | b"Type3")) || pattern_type.as_int() == Some(1); + for (key, value) in dict.iter() { + if owns_resources && key.as_slice() == b"Resources" { + self.resources(doc, value, inside, depth + 1)?; + } else if inside || !owns_resources { + self.walk(doc, value, inside, depth + 1)?; + } + } + Ok(()) + } + + fn resources(&mut self, doc: &Document, object: &Object, inside: bool, depth: usize) -> Result<(), PatternProblem> { + self.charge(depth)?; + let object = resolved(doc, Some(object))?; + // A Form's null Resources is equivalent to an absent entry: it uses the already + // inspected caller resources. Tiling-pattern resources are required separately. + if object == Object::Null { + return Ok(()); + } + let Some(resources) = object.as_dict() else { return Err(PatternProblem::Unreadable) }; + for (category, value) in resources.iter() { + // Resource dictionaries may include large embedded fonts and colour tables. Only + // structures that can own another pattern cell need recursive inspection. + if !matches!(category.as_slice(), b"Pattern" | b"XObject" | b"Font" | b"Properties" | b"ExtGState") { + continue; + } + if category.as_slice() == b"Font" { + let fonts = resolved(doc, Some(value))?; + let Some(fonts) = fonts.as_dict() else { return Err(PatternProblem::Unreadable) }; + for (_, font) in fonts.iter() { + self.charge(depth)?; + let font = resolved(doc, Some(font))?; + let Some(dict) = font.as_dict() else { return Err(PatternProblem::Unreadable) }; + if resolved(doc, dict.get(b"Subtype"))?.as_name() == Some(b"Type3") + && let Some(resources) = dict.get(b"Resources") + { + self.resources(doc, resources, inside, depth + 1)?; + } + } + continue; + } + if matches!(category.as_slice(), b"Pattern" | b"XObject" | b"Properties" | b"ExtGState") && resolved(doc, Some(value))? == Object::Null { + continue; + } + if category.as_slice() == b"Pattern" { + let object = resolved(doc, Some(value))?; + let Some(patterns) = object.as_dict() else { return Err(PatternProblem::Unreadable) }; + for (_, pattern) in patterns.iter() { + self.charge(depth)?; + let object = resolved(doc, Some(pattern))?; + if object == Object::Null { + continue; + } + let Some(dict) = object.as_dict() else { return Err(PatternProblem::Unreadable) }; + let kind = resolved(doc, dict.get(b"PatternType"))?; + match (kind.as_int(), &object) { + (Some(1), Object::Stream(_)) => { + // Do not guess a fallback for a malformed cell, which different + // readers may resolve differently. Pattern resources are mandatory. + if resolved(doc, dict.get(b"Resources"))?.as_dict().is_none() { + return Err(PatternProblem::Unreadable); + } + } + (Some(2), Object::Dict(_)) => {} + _ => return Err(PatternProblem::Unreadable), + } + self.walk(doc, pattern, true, depth + 1)?; + } + } else { + if (inside && matches!(category.as_slice(), b"XObject" | b"Properties")) || category.as_slice() == b"ExtGState" { + let object = resolved(doc, Some(value))?; + let Some(entries) = object.as_dict() else { return Err(PatternProblem::Unreadable) }; + for (_, entry) in entries.iter() { + self.charge(depth)?; + let object = resolved(doc, Some(entry))?; + if object == Object::Null { + continue; + } + if category.as_slice() == b"Properties" { + let Some(properties) = object.as_dict() else { return Err(PatternProblem::Unreadable) }; + replacement_text(doc, properties)?; + } else if category.as_slice() == b"ExtGState" { + let Some(state) = object.as_dict() else { return Err(PatternProblem::Unreadable) }; + match resolved(doc, state.get(b"SMask"))? { + Object::Null => {} + Object::Name(name) if name == b"None" => {} + Object::Dict(mask) => { + let group = resolved(doc, mask.get(b"G"))?; + let Object::Stream(group) = group else { return Err(PatternProblem::Unreadable) }; + readable_form(doc, &group)?; + if let Some(value) = mask.get(b"G") { + self.walk(doc, value, inside, depth + 1)?; + } + } + _ => return Err(PatternProblem::Unreadable), + } + } else { + let Object::Stream(stream) = object else { return Err(PatternProblem::Unreadable) }; + let subtype = resolved(doc, stream.dict.get(b"Subtype"))?; + if subtype.as_name() == Some(b"Image") { + return Err(PatternProblem::Content); + } + readable_form(doc, &stream)?; + } + } + } + if category.as_slice() == b"XObject" { + let entries = resolved(doc, Some(value))?; + let Some(entries) = entries.as_dict() else { return Err(PatternProblem::Unreadable) }; + for (_, entry) in entries.iter() { + self.charge(depth)?; + let object = resolved(doc, Some(entry))?; + if let Object::Stream(stream) = object + && resolved(doc, stream.dict.get(b"Subtype"))?.as_name() == Some(b"Form") + { + self.walk(doc, entry, inside, depth + 1)?; + } + } + } + } + } + Ok(()) + } + + fn content(&mut self, doc: &Document, stream: &Stream) -> Result<(), PatternProblem> { + // Strict decoding with a small bound, unlike the tolerant general reader. Normalize + // indirect filter declarations before using Stream::filters, which accepts direct data. + let normalized = normalized_stream(doc, stream)?; + let chain = normalized.filters(); + let max = MAX_STREAM_BYTES.min(self.bytes_left); + if chain.is_empty() && normalized.raw.len() > max { + return Err(PatternProblem::Limit); + } + let data = pdfcraft_filters::decode(&chain, &normalized.raw, max).map_err(|error| match error { + pdfcraft_filters::FilterError::LimitExceeded(_) => PatternProblem::Limit, + _ => PatternProblem::Unreadable, + })?; + self.bytes_left = self.bytes_left.checked_sub(data.len()).ok_or(PatternProblem::Limit)?; + let parsed = pdfcraft_content::parse(&data); + if parsed.skipped != 0 { + return Err(PatternProblem::Unreadable); + } + for op in parsed.ops { + self.charge(0)?; + if matches!(op.op.as_slice(), b"BDC" | b"DP") { + for operand in &op.operands { + if let Some(properties) = resolved(doc, Some(operand))?.as_dict() { + replacement_text(doc, properties)?; + } + } + } + match op.op.as_slice() { + b"Tj" | b"TJ" | b"'" | b"\"" | b"BI" => return Err(PatternProblem::Content), + // Unknown operators cannot establish that no hidden content was skipped. + b"q" | b"Q" | b"cm" | b"w" | b"J" | b"j" | b"M" | b"d" | b"ri" | b"i" | b"gs" | b"m" | b"l" | b"c" | b"v" | b"y" | b"h" | b"re" + | b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"n" | b"W" | b"W*" | b"BT" | b"ET" | b"Tc" | b"Tw" | b"Tz" + | b"TL" | b"Tf" | b"Tr" | b"Ts" | b"Td" | b"TD" | b"Tm" | b"T*" | b"d0" | b"d1" | b"CS" | b"cs" | b"SC" | b"SCN" | b"sc" | b"scn" + | b"G" | b"g" | b"RG" | b"rg" | b"K" | b"k" | b"sh" | b"Do" | b"MP" | b"DP" | b"BMC" | b"BDC" | b"EMC" | b"BX" | b"EX" => {} + _ => return Err(PatternProblem::Unreadable), + } + } + Ok(()) + } +} + +fn readable_form(doc: &Document, stream: &Stream) -> Result<(), PatternProblem> { + // PS, Form/Subtype2 PS, reference XObjects and unknown/missing subtypes cannot be + // proven free of recoverable pattern content by the PDF content parser. + if resolved(doc, stream.dict.get(b"Subtype"))?.as_name() != Some(b"Form") + || resolved(doc, stream.dict.get(b"Subtype2"))? != Object::Null + || resolved(doc, stream.dict.get(b"Ref"))? != Object::Null + { + return Err(PatternProblem::Unreadable); + } + Ok(()) +} + +fn replacement_text(doc: &Document, properties: &Dict) -> Result<(), PatternProblem> { + for key in [b"ActualText".as_slice(), b"Alt", b"E"] { + if resolved(doc, properties.get(key))? != Object::Null { + return Err(PatternProblem::Content); + } + } + Ok(()) +} + +fn resolved(doc: &Document, value: Option<&Object>) -> Result { + let mut value = value.cloned().unwrap_or(Object::Null); + for _ in 0..MAX_DEPTH { + let Object::Ref(reference) = value else { return Ok(value) }; + value = (*doc.try_get(reference.num).map_err(|_| PatternProblem::Unreadable)?).clone(); + } + Err(PatternProblem::Limit) +} + +fn normalized_stream(doc: &Document, stream: &Stream) -> Result { + if stream.dict.contains(b"F") || stream.dict.contains(b"FFilter") || stream.dict.contains(b"FDecodeParms") || stream.dict.contains(b"DP") { + return Err(PatternProblem::Unreadable); + } + let mut normalized = stream.clone(); + let filter = resolved(doc, stream.dict.get(b"Filter"))?; + let names = match &filter { + Object::Null => Vec::new(), + Object::Name(_) => vec![filter], + Object::Array(values) if values.len() <= MAX_FILTERS => values.iter().map(|v| resolved(doc, Some(v))).collect::, _>>()?, + _ => return Err(PatternProblem::Unreadable), + }; + if names.iter().any(|name| name.as_name().is_none()) { + return Err(PatternProblem::Unreadable); + } + let parms = resolved(doc, stream.dict.get(b"DecodeParms"))?; + let parms = match parms { + Object::Null => vec![Object::Null; names.len()], + Object::Dict(_) if names.len() == 1 => vec![parms], + Object::Array(values) if values.len() == names.len() => values, + _ => return Err(PatternProblem::Unreadable), + }; + let parms = parms + .iter() + .map(|value| { + let value = resolved(doc, Some(value))?; + let Object::Dict(mut dict) = value else { + return if value == Object::Null { Ok(value) } else { Err(PatternProblem::Unreadable) }; + }; + for key in [b"Predictor".as_slice(), b"Colors", b"BitsPerComponent", b"Columns", b"EarlyChange"] { + if let Some(value) = dict.get(key) { + let value = resolved(doc, Some(value))?; + if value.as_int().is_none() { + return Err(PatternProblem::Unreadable); + } + dict.set(key.to_vec(), value); + } + } + if !matches!(dict.int(b"Predictor").unwrap_or(1), 1 | 2 | 10..=15) + || dict.int(b"Colors").unwrap_or(1) <= 0 + || dict.int(b"Columns").unwrap_or(1) <= 0 + || !matches!(dict.int(b"BitsPerComponent").unwrap_or(8), 1 | 2 | 4 | 8 | 16) + || !matches!(dict.int(b"EarlyChange").unwrap_or(1), 0 | 1) + { + return Err(PatternProblem::Unreadable); + } + Ok(Object::Dict(dict)) + }) + .collect::, _>>()?; + normalized.dict.set(b"Filter".to_vec(), Object::Array(names)); + normalized.dict.set(b"DecodeParms".to_vec(), Object::Array(parms)); + Ok(normalized) +} + +/// Page-tree nodes a walk of the tree may visit, pages included, before it refuses: the page tree +/// is walked whole elsewhere too (`pdfcraft_model::pages`, `unlink_doomed`), so a long document +/// must not refuse here. +const MAX_TREE_NODES: usize = 1_000_000; + +/// Every node a full save reaches through the page tree -- down `/Kids` and up the `/Parent` of +/// every node visited, page or `/Pages`, so a node reached only through a `/Parent` link counts +/// too -- that has `/Resources` of its own and is not one of the document's pages: those are +/// walked with their effective resources. Cycles are cut by the seen-set. Holders are bounded by +/// `MAX_OBJECTS`, pages are not. +fn tree_holders(doc: &Document) -> Result, PatternProblem> { + let Some(top) = doc.root().and_then(|root| doc.get(root).as_dict().and_then(|d| d.reference(b"Pages"))) else { return Ok(Vec::new()) }; + let pages: HashSet = pdfcraft_model::pages(doc).iter().map(|p| p.obj).collect(); + // The `/Resources` values to inspect. A direct `/Parent` dictionary is written inline by a + // full save, so it is a node like any other: its resources, `/Parent` and `/Kids` count. + let mut holders = Vec::new(); + let mut seen = HashSet::new(); + seen.insert(top); + let mut stack = vec![top]; + let mut inline: Vec = Vec::new(); + // One node: queue what it links to that was not seen yet (so a /Kids array shared by many + // nodes costs its length once per node, never a copy), and keep its resources. + let mut visit = + |dict: &Dict, is_page: bool, stack: &mut Vec, inline: &mut Vec, holders: &mut Vec| -> Result<(), PatternProblem> { + match dict.get(b"Parent") { + Some(Object::Ref(parent)) => { + if seen.insert(*parent) { + stack.push(*parent); + } + } + Some(Object::Dict(parent)) => { + if inline.len() >= MAX_DEPTH { + return Err(PatternProblem::Limit); + } + inline.push(parent.clone()); + } + _ => {} + } + if let Some(resources) = dict.get(b"Resources") + && !is_page + { + holders.push(resources.clone()); + if holders.len() > MAX_OBJECTS { + return Err(PatternProblem::Limit); + } + } + let is_pages = dict.name(b"Type") == Some(b"Pages") || (dict.contains(b"Kids") && dict.name(b"Type") != Some(b"Page")); + if is_pages { + let kids_object = match dict.get(b"Kids") { + Some(Object::Ref(reference)) => Some(doc.try_get(reference.num).map_err(|_| PatternProblem::Unreadable)?), + _ => None, + }; + let kids = match &kids_object { + Some(object) => object.as_array(), + None => dict.get(b"Kids").and_then(Object::as_array), + }; + for kid in kids.into_iter().flatten().filter_map(Object::as_ref) { + if seen.insert(kid) { + stack.push(kid); + } + } + } + if seen.len() > MAX_TREE_NODES { + return Err(PatternProblem::Limit); + } + Ok(()) + }; + loop { + if let Some(node) = stack.pop() { + let object = doc.try_get(node.num).map_err(|_| PatternProblem::Unreadable)?; + if let Some(dict) = object.as_dict() { + visit(dict, pages.contains(&node), &mut stack, &mut inline, &mut holders)?; + } + } else if let Some(dict) = inline.pop() { + visit(&dict, false, &mut stack, &mut inline, &mut holders)?; + } else { + break; + } + } + Ok(holders) +} + +/// Validate the requested pages before `apply` commits its working copy. With `tree`, the +/// `/Resources` of every page-tree node above the pages is inspected as well: a node no page +/// inherits from, or one a page with a broken `/Parent` link still reaches, can keep a pruned +/// cell in the saved file. +pub(crate) fn check(doc: &Document, selected: &[usize], tree: bool) -> Result<(), RedactError> { + if tree && let Some(&first) = selected.first() { + let page = first.saturating_add(1); + let holders = tree_holders(doc).map_err(|reason| RedactError::Pattern { page, reason })?; + for resources in holders { + let mut guard = Guard::new(); + guard.resources(doc, &resources, false, 0).map_err(|reason| RedactError::Pattern { page, reason })?; + } + } + let pages = pdfcraft_model::pages(doc); + for &index in selected { + let Some(page) = pages.get(index) else { return Err(RedactError::Unreadable(index.saturating_add(1))) }; + let mut guard = Guard::new(); + let checked = (|| { + if let Some(resources) = page.dict.get(b"Resources") { + guard.resources(doc, resources, false, 0)?; + } + // Kept appearance streams may own their own pattern resources. + for annotation in crate::annots_of(doc, &page.dict) { + let object = resolved(doc, Some(&annotation))?; + if let Some(appearance) = object.as_dict().and_then(|dict| dict.get(b"AP")) { + guard.walk(doc, appearance, false, 0)?; + } + } + Ok(()) + })(); + checked.map_err(|reason| RedactError::Pattern { page: index.saturating_add(1), reason })?; + } + Ok(()) +} + +/// The backstop after every other check: a full save writes whatever the trailer reaches, by any +/// reference at all -- a stray resource category, a key on a page, a direct /Parent, `/AcroForm +/// /DR`, `/PieceInfo` -- and the walks above follow only the references they know. A pattern +/// pruned from a page that a full save would still write must therefore be proven safe like any +/// retained cell (vector-only), or the redaction is refused. Costs one walk of the reachable +/// objects, and only when a pattern was pruned. +pub(crate) fn check_pruned_unreachable(doc: &Document, pruned: &[ObjRef], page: usize) -> Result<(), RedactError> { + if pruned.is_empty() { + return Ok(()); + } + let reachable = pdfcraft_cos::reachable_objects(doc)?; + let mut guard = Guard::new(); + let mut checked = HashSet::new(); + for &pattern in pruned { + if reachable.contains(&pattern.num) && checked.insert(pattern.num) { + guard.walk(doc, &Object::Ref(pattern), true, 0).map_err(|reason| RedactError::Pattern { page, reason })?; + } + } + Ok(()) +} diff --git a/crates/redact/src/tests.rs b/crates/redact/src/tests.rs index 78cff889d3..8978e0e7d6 100644 --- a/crates/redact/src/tests.rs +++ b/crates/redact/src/tests.rs @@ -290,6 +290,200 @@ fn vectors_are_removed_or_clipped_and_inline_images_go() { assert!(c.contains("150 150 m"), "the line elsewhere stays"); } +/// A tiling pattern's cell is a content stream of its own and can draw text. Redacting a shape +/// filled with such a pattern removed the fill from the page but left the pattern in the page's +/// resources, so its cell -- with the text that was under the mark -- stayed in the saved file +/// (#3647). A pattern nothing paints with any more now leaves the resources; one some surviving +/// paint still uses stays. +#[test] +fn a_pattern_whose_only_painting_was_redacted_leaves_the_page() { + fn page_with(fills: &str) -> Document { + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 10 150 Td (SECRET) Tj ET", + ); + let content = format!("BT /F1 10 Tf 10 280 Td (PUBLIC) Tj ET {fills}"); + one_page(content.as_bytes(), "/Pattern << /P0 7 0 R >>", vec![cell]) + } + let has_pattern = |doc: &Document| { + pdfcraft_model::pages(doc)[0] + .dict + .get(b"Resources") + .and_then(|r| doc.resolve(r).as_dict().cloned()) + .is_some_and(|res| res.get(b"Pattern").is_some()) + }; + + // One box, marked: the pattern is no longer painted, so it goes with the fill. + let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f"); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(report.patterns_removed, 1, "the pattern left the page"); + assert!(!has_pattern(&doc), "the page has no /Pattern left"); + let saved = write_incremental(&doc, &SaveOptions::default()).unwrap(); + assert!(!saved.windows(8).any(|w| w == b"(SECRET)"), "the cell's text is gone from the saved file"); + + // Two boxes share a text-bearing cell. The second still needs its pattern, but it is not + // safe to infer that it visibly displays every byte of the cell. + let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f /Pattern cs /P0 scn 20 40 100 30 re f"); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. }))); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "refusal keeps the pending mark and original bytes"); + + // Paint is not only path painting: text shows with the fill pattern too, and the text here + // survives the mark. Dropping the pattern would take the fill away from text still on the + // page, so a pattern is kept for every operator that can paint with it. + let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f BT /F1 10 Tf 10 240 Td (KEEP) Tj ET"); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. }))); + assert!(has_pattern(&doc), "the refused operation keeps the original pattern"); + assert!(content(&doc, 0).contains("(KEEP)"), "the text is still there: {}", content(&doc, 0)); + + // Text in render mode 3 paints nothing, so it keeps no pattern: the unpainted pattern goes. + let mut doc = page_with("/Pattern cs /P0 scn 20 140 100 30 re f BT 3 Tr /F1 10 Tf 10 240 Td (KEEP) Tj ET"); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + assert_eq!(apply(&mut doc, None).unwrap().patterns_removed, 1, "invisible text paints no pattern"); + + // `/Pattern` may be an indirect object, like any value. Reading the reference itself as a + // dictionary found none, so nothing was pruned and the cell stayed in the saved file. + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 10 150 Td (SECRET) Tj ET", + ); + let mut doc = one_page(b"/Pattern cs /P0 scn 20 140 100 30 re f", "/Pattern 8 0 R", vec![cell, b"<< /P0 7 0 R >>".to_vec()]); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + assert_eq!(apply(&mut doc, None).unwrap().patterns_removed, 1, "an indirect /Pattern is resolved before it is pruned"); + assert!(!has_pattern(&doc), "the page has no /Pattern left"); + let saved = write_incremental(&doc, &SaveOptions::default()).unwrap(); + assert!(!saved.windows(8).any(|w| w == b"(SECRET)"), "the cell's text is gone from the saved file"); +} + +/// Pruning takes the pattern out of `/Resources` but leaves the `/P0 scn` that selected it, so +/// the saved page names a pattern that is no longer there. ISO 32000-1 (8.6.6.2) says that +/// operand is a key of the `/Pattern` subdictionary, so this is what a reader is handed: the +/// question is whether it stops reading the stream there. Asked in the review of #3647. +#[test] +fn a_page_whose_pruned_pattern_is_still_selected_renders() { + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 10 150 Td (SECRET) Tj ET", + ); + // `0 g` after the fill puts the text back on a plain colour: with the pattern still selected + // the text would paint with it, and a pattern something still paints with is never pruned. + // The text is *after* the selector on purpose -- that is what a reader would lose if it gave + // up on the stale `/P0`. + let content_stream = b"/Pattern cs /P0 scn 20 140 100 30 re f 0 g BT /F1 24 Tf 10 40 Td (PUBLIC) Tj ET"; + let mut doc = one_page(content_stream, "/Pattern << /P0 7 0 R >>", vec![cell]); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + assert_eq!(apply(&mut doc, None).unwrap().patterns_removed, 1, "the pattern is pruned"); + + // The state this test is about: the selector outlived the resource it names. + let after = content(&doc, 0); + assert!(after.contains("/P0 scn"), "the selector is still in the content: {after}"); + assert!(!after.contains("20 140 100 30 re"), "the fill that used it is gone: {after}"); + + let bytes = write_full(&doc, &SaveOptions::default()).unwrap(); + assert!(!bytes.windows(8).any(|w| w == b"(SECRET)"), "the cell's text is gone from the saved file"); + let mut r = pdfcraft_render::PageRenderer::new(std::sync::Arc::new(bytes), pdfcraft_render::RenderConfig::default()); + let out = r.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + assert!(out.error.is_none(), "the page still renders: {:?}, warnings {:?}", out.error, out.warnings); + assert_eq!((out.width, out.height), (300, 300), "the page box is unchanged"); + + // The text after the stale selector is still drawn: 24 pt at y 40, so the top-left origin + // puts it in the rows around 240..265. Without it the reader gave up at the bad operand. + let painted = |rows: std::ops::Range, cols: std::ops::Range| { + rows.flat_map(|y| cols.clone().map(move |x| (y, x))).any(|(y, x)| { + let i = ((y * out.width + x) * 4) as usize; + out.rgba[i..i + 3] != [255, 255, 255] + }) + }; + assert!(painted(236..268, 8..120), "the text after the stale selector is still painted"); + // The mark itself is drawn over the area the pattern filled -- `apply` writes a black + // overlay there (`q 0 0 0 rg 15 135 115 40 re f Q`), so the cell cannot show through it + // even if a reader kept the stale selector. + let black = |y: u32, x: u32| { + let i = ((y * out.width + x) * 4) as usize; + out.rgba[i..i + 3] == [0, 0, 0] + }; + assert!(black(145, 70), "the redaction overlay is painted over the pattern's box"); +} + +#[test] +fn redaction_cannot_leave_hidden_cell_text_when_pattern_is_used_elsewhere() { + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"0.9 g 0 0 300 300 re f 0 g BT /F1 10 Tf 30 150 Td (SECRET) Tj ET", + ); + let content = b"/Pattern cs /P0 scn 20 140 100 30 re f 20 40 100 30 re f"; + let mut doc = one_page(content, "/Pattern << /P0 7 0 R >>", vec![cell]); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. }))); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "refusal must be atomic"); +} + +#[test] +fn redaction_cannot_leave_hidden_cell_text_on_an_unselected_page() { + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 30 150 Td (SECRET) Tj ET", + ); + let mut doc = pdf(vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R 8 0 R] /Count 2 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Pattern << /P0 7 0 R >> >> >>".to_vec(), + stream("", b"/Pattern cs /P0 scn 20 140 100 30 re f"), + FONT.replace("95 0 R", "6 0 R").into_bytes(), + widths(), + cell, + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 9 0 R /Resources << /Pattern << /P0 7 0 R >> >> >>".to_vec(), + stream("", b"/Pattern cs /P0 scn 20 40 100 30 re f"), + ]); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + match apply(&mut doc, Some(&[0])) { + Ok(_) => { + let saved = write_full(&doc, &SaveOptions::default()).unwrap(); + assert!(!saved.windows(8).any(|w| w == b"(SECRET)"), "the unselected page kept the hidden cell text"); + } + Err(RedactError::Pattern { reason: PatternProblem::Content, .. }) => { + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "refusal must keep both pages and the mark"); + } + Err(error) => panic!("unexpected refusal: {error}"), + } +} + +#[test] +fn redacted_form_must_not_retain_its_own_pattern_cell_text() { + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 30 150 Td (SECRET) Tj ET", + ); + let form = stream( + "/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Pattern << /P0 7 0 R >> >>", + b"/Pattern cs /P0 scn 20 140 100 30 re f", + ); + let mut doc = one_page(b"/Fm Do", "/XObject << /Fm 8 0 R >>", vec![cell, form]); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. }))); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "form-owned pattern refusal must be atomic"); +} + +#[test] +fn redacted_form_must_not_retain_inherited_pattern_cell_text() { + let cell = stream( + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 300 300] /XStep 300 /YStep 300 /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 30 150 Td (SECRET) Tj ET", + ); + let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"20 140 100 30 re f"); + let mut doc = one_page(b"/Pattern cs /P0 scn /Fm Do", "/Pattern << /P0 7 0 R >> /XObject << /Fm 8 0 R >>", vec![cell, form]); + mark(&mut doc, 0, &[[15.0, 135.0, 130.0, 175.0]], ""); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + assert!(matches!(apply(&mut doc, None), Err(RedactError::Pattern { reason: PatternProblem::Content, .. }))); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before, "inherited pattern refusal must be atomic"); +} + #[test] fn partially_covered_vector_paths_do_not_leak_drawing_commands() { // A vector path with detailed drawing commands (e.g. outlined text or confidential geometry) diff --git a/crates/redact/tests/pattern_page_tree.rs b/crates/redact/tests/pattern_page_tree.rs new file mode 100644 index 0000000000..c2b68a4485 --- /dev/null +++ b/crates/redact/tests/pattern_page_tree.rs @@ -0,0 +1,221 @@ +//! Pattern redaction through the page tree and through glyphs: synthetic files whose pattern +//! cells could stay reachable, or whose surviving paint could lose its pattern, after Apply. +//! Each test reads the saved bytes back (full and incremental, with and without object +//! streams) and searches every decoded stream. + +use std::sync::Arc; + +use pdfcraft_annot::{Meta, NewAnnotation, Shape, Style, add_annotation, rect_quad}; +use pdfcraft_cos::{Document, Object, SaveOptions, write_full, write_incremental}; +use pdfcraft_redact::{PatternProblem, RedactError, apply, marks}; +use pdfcraft_testkit::{Pdf, stream_obj}; + +const SECRET: &[u8] = b"ADV2CELLSECRET"; +const TEXT_CELL: &[u8] = b"BT /F1 20 Tf 40 120 Td (ADV2CELLSECRET) Tj ET\n"; +const PATTERN: &str = + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 500 300] /XStep 500 /YStep 300 /Resources << /Font << /F1 4 0 R >> >>"; +const FONT: &[u8] = b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>"; +const RES: &str = "<< /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> >>"; +const AREA: [f64; 4] = [30.0, 105.0, 215.0, 150.0]; +/// Public text outside the mark, and a pattern fill wholly under it. +const CONTENT: &[u8] = b"BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\n"; + +fn open(objs: Vec>) -> Document { + Document::open(Arc::new(Pdf::new().objs(objs).build())).unwrap() +} + +fn mark(doc: &mut Document, page: usize) { + let shape = Shape::Redact { quads: vec![rect_quad(AREA)], overlay: String::new(), look: Default::default() }; + let style = Style::default_for(&shape); + add_annotation(doc, &NewAnnotation { page, shape, style, contents: String::new(), author: "Synthetic test".into() }, &Meta::default()).unwrap(); +} + +fn decoded_streams_contain(bytes: Vec, needle: &[u8]) -> bool { + let doc = Document::open(Arc::new(bytes)).unwrap(); + doc.object_numbers().into_iter().any(|num| { + let object = doc.try_get(num).unwrap(); + matches!(&*object, Object::Stream(s) if s.decoded().unwrap_or_default().windows(needle.len()).any(|w| w == needle)) + }) +} + +/// The needle in any decoded stream, with and without object streams, and in an incremental save. +fn saved_contains(doc: &Document, needle: &[u8]) -> bool { + [true, false].into_iter().any(|object_streams| { + let opts = SaveOptions { object_streams, ..SaveOptions::default() }; + decoded_streams_contain(write_full(doc, &opts).unwrap(), needle) || decoded_streams_contain(write_incremental(doc, &opts).unwrap(), needle) + }) +} + +/// A successful apply must leave no trace of the cell; a refusal must leave the document as it was. +fn assert_confidential_or_refused(mut doc: Document) { + mark(&mut doc, 0); + assert!(saved_contains(&doc, SECRET), "the fixture must carry the secret"); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + match apply(&mut doc, None) { + Ok(report) => { + assert_eq!(report.paths_removed, 1, "{report:?}"); + assert!(!saved_contains(&doc, SECRET), "Apply succeeded but the saved file still holds the pattern cell's text ({report:?})"); + } + Err(error) => { + assert!(matches!(error, RedactError::Pattern { .. }), "{error}"); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before); + assert_eq!(marks(&doc).len(), 1); + } + } +} + +fn catalog() -> Vec { + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec() +} + +/// The page inherits its resources, but its `/Parent` does not lead back to the node it inherits +/// from (missing here). Readers resolve inheritance down the `/Kids` tree, so the page draws with +/// the root's `/Pattern`; that root keeps its `/Resources` after the page's copy is pruned. +#[test] +fn a_page_whose_parent_link_is_missing_does_not_keep_the_pruned_cell_in_the_page_tree() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]); + assert_confidential_or_refused(doc); +} + +/// As above with a `/Parent` that names a different, resource-less `/Pages` node. +#[test] +fn a_page_whose_parent_link_disagrees_with_the_tree_does_not_keep_the_pruned_cell() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /Parent 7 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + b"<< /Type /Pages /Kids [] /Count 0 >>".to_vec(), + ]); + assert_confidential_or_refused(doc); +} + +/// Two ancestors both hold `/Resources` naming the pattern; the page inherits from the nearer one, +/// which is retired once the page gets its own pruned copy. The root's dictionary, which no page +/// inherits, still names the cell. +#[test] +fn a_shadowed_ancestor_resource_dictionary_does_not_keep_the_pruned_cell() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [7 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /Parent 7 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("<< /Type /Pages /Parent 2 0 R /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + ]); + // The tree is consistent and no page inherits the root's dictionary any more, so the pruned + // pattern is unlinked from it and the redaction applies rather than refusing. + let mut doc = doc; + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(report.patterns_removed, 1, "{report:?}"); + assert!(!saved_contains(&doc, SECRET), "the root's shadowed /Resources still names the cell ({report:?})"); +} + +/// Control: the same shape with only the nearer ancestor holding resources applies cleanly. +#[test] +fn control_a_single_inheriting_ancestor_is_retired_with_the_cell() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [7 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 7 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("<< /Type /Pages /Parent 2 0 R /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + ]); + assert_confidential_or_refused(doc); +} + +/// A hostile chain of tiling patterns, each cell selecting the next, read from a file: the guard +/// must stop with an error (or apply), never overflow the stack. +#[test] +fn a_hostile_chain_of_nested_patterns_is_bounded() { + const N: usize = 3000; + let mut objs = vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources {RES} >>").into_bytes(), + FONT.to_vec(), + // The fill outside the mark keeps /P0 in use, so the guard must inspect the chain. + stream_obj("", b"/Pattern cs /P0 scn 35 110 170 35 re f /P0 scn 300 200 100 40 re f\n"), + ]; + for i in 0..N { + let num = 6 + i; + let (res, body) = if i + 1 < N { + (format!("/Resources << /Pattern << /Q {} 0 R >> >>", num + 1), b"/Pattern cs /Q scn 0 0 10 10 re f\n".to_vec()) + } else { + ("/Resources << /Font << /F1 4 0 R >> >>".to_string(), TEXT_CELL.to_vec()) + }; + objs.push(stream_obj(&format!("/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 10 10] /XStep 10 /YStep 10 {res}"), body)); + } + let mut doc = open(objs); + mark(&mut doc, 0); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + let error = apply(&mut doc, None).unwrap_err(); + assert!(matches!(error, RedactError::Pattern { reason: PatternProblem::Limit | PatternProblem::Content, .. }), "{error}"); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before); +} +/// A Type 3 font without `/Resources` draws its glyph with the page's `/P0` (ISO 32000-1 9.6.5: +/// names are then looked up in the page's resources). The glyph is outside the mark; the page's +/// own fill with `/P0` is under it. Pruning `/P0` loses the glyph's paint. +#[test] +fn a_type3_glyph_without_resources_keeps_the_page_pattern_it_paints_with() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R /T3 7 0 R >> /Pattern << /P0 6 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", b"BT /T3 1 Tf 300 230 Td (a) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\n"), + stream_obj(PATTERN, b"0 0 1 rg 0 0 500 300 re f\n"), + b"<< /Type /Font /Subtype /Type3 /FontBBox [0 0 40 40] /FontMatrix [1 0 0 1 0 0] /CharProcs << /a 8 0 R >> /Encoding << /Differences [97 /a] >> /FirstChar 97 /LastChar 97 /Widths [40] >>".to_vec(), + stream_obj("", b"40 0 d0 /Pattern cs /P0 scn 0 0 40 40 re f\n"), + ]); + let mut doc = doc; + let blue_glyph = |doc: &Document| { + let bytes = write_full(doc, &SaveOptions::default()).unwrap(); + let mut r = pdfcraft_render::PageRenderer::new(Arc::new(bytes), pdfcraft_render::RenderConfig::default()); + let out = r.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + // The glyph covers x 300..340, y 230..270 (rows 30..70 from the top). + let i = ((50 * out.width + 320) * 4) as usize; + out.rgba[i..i + 3].to_vec() + }; + let before = blue_glyph(&doc); + assert_eq!(before, vec![0, 0, 255], "the glyph paints with the page pattern before the redaction"); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(blue_glyph(&doc), before, "the unmarked glyph lost its pattern paint ({report:?})"); +} + +/// Text, inline images and XObjects record their fill pattern as they are met, before the hit +/// test, so a pattern whose only paint was text wholly under the mark is kept, and so is its cell. +/// That is the documented direction of error (keeping paint over losing it): a cell that carries +/// text or images is then refused by the guard, so what can stay is vector drawing nothing shows. +#[test] +fn a_pattern_painted_only_by_redacted_text_keeps_its_vector_cell() { + const DRAWING: &[u8] = b"0.1 0.2 0.3 rg 111.11 22.22 m 133.33 44.44 l 155.55 22.22 l f\n"; + let mut doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources {RES} >>").into_bytes(), + FONT.to_vec(), + stream_obj("", b"BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\nBT /F1 20 Tf /Pattern cs /P0 scn 40 120 Td (HIDDEN) Tj ET\n"), + stream_obj(PATTERN, DRAWING), + ]); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert!(report.glyphs >= 6, "{report:?}"); + assert_eq!(report.patterns_removed, 0, "text paint keeps its pattern"); + assert!(!saved_contains(&doc, b"(HIDDEN)"), "the glyphs are gone"); + assert!(saved_contains(&doc, b"133.33 44.44 l"), "the vector cell is kept, as documented"); +} diff --git a/crates/redact/tests/pattern_page_tree_shapes.rs b/crates/redact/tests/pattern_page_tree_shapes.rs new file mode 100644 index 0000000000..dafedd56b4 --- /dev/null +++ b/crates/redact/tests/pattern_page_tree_shapes.rs @@ -0,0 +1,275 @@ +//! Pattern redaction over unusual page trees -- cyclic, deep, wide, inconsistent, packed in +//! object streams, or reached only through `/Parent` -- and over Type 3 fonts reached other than +//! by `Tf` in the scope that paints. Each test reads the saved bytes back. + +use std::sync::Arc; + +use pdfcraft_annot::{Meta, NewAnnotation, Shape, Style, add_annotation, rect_quad}; +use pdfcraft_cos::{Document, Object, SaveOptions, write_full, write_incremental}; +use pdfcraft_redact::{RedactError, apply, marks}; +use pdfcraft_testkit::{Pdf, stream_obj}; + +const SECRET: &[u8] = b"ADV2CELLSECRET"; +const TEXT_CELL: &[u8] = b"BT /F1 20 Tf 40 120 Td (ADV2CELLSECRET) Tj ET\n"; +const PATTERN: &str = + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 500 300] /XStep 500 /YStep 300 /Resources << /Font << /F1 4 0 R >> >>"; +const FONT: &[u8] = b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>"; +const RES: &str = "<< /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> >>"; +const AREA: [f64; 4] = [30.0, 105.0, 215.0, 150.0]; +const CONTENT: &[u8] = b"BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\n"; + +fn open(objs: Vec>) -> Document { + Document::open(Arc::new(Pdf::new().objs(objs).build())).unwrap() +} + +fn mark(doc: &mut Document, page: usize) { + let shape = Shape::Redact { quads: vec![rect_quad(AREA)], overlay: String::new(), look: Default::default() }; + let style = Style::default_for(&shape); + add_annotation(doc, &NewAnnotation { page, shape, style, contents: String::new(), author: "Synthetic test".into() }, &Meta::default()).unwrap(); +} + +fn decoded_streams_contain(bytes: Vec, needle: &[u8]) -> bool { + let doc = Document::open(Arc::new(bytes)).unwrap(); + doc.object_numbers().into_iter().any(|num| { + let object = doc.try_get(num).unwrap(); + matches!(&*object, Object::Stream(s) if s.decoded().unwrap_or_default().windows(needle.len()).any(|w| w == needle)) + }) +} + +fn saved_contains(doc: &Document, needle: &[u8]) -> bool { + [true, false].into_iter().any(|object_streams| { + let opts = SaveOptions { object_streams, ..SaveOptions::default() }; + decoded_streams_contain(write_full(doc, &opts).unwrap(), needle) || decoded_streams_contain(write_incremental(doc, &opts).unwrap(), needle) + }) +} + +/// Round-trip through a full save with object streams, so every dictionary lives in one. +fn packed(doc: &Document) -> Document { + Document::open(Arc::new(write_full(doc, &SaveOptions { object_streams: true, ..SaveOptions::default() }).unwrap())).unwrap() +} + +/// Ok means confidential; Err must be a pattern refusal that leaves the document unchanged. +/// Returns whether it applied. +fn confidential_or_refused(mut doc: Document, page: usize) -> bool { + mark(&mut doc, page); + assert!(saved_contains(&doc, SECRET), "the fixture must carry the secret"); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + match apply(&mut doc, None) { + Ok(report) => { + assert_eq!(report.paths_removed, 1, "{report:?}"); + assert!(!saved_contains(&doc, SECRET), "Apply succeeded but the saved file still holds the pattern cell's text ({report:?})"); + true + } + Err(error) => { + assert!(matches!(error, RedactError::Pattern { .. }), "{error}"); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before); + assert_eq!(marks(&doc).len(), 1); + false + } + } +} + +fn catalog() -> Vec { + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec() +} + +/// The root `/Pages` node carries a `/Parent` to a node outside the `/Kids` tree that names the +/// pattern. A full save follows that `/Parent`, so the node and the cell are written. +#[test] +fn a_node_reached_only_through_the_roots_parent_does_not_keep_the_pruned_cell() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Parent 7 0 R /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("<< /Type /Pages /Kids [] /Count 0 /Resources {RES} >>").into_bytes(), + ]); + confidential_or_refused(doc, 0); +} +/// As above, one level down: an intermediate node in `/Kids` whose `/Parent` leads elsewhere. +#[test] +fn a_node_reached_only_through_an_intermediate_parent_does_not_keep_the_pruned_cell() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [8 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 8 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("<< /Type /Pages /Kids [] /Count 0 /Resources {RES} >>").into_bytes(), + format!("<< /Type /Pages /Parent 7 0 R /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + ]); + confidential_or_refused(doc, 0); +} +/// `/Parent` cycles: the root's parent is itself, and the page's chain loops through it. +#[test] +fn a_parent_cycle_is_confidential_or_refused() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Parent 2 0 R /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]); + confidential_or_refused(doc, 0); +} +/// A page 5,000 levels below the root that holds the pattern (past `resource_source`'s 4,096). +#[test] +fn a_very_deep_tree_is_confidential_or_refused() { + const DEPTH: usize = 5000; + let mut objs = vec![ + catalog(), + format!("<< /Type /Pages /Kids [7 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + format!("<< /Type /Page /Parent {} 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>", 7 + DEPTH - 1).into_bytes(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]; + for i in 0..DEPTH { + let num = 7 + i; + let parent = if i == 0 { 2 } else { num - 1 }; + let kid = if i + 1 == DEPTH { 3 } else { num + 1 }; + objs.push(format!("<< /Type /Pages /Parent {parent} 0 R /Kids [{kid} 0 R] /Count 1 >>").into_bytes()); + } + confidential_or_refused(doc_or(objs), 0); +} +fn doc_or(objs: Vec>) -> Document { + open(objs) +} +/// The shadowed-root and missing-parent shapes again, with every dictionary in object streams. +#[test] +fn page_tree_shapes_in_object_streams_are_confidential_or_refused() { + let shadowed = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [7 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /Parent 7 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("<< /Type /Pages /Parent 2 0 R /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + ]); + assert!(confidential_or_refused(packed(&shadowed), 0), "the shadowed root applies"); + let missing = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]); + confidential_or_refused(packed(&missing), 0); +} +/// The shadowed root's `/Resources` object is also a form's resources, drawn outside the mark on +/// the redacted page: unlinking from the root must not reach through to the form. +#[test] +fn a_root_resource_dictionary_shared_with_a_form_is_confidential_or_refused() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [7 0 R] /Count 1 /Resources 9 0 R >>".to_vec(), + b"<< /Type /Page /Parent 7 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", b"BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\nq 1 0 0 1 300 200 cm /Fm Do Q\n"), + stream_obj(PATTERN, TEXT_CELL), + b"<< /Type /Pages /Parent 2 0 R /Kids [3 0 R] /Count 1 /Resources << /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> /XObject << /Fm 8 0 R >> >> >>".to_vec(), + stream_obj("/Type /XObject /Subtype /Form /BBox [0 0 100 50] /Resources 9 0 R", b"0 0 1 rg 0 0 50 20 re f\n"), + RES.as_bytes().to_vec(), + ]); + confidential_or_refused(doc, 0); +} +/// A document of 17,000 pages whose first page names a pattern: only that page has patterns. +/// Redacting it should apply as it does in a short document. +#[test] +fn a_long_document_with_a_pattern_on_one_page_still_applies() { + const PAGES: usize = 17_000; + let kids: String = (0..PAGES).map(|i| format!("{} 0 R ", if i == 0 { 3 } else { 7 + i - 1 })).collect(); + let mut objs = vec![ + catalog(), + format!("<< /Type /Pages /Kids [{kids}] /Count {PAGES} >>").into_bytes(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources {RES} >>").into_bytes(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]; + for _ in 1..PAGES { + objs.push(b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] >>".to_vec()); + } + let mut doc = open(objs); + mark(&mut doc, 0); + let result = apply(&mut doc, None); + assert!(result.is_ok(), "a long document refuses: {result:?}"); + assert!(!saved_contains(&doc, SECRET)); +} +/// Control for the above: the same with 1,000 pages. +#[test] +fn control_a_shorter_document_with_a_pattern_on_one_page_applies() { + const PAGES: usize = 1000; + let kids: String = (0..PAGES).map(|i| format!("{} 0 R ", if i == 0 { 3 } else { 7 + i - 1 })).collect(); + let mut objs = vec![ + catalog(), + format!("<< /Type /Pages /Kids [{kids}] /Count {PAGES} >>").into_bytes(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources {RES} >>").into_bytes(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]; + for _ in 1..PAGES { + objs.push(b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] >>".to_vec()); + } + let mut doc = open(objs); + mark(&mut doc, 0); + assert!(apply(&mut doc, None).is_ok()); + assert!(!saved_contains(&doc, SECRET)); +} +fn glyph_pixel(doc: &Document, x: u32, row: u32) -> Vec { + let bytes = write_full(doc, &SaveOptions::default()).unwrap(); + let mut r = pdfcraft_render::PageRenderer::new(Arc::new(bytes), pdfcraft_render::RenderConfig::default()); + let out = r.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + let i = ((row * out.width + x) * 4) as usize; + out.rgba[i..i + 3].to_vec() +} +const T3: &[u8] = b"<< /Type /Font /Subtype /Type3 /FontBBox [0 0 40 40] /FontMatrix [1 0 0 1 0 0] /CharProcs << /a 8 0 R >> /Encoding << /Differences [97 /a] >> /FirstChar 97 /LastChar 97 /Widths [40] >>"; +const BLUE_CELL: &[u8] = b"0 0 1 rg 0 0 500 300 re f\n"; + +/// The Type 3 font without `/Resources` is selected through an ExtGState `/Font` entry, not `Tf`. +#[test] +fn a_type3_font_set_by_extgstate_keeps_the_page_pattern_it_paints_with() { + let mut doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /ExtGState << /G 9 0 R >> /Pattern << /P0 6 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", b"/G gs BT 300 230 Td (a) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\n"), + stream_obj(PATTERN, BLUE_CELL), + T3.to_vec(), + stream_obj("", b"40 0 d0 /Pattern cs /P0 scn 0 0 40 40 re f\n"), + b"<< /Type /ExtGState /Font [7 0 R 1] >>".to_vec(), + ]); + let before = glyph_pixel(&doc, 320, 50); + assert_eq!(before, vec![0, 0, 255], "the glyph paints with the page pattern before the redaction"); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(glyph_pixel(&doc, 320, 50), before, "the unmarked glyph lost its pattern paint ({report:?})"); +} +/// The Type 3 font without `/Resources` is shown inside a form that has its own `/Resources` +/// (with no `/Pattern`); the page paints `/P0` under the mark. +#[test] +fn a_type3_glyph_in_a_form_with_resources_renders_the_same_after_the_redaction() { + let mut doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /XObject << /Fm 9 0 R >> /Pattern << /P0 6 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", b"/Fm Do\n/Pattern cs /P0 scn 35 110 170 35 re f\n"), + stream_obj(PATTERN, BLUE_CELL), + T3.to_vec(), + stream_obj("", b"40 0 d0 /Pattern cs /P0 scn 0 0 40 40 re f\n"), + stream_obj("/Type /XObject /Subtype /Form /BBox [0 0 500 300] /Resources << /Font << /T3 7 0 R >> >>", b"BT /T3 1 Tf 300 230 Td (a) Tj ET\n"), + ]); + let before = glyph_pixel(&doc, 320, 50); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(glyph_pixel(&doc, 320, 50), before, "the unmarked glyph renders differently ({report:?}, before {before:?})"); +} diff --git a/crates/redact/tests/pattern_reachability.rs b/crates/redact/tests/pattern_reachability.rs new file mode 100644 index 0000000000..9ca4be0c27 --- /dev/null +++ b/crates/redact/tests/pattern_reachability.rs @@ -0,0 +1,304 @@ +//! Pattern redaction against references no walk follows -- a direct `/Parent`, `/Kids` on a +//! page, a stray resource entry -- Type 3 fonts reached in other ways, and hostile wide trees. +//! Each test reads the saved bytes back. + +use std::sync::Arc; + +use pdfcraft_annot::{Meta, NewAnnotation, Shape, Style, add_annotation, rect_quad}; +use pdfcraft_cos::{Document, Object, SaveOptions, write_full, write_incremental}; +use pdfcraft_redact::{RedactError, apply, marks}; +use pdfcraft_testkit::{Pdf, stream_obj}; + +const SECRET: &[u8] = b"ADV2CELLSECRET"; +const TEXT_CELL: &[u8] = b"BT /F1 20 Tf 40 120 Td (ADV2CELLSECRET) Tj ET\n"; +const PATTERN: &str = + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 500 300] /XStep 500 /YStep 300 /Resources << /Font << /F1 4 0 R >> >>"; +const FONT: &[u8] = b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>"; +const RES: &str = "<< /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> >>"; +const AREA: [f64; 4] = [30.0, 105.0, 215.0, 150.0]; +const CONTENT: &[u8] = b"BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\n"; + +fn open(objs: Vec>) -> Document { + Document::open(Arc::new(Pdf::new().objs(objs).build())).unwrap() +} + +fn mark(doc: &mut Document, page: usize) { + let shape = Shape::Redact { quads: vec![rect_quad(AREA)], overlay: String::new(), look: Default::default() }; + let style = Style::default_for(&shape); + add_annotation(doc, &NewAnnotation { page, shape, style, contents: String::new(), author: "Synthetic test".into() }, &Meta::default()).unwrap(); +} + +fn decoded_streams_contain(bytes: Vec, needle: &[u8]) -> bool { + let doc = Document::open(Arc::new(bytes)).unwrap(); + doc.object_numbers().into_iter().any(|num| { + let object = doc.try_get(num).unwrap(); + matches!(&*object, Object::Stream(s) if s.decoded().unwrap_or_default().windows(needle.len()).any(|w| w == needle)) + }) +} + +fn saved_contains(doc: &Document, needle: &[u8]) -> bool { + [true, false].into_iter().any(|object_streams| { + let opts = SaveOptions { object_streams, ..SaveOptions::default() }; + decoded_streams_contain(write_full(doc, &opts).unwrap(), needle) || decoded_streams_contain(write_incremental(doc, &opts).unwrap(), needle) + }) +} + +/// Ok means confidential; Err must be a pattern refusal that leaves the document unchanged. +/// Returns whether it applied. +fn confidential_or_refused(mut doc: Document, page: usize) -> bool { + mark(&mut doc, page); + assert!(saved_contains(&doc, SECRET), "the fixture must carry the secret"); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + match apply(&mut doc, None) { + Ok(report) => { + assert_eq!(report.paths_removed, 1, "{report:?}"); + assert!(!saved_contains(&doc, SECRET), "Apply succeeded but the saved file still holds the pattern cell's text ({report:?})"); + true + } + Err(error) => { + assert!(matches!(error, RedactError::Pattern { .. }), "{error}"); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before); + assert_eq!(marks(&doc).len(), 1); + false + } + } +} + +fn catalog() -> Vec { + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec() +} + +fn glyph_pixel(doc: &Document, x: u32, row: u32) -> Vec { + let bytes = write_full(doc, &SaveOptions::default()).unwrap(); + let mut r = pdfcraft_render::PageRenderer::new(Arc::new(bytes), pdfcraft_render::RenderConfig::default()); + let out = r.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + let i = ((row * out.width + x) * 4) as usize; + out.rgba[i..i + 3].to_vec() +} + +const T3: &[u8] = b"<< /Type /Font /Subtype /Type3 /FontBBox [0 0 40 40] /FontMatrix [1 0 0 1 0 0] /CharProcs << /a 8 0 R >> /Encoding << /Differences [97 /a] >> /FirstChar 97 /LastChar 97 /Widths [40] >>"; +const BLUE_CELL: &[u8] = b"0 0 1 rg 0 0 500 300 re f\n"; + +/// The root's `/Parent` is a direct dictionary, not a reference: a full save writes it inline +/// with the root, and the pattern it names with it. +#[test] +fn a_direct_parent_dictionary_on_the_root_does_not_keep_the_pruned_cell() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Parent << /Type /Pages /Kids [] /Count 0 /Resources {RES} >> /Resources {RES} >>") + .into_bytes(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]); + confidential_or_refused(doc, 0); +} +/// A leaf (`/Type /Page`) that also carries `/Kids`: readers stop at the leaf, a full save does +/// not, and the hidden node names the pattern. +#[test] +fn a_node_hidden_under_a_leaf_s_kids_does_not_keep_the_pruned_cell() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources {RES} /Kids [7 0 R] >>").into_bytes(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("<< /Type /Pages /Parent 3 0 R /Kids [] /Count 0 /Resources {RES} >>").into_bytes(), + ]); + confidential_or_refused(doc, 0); +} +/// A stray reference to the cell from a resource category the guard does not inspect +/// (`/Shading`): the class #3999 tracks, shown here for patterns. +#[test] +fn a_stray_resource_reference_to_the_cell_does_not_keep_it() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> /Shading << /Sh 6 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]); + confidential_or_refused(doc, 0); +} +fn type3_page(page_res_extra: &str, content: &[u8], extra: Vec>) -> Document { + let mut objs = vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R /T3 7 0 R >> /Pattern << /P0 6 0 R >> {page_res_extra} >> >>").into_bytes(), + FONT.to_vec(), + stream_obj("", content), + stream_obj(PATTERN, BLUE_CELL), + T3.to_vec(), + stream_obj("", b"40 0 d0 /Pattern cs /P0 scn 0 0 40 40 re f\n"), + ]; + objs.extend(extra); + open(objs) +} +fn assert_glyph_kept(mut doc: Document) { + let before = glyph_pixel(&doc, 320, 50); + assert_eq!(before, vec![0, 0, 255], "the glyph paints with the page pattern before the redaction"); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(glyph_pixel(&doc, 320, 50), before, "the unmarked glyph lost its pattern paint ({report:?})"); +} +/// The Type 3 font is shown in a form without `/Resources`, which draws through the page's. +#[test] +fn a_type3_glyph_in_a_resourceless_form_keeps_the_page_pattern() { + assert_glyph_kept(type3_page( + "/XObject << /Fm 9 0 R >>", + b"/Fm Do\n/Pattern cs /P0 scn 35 110 170 35 re f\n", + vec![stream_obj("/Type /XObject /Subtype /Form /BBox [0 0 500 300]", b"BT /T3 1 Tf 300 230 Td (a) Tj ET\n")], + )); +} +/// The Type 3 font is shown inside a pattern cell the page paints outside the mark: the cell +/// carries text, so the guard refuses rather than dropping anything. +#[test] +fn a_type3_glyph_inside_a_pattern_cell_is_refused() { + let mut doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R /P1 9 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", b"/Pattern cs /P1 scn 300 200 100 40 re f\n/Pattern cs /P0 scn 35 110 170 35 re f\n"), + stream_obj(PATTERN, BLUE_CELL), + T3.to_vec(), + stream_obj("", b"40 0 d0 /Pattern cs /P0 scn 0 0 40 40 re f\n"), + stream_obj("/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 500 300] /XStep 500 /YStep 300 /Resources << /Font << /T3 7 0 R >> >>", b"BT /T3 1 Tf 300 230 Td (a) Tj ET\n"), + ]); + mark(&mut doc, 0); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + let error = apply(&mut doc, None).unwrap_err(); + assert!(matches!(error, RedactError::Pattern { .. }), "{error}"); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before); +} +/// A hostile wide tree: `K` `/Pages` nodes that all share one `/Kids` array of `M` references +/// back to themselves, beside the real page. Time and memory are measured from outside +/// (`PDF_WIDE_K`, `PDF_WIDE_M`). Ignored by default. +#[test] +#[ignore] +fn hostile_wide_tree_probe() { + let k: usize = std::env::var("PDF_WIDE_K").ok().and_then(|v| v.parse().ok()).unwrap_or(200); + let m: usize = std::env::var("PDF_WIDE_M").ok().and_then(|v| v.parse().ok()).unwrap_or(500_000); + let mut array = String::with_capacity(m * 9); + for i in 0..m { + array.push_str(&format!("{} 0 R ", 10 + i % k)); + } + let mut objs = vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R 10 0 R] /Count 1 >>".to_vec(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources {RES} >>").into_bytes(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + format!("[{array}]").into_bytes(), + b"null".to_vec(), + b"null".to_vec(), + ]; + for _ in 0..k { + objs.push(b"<< /Type /Pages /Kids 7 0 R /Count 0 >>".to_vec()); + } + let mut doc = open(objs); + mark(&mut doc, 0); + let start = std::time::Instant::now(); + let result = apply(&mut doc, None); + eprintln!("WIDE k={k} m={m} apply={:?} in {:?}", result.as_ref().map(|r| r.patterns_removed), start.elapsed()); +} +/// The Type 3 font has `/Resources null`, which readers treat as absent. +#[test] +fn a_type3_font_with_null_resources_keeps_the_page_pattern() { + let mut doc = type3_page("", b"BT /T3 1 Tf 300 230 Td (a) Tj ET\n/Pattern cs /P0 scn 35 110 170 35 re f\n", vec![]); + doc.update_dict(pdfcraft_cos::ObjRef::new(7, 0), |d| d.set(b"Resources".to_vec(), Object::Null)).unwrap(); + assert_glyph_kept(doc); +} +/// Known limit (fidelity only, never a leak: a missed use can only prune more). Kept annotation +/// appearances are not interpreted for pattern use, so a Type 3 font without resources shown in +/// one does not keep the page's patterns. ISO 32000 resolves the glyph's names in the +/// appearance's own resources, where `/P0` does not exist, so only a lenient renderer draws it. +/// The Type 3 font is shown by a kept annotation's appearance stream (outside the mark). +#[test] +#[ignore = "known limit: kept annotation appearances are not interpreted, so a resourceless Type 3 glyph in one does not keep the page pattern; ISO 32000 resolves it in the appearance's own resources, which pdfcraft's renderer does not"] +fn a_type3_glyph_in_an_annotation_appearance_renders_the_same() { + let mut doc = type3_page( + "", + b"/Pattern cs /P0 scn 35 110 170 35 re f\n", + vec![ + stream_obj("/Type /XObject /Subtype /Form /BBox [0 0 40 40] /Resources << /Font << /T3 7 0 R >> >>", b"BT /T3 1 Tf 0 0 Td (a) Tj ET\n"), + b"<< /Type /Annot /Subtype /Stamp /Rect [300 230 340 270] /F 4 /AP << /N 9 0 R >> >>".to_vec(), + ], + ); + doc.update_dict(pdfcraft_cos::ObjRef::new(3, 0), |d| { + d.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(pdfcraft_cos::ObjRef::new(10, 0))])) + }) + .unwrap(); + let before = glyph_pixel(&doc, 320, 50); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(glyph_pixel(&doc, 320, 50), before, "the annotation renders differently ({report:?}, before {before:?})"); +} +/// A form wholly under the mark is removed; its own `/Pattern` cell carries text. Nothing prunes +/// that pattern, so the backstop never looks, and a stray reference (`/Shading`) keeps the cell. +#[test] +fn a_removed_form_s_pattern_does_not_survive_through_a_stray_reference() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /XObject << /Fm 7 0 R >> /Shading << /Sh 6 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", b"BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\nq 1 0 0 1 35 110 cm /Fm Do Q\n"), + stream_obj(PATTERN, TEXT_CELL), + stream_obj("/Type /XObject /Subtype /Form /BBox [0 0 170 35] /Resources << /Pattern << /P0 6 0 R >> >>", b"/Pattern cs /P0 scn 0 0 170 35 re f\n"), + ]); + let mut doc = doc; + mark(&mut doc, 0); + let before = write_full(&doc, &SaveOptions::default()).unwrap(); + match apply(&mut doc, None) { + Ok(report) => assert!(!saved_contains(&doc, SECRET), "Apply succeeded but the removed form's cell is in the saved file ({report:?})"), + Err(error) => { + assert!(matches!(error, RedactError::Pattern { .. }), "{error}"); + assert_eq!(write_full(&doc, &SaveOptions::default()).unwrap(), before); + } + } +} +/// `/P0` names an indirect object whose value is a reference to the cell; the cell is also named +/// by a stray reference. The backstop checks the number it pruned, not what it points at. +#[test] +fn a_pruned_pattern_named_through_a_reference_object_does_not_dodge_the_backstop() { + let doc = open(vec![ + catalog(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /Pattern << /P0 7 0 R >> /Shading << /Sh 6 0 R >> >> >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + b"6 0 R".to_vec(), + ]); + confidential_or_refused(doc, 0); +} +/// The backstop's reachability is a full save's: after Apply an incremental save must be that +/// full save, byte for byte, never an appended revision over the old bytes. +#[test] +fn an_incremental_save_after_apply_is_the_full_save() { + let doc = open(vec![ + catalog(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 /Resources {RES} >>").into_bytes(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R >>".to_vec(), + FONT.to_vec(), + stream_obj("", CONTENT), + stream_obj(PATTERN, TEXT_CELL), + ]); + let mut doc = doc; + mark(&mut doc, 0); + apply(&mut doc, None).unwrap(); + for object_streams in [true, false] { + // A redacted rewrite gets a new identifier from fresh entropy; pin it so the two saves + // can be compared byte for byte. + let opts = SaveOptions { object_streams, id_entropy: Some([7; 16]), ..SaveOptions::default() }; + assert_eq!(write_incremental(&doc, &opts).unwrap(), write_full(&doc, &opts).unwrap()); + } + // A later edit after a save-and-reopen starts from the clean file. + let reopened = Document::open(Arc::new(write_incremental(&doc, &SaveOptions::default()).unwrap())).unwrap(); + assert!(!reopened.bytes().windows(SECRET.len()).any(|w| w == SECRET)); +} diff --git a/crates/redact/tests/pattern_redaction.rs b/crates/redact/tests/pattern_redaction.rs new file mode 100644 index 0000000000..4fd898c9ff --- /dev/null +++ b/crates/redact/tests/pattern_redaction.rs @@ -0,0 +1,758 @@ +//! Saved-output confidentiality regressions for #3647, using only synthetic PDFs. + +use std::sync::Arc; + +use pdfcraft_annot::{Meta, NewAnnotation, Shape, Style, add_annotation, rect_quad}; +use pdfcraft_cos::{Dict, Document, ObjRef, Object, SaveOptions, Stream, write_full}; +use pdfcraft_redact::{PatternProblem, RedactError, apply, marks}; +use pdfcraft_testkit::{Pdf, stream_obj}; + +const SECRET: &[u8] = b"CELLSECRET"; +const CELL: &[u8] = b"BT /F1 20 Tf 40 120 Td (CELLSECRET) Tj ET\n"; +const PATTERN: &str = + "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 500 300] /XStep 500 /YStep 300 /Resources << /Font << /F1 4 0 R >> >>"; +const AREA: [f64; 4] = [30.0, 105.0, 215.0, 150.0]; + +fn fixture(cell: &[u8], compressed: bool, partial: bool, inherited: bool) -> Document { + let resources = "/Resources << /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> >>"; + let parent_res = if inherited { resources } else { "" }; + let own_res = if inherited { "" } else { resources }; + let width = if partial { 400 } else { 170 }; + let content = format!("BT /F1 20 Tf 40 250 Td (PUBLIC) Tj ET\n/Pattern cs /P0 scn 35 110 {width} 35 re f\n"); + let bytes = Pdf::new() + .objs(vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + format!("<< /Type /Pages /Kids [3 0 R] /Count 1 {parent_res} >>").into_bytes(), + format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R {own_res} >>").into_bytes(), + b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(), + stream_obj("", content.as_bytes()), + stream_obj(PATTERN, cell), + ]) + .build(); + let mut doc = Document::open(Arc::new(bytes)).unwrap(); + if compressed { + let pattern = doc.get(ObjRef::new(6, 0)); + let Object::Stream(pattern) = &*pattern else { panic!("fixture pattern must be a stream") }; + doc.set(ObjRef::new(6, 0), Object::Stream(Stream::flate(pattern.dict.clone(), cell))); + } + doc +} + +fn mark(doc: &mut Document, page: usize) { + let shape = Shape::Redact { quads: vec![rect_quad(AREA)], overlay: String::new(), look: Default::default() }; + let style = Style::default_for(&shape); + add_annotation(doc, &NewAnnotation { page, shape, style, contents: String::new(), author: "Synthetic test".into() }, &Meta::default()).unwrap(); +} + +fn saved(doc: &Document) -> Vec { + write_full(doc, &SaveOptions::default()).unwrap() +} + +fn has_decoded_secret(bytes: Vec) -> bool { + let doc = Document::open(Arc::new(bytes)).unwrap(); + doc.object_numbers().into_iter().any(|num| { + let object = doc.try_get(num).unwrap(); + matches!(&*object, Object::Stream(s) if s.decoded().unwrap().windows(SECRET.len()).any(|w| w == SECRET)) + }) +} + +/// Success must remove the payload from the saved object graph. Refusal must be atomic even +/// for the direct library caller, not just the engine's working clone. +fn assert_confidential_or_refused(mut doc: Document, expect_refusal: bool) { + mark(&mut doc, 0); + let before = saved(&doc); + assert!(has_decoded_secret(before.clone()), "the fixture must carry the secret"); + match apply(&mut doc, None) { + Ok(report) => { + assert!(!expect_refusal, "a retained text-bearing cell must refuse"); + assert_eq!(report.patterns_removed, 1); + assert!(!has_decoded_secret(saved(&doc)), "successful redaction retained the pattern cell's secret in the saved PDF"); + } + Err(error) => { + assert!(expect_refusal, "a fully removed pattern should apply: {error}"); + assert!(matches!(error, RedactError::Pattern { reason: PatternProblem::Content, .. }), "{error}"); + assert_eq!(saved(&doc), before, "a refused pattern must leave the entire document unchanged"); + assert_eq!(marks(&doc).len(), 1, "the pending mark must survive refusal"); + } + } +} + +#[test] +fn pattern_whole_fill_saved_output_is_confidential_or_refused() { + for compressed in [false, true] { + assert_confidential_or_refused(fixture(CELL, compressed, false, false), false); + } +} + +#[test] +fn pattern_partial_fill_saved_output_is_confidential_or_refused() { + // The mark hits part of the patterned path, which is removed whole (F-008), so nothing paints + // with the pattern any more: it is pruned with its cell, and the redaction is confidential + // rather than refused. Once a hit path was clipped instead, its paint kept the cell in use. + let cell = [CELL, b"BT /F1 20 Tf 300 120 Td (SAFE) Tj ET\n"].concat(); + for compressed in [false, true] { + assert_confidential_or_refused(fixture(&cell, compressed, true, false), false); + } +} + +#[test] +fn pattern_inherited_resources_saved_output_is_confidential_or_refused() { + assert_confidential_or_refused(fixture(CELL, true, false, true), false); +} + +#[test] +fn pattern_vector_only_control_still_applies_and_preserves_the_cell() { + let cell = b"0.2 0.4 0.6 rg 40 120 15 15 re f\n"; + let mut doc = fixture(cell, true, true, false); + // The guard's control: a plain vector cell some surviving paint still uses is inspected and + // kept, not refused. The hit path is removed whole (F-008), so keep another paint in use. + keep_pattern_in_use(&mut doc, 0); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(report.marks, 1); + assert_eq!((report.paths_removed, report.paths_clipped, report.patterns_removed), (1, 0, 0)); + assert!(marks(&doc).is_empty()); + let reopened = Document::open(Arc::new(saved(&doc))).unwrap(); + let page = &pdfcraft_model::pages(&reopened)[0]; + let res = reopened.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + let patterns = reopened.dict(res.get(b"Pattern").unwrap()).unwrap(); + let pattern = reopened.resolve(patterns.get(b"P0").unwrap()); + let Object::Stream(pattern) = &*pattern else { panic!("pattern must be preserved") }; + assert_eq!(pattern.decoded().unwrap(), cell); +} + +#[test] +fn unrelated_font_widths_do_not_exhaust_pattern_budget() { + let mut doc = fixture(b"40 120 15 15 re f\n", false, true, false); + doc.update_dict(ObjRef::new(4, 0), |font| { + font.set(b"W".to_vec(), Object::Array(vec![Object::Int(500); 17_000])); + }) + .unwrap(); + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1); +} + +#[test] +fn shared_annotation_appearance_on_unselected_page_is_refused() { + let mut doc = two_pages(); + let pages = pdfcraft_model::pages(&doc); + for page in &pages { + let mut resources = doc.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + resources.remove(b"Pattern"); + doc.update_dict(page.obj, |dict| dict.set(b"Resources".to_vec(), Object::Dict(resources.clone()))).unwrap(); + } + let ap = doc.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Subtype".to_vec(), Object::name("Form")), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + ( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([( + b"Pattern".to_vec(), + Object::Dict(Dict::from_iter([(b"P0".to_vec(), Object::Ref(ObjRef::new(6, 0)))])), + )])), + ), + ]), + b"/Pattern cs /P0 scn 35 110 170 35 re f".to_vec(), + ))); + for page in pages { + let annotation = doc.add(Object::Dict(Dict::from_iter([ + (b"Type".to_vec(), Object::name("Annot")), + (b"Subtype".to_vec(), Object::name("Stamp")), + (b"Rect".to_vec(), Object::Array(vec![30.into(), 105.into(), 215.into(), 150.into()])), + (b"AP".to_vec(), Object::Dict(Dict::from_iter([(b"N".to_vec(), Object::Ref(ap))]))), + ]))); + doc.update_dict(page.obj, |dict| dict.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(annotation)]))).unwrap(); + } + mark(&mut doc, 0); + let before = saved(&doc); + let error = apply(&mut doc, Some(&[0])).unwrap_err(); + assert_eq!(error, RedactError::Pattern { page: 2, reason: PatternProblem::Content }); + assert_eq!(saved(&doc), before); + assert_eq!(marks(&doc).len(), 1); +} + +#[test] +fn pattern_refusal_on_second_page_does_not_change_the_first_page() { + let mut doc = two_pages(); + // The second page's pattern must stay in use for it to refuse: its hit path alone would be + // removed whole (F-008) and the pattern pruned. + keep_pattern_in_use(&mut doc, 1); + mark(&mut doc, 0); + mark(&mut doc, 1); + let before = saved(&doc); + let result = apply(&mut doc, None); + assert!(result.is_err(), "text-carrying pattern must refuse the whole apply"); + assert_eq!(saved(&doc), before); + assert_eq!(marks(&doc).len(), 2); +} + +fn two_pages() -> Document { + let mut doc = fixture(CELL, false, true, false); + let source_page = pdfcraft_model::pages(&doc)[0].dict.clone(); + let mut clean = source_page.clone(); + let mut clean_resources = doc.dict(clean.get(b"Resources").unwrap()).unwrap(); + clean_resources.remove(b"Pattern"); + clean.set(b"Resources".to_vec(), Object::Dict(clean_resources)); + let content = doc.add(Object::Stream(Stream::from_raw(Default::default(), b"BT /F1 20 Tf 40 120 Td (DIRECTSECRET) Tj ET".to_vec()))); + clean.set(b"Contents".to_vec(), Object::Ref(content)); + let first = doc.add(Object::Dict(clean)); + doc.update_dict(ObjRef::new(2, 0), |pages| { + pages.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(first), Object::Ref(ObjRef::new(3, 0))])); + pages.set(b"Count".to_vec(), Object::Int(2)); + }) + .unwrap(); + doc +} + +fn pattern_stream(doc: &Document) -> Stream { + let object = doc.get(ObjRef::new(6, 0)); + let Object::Stream(stream) = &*object else { panic!("fixture pattern must be a stream") }; + stream.clone() +} + +/// Paint with the pattern once more, well away from the mark. A path the mark hits is removed +/// whole (F-008), and a pattern nothing surviving paints with is pruned along with its cell, so +/// the guard only ever inspects a pattern that some surviving paint still uses. This keeps one, +/// so the guard has to inspect this cell rather than the pattern simply leaving the file. +fn keep_pattern_in_use(doc: &mut Document, page: usize) { + let page = pdfcraft_model::pages(doc).swap_remove(page); + let extra = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"/Pattern cs /P0 scn 300 200 100 40 re f".to_vec()))); + let mut list = match page.dict.get(b"Contents") { + Some(Object::Array(items)) => items.clone(), + Some(other) => vec![other.clone()], + None => Vec::new(), + }; + list.push(Object::Ref(extra)); + doc.update_dict(page.obj, |d| d.set(b"Contents".to_vec(), Object::Array(list))).unwrap(); +} + +fn assert_refused(mut doc: Document, reason: PatternProblem) { + keep_pattern_in_use(&mut doc, 0); + mark(&mut doc, 0); + let before = saved(&doc); + let error = apply(&mut doc, None).unwrap_err(); + assert_eq!(error, RedactError::Pattern { page: 1, reason }); + assert!(!error.to_string().contains("CELLSECRET"), "errors must not disclose page text"); + assert_eq!(saved(&doc), before, "even direct callers must be unchanged"); + assert_eq!(marks(&doc).len(), 1); +} + +#[test] +fn pattern_refusal_does_not_block_redacting_other_selected_pages() { + let mut doc = two_pages(); + mark(&mut doc, 0); + mark(&mut doc, 1); + let report = apply(&mut doc, Some(&[0])).unwrap(); + assert_eq!(report.marks, 1); + assert_eq!(marks(&doc).len(), 1); + assert_eq!(marks(&doc)[0].page, 1); + assert!(has_decoded_secret(saved(&doc)), "the unselected page must remain intact"); +} + +#[test] +fn pattern_nested_form_and_nested_pattern_text_are_refused() { + for nested_pattern in [false, true] { + let mut doc = fixture(CELL, false, true, false); + let mut inner = pattern_stream(&doc); + if !nested_pattern { + inner.dict.remove(b"PatternType"); + inner.dict.set(b"Type".to_vec(), Object::name("XObject")); + inner.dict.set(b"Subtype".to_vec(), Object::name("Form")); + } + let reference = doc.add(Object::Stream(inner)); + let resources = Dict::from_iter([( + if nested_pattern { b"Pattern".to_vec() } else { b"XObject".to_vec() }, + Object::Dict(Dict::from_iter([(b"Inner".to_vec(), Object::Ref(reference))])), + )]); + let mut outer = pattern_stream(&doc); + outer.dict.set(b"Resources".to_vec(), Object::Dict(resources)); + let content = if nested_pattern { b"/Pattern cs /Inner scn 35 110 400 35 re f".as_slice() } else { b"/Inner Do" }; + doc.set(ObjRef::new(6, 0), Object::Stream(Stream::flate(outer.dict, content))); + assert_refused(doc, PatternProblem::Content); + } +} + +#[test] +fn pattern_inside_form_resources_is_refused() { + let mut doc = fixture(CELL, false, false, false); + let page = pdfcraft_model::pages(&doc)[0].clone(); + let resources = doc.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + let form = doc.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Type".to_vec(), Object::name("XObject")), + (b"Subtype".to_vec(), Object::name("Form")), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + (b"Resources".to_vec(), Object::Dict(resources)), + ]), + b"/Pattern cs /P0 scn 35 110 170 35 re f".to_vec(), + ))); + let resources = Dict::from_iter([(b"XObject".to_vec(), Object::Dict(Dict::from_iter([(b"Fm".to_vec(), Object::Ref(form))])))]); + doc.update_dict(page.obj, |dict| dict.set(b"Resources".to_vec(), Object::Dict(resources))).unwrap(); + doc.set(ObjRef::new(5, 0), Object::Stream(Stream::from_raw(Dict::new(), b"/Fm Do".to_vec()))); + assert_refused(doc, PatternProblem::Content); +} + +#[test] +fn pattern_images_and_uncolored_text_are_refused() { + let inline = b"q 10 0 0 10 40 120 cm BI /W 1 /H 1 /CS /G /BPC 8 ID \x00 EI Q"; + assert_refused(fixture(inline, true, true, false), PatternProblem::Content); + let mut uncolored = fixture(CELL, false, true, false); + uncolored.update_dict(ObjRef::new(6, 0), |dict| dict.set(b"PaintType".to_vec(), Object::Int(2))).unwrap(); + assert_refused(uncolored, PatternProblem::Content); +} + +#[test] +fn pattern_indirect_filter_and_parameters_are_decoded_not_skipped() { + let mut doc = fixture(CELL, true, true, false); + let filter = doc.add(Object::name("FlateDecode")); + let predictor = doc.add(Object::Int(1)); + let parms = doc.add(Object::Dict(Dict::from_iter([(b"Predictor".to_vec(), Object::Ref(predictor))]))); + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set(b"Filter".to_vec(), Object::Ref(filter)); + dict.set(b"DecodeParms".to_vec(), Object::Ref(parms)); + }) + .unwrap(); + assert_refused(doc, PatternProblem::Content); +} + +#[test] +fn pattern_damaged_filters_and_unparsed_content_fail_closed() { + for bad in [b"BT (unterminated".as_slice(), b"] BT /F1 20 Tf (CELLSECRET) Tj ET", b"BX UnknownPaint EX"] { + assert_refused(fixture(bad, false, true, false), PatternProblem::Unreadable); + } + let mut damaged = fixture(CELL, true, true, false); + let stream = pattern_stream(&damaged); + let cut = stream.raw.get(..stream.raw.len() / 2).unwrap().to_vec(); + damaged.set(ObjRef::new(6, 0), Object::Stream(Stream::from_raw(stream.dict, cut))); + assert_refused(damaged, PatternProblem::Unreadable); + let mut parameters = fixture(CELL, true, true, false); + parameters + .update_dict(ObjRef::new(6, 0), |dict| { + dict.set(b"DecodeParms".to_vec(), Object::Dict(Dict::from_iter([(b"Predictor".to_vec(), Object::Int(99))]))); + }) + .unwrap(); + assert_refused(parameters, PatternProblem::Unreadable); +} + +#[test] +fn pattern_oversized_decoding_and_resource_cycles_fail_closed() { + let large = vec![b' '; (1 << 20) + 1]; + assert_refused(fixture(&large, true, true, false), PatternProblem::Limit); + let mut cycle = fixture(b"0 0 20 20 re f", false, true, false); + cycle + .update_dict(ObjRef::new(6, 0), |dict| { + dict.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([( + b"Pattern".to_vec(), + Object::Dict(Dict::from_iter([(b"Self".to_vec(), Object::Ref(ObjRef::new(6, 0)))])), + )])), + ); + }) + .unwrap(); + assert_refused(cycle, PatternProblem::Unreadable); +} + +#[test] +fn pattern_plain_nested_form_control_still_applies() { + let mut doc = fixture(b"/Plain Do", false, true, false); + let form = doc.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Type".to_vec(), Object::name("XObject")), + (b"Subtype".to_vec(), Object::name("Form")), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + ]), + b"40 120 15 15 re f".to_vec(), + ))); + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([(b"XObject".to_vec(), Object::Dict(Dict::from_iter([(b"Plain".to_vec(), Object::Ref(form))])))])), + ); + }) + .unwrap(); + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1); + assert!(marks(&doc).is_empty()); +} + +#[test] +fn pattern_unused_and_annotation_appearance_resources_are_checked() { + let mut unused = fixture(CELL, false, false, false); + unused.set(ObjRef::new(5, 0), Object::Stream(Stream::from_raw(Dict::new(), b"0 0 20 20 re f".to_vec()))); + assert_refused(unused, PatternProblem::Content); + + let mut appearance = fixture(CELL, false, false, false); + let page = pdfcraft_model::pages(&appearance)[0].clone(); + let mut resources = appearance.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + let patterns = resources.remove(b"Pattern").unwrap(); + appearance.update_dict(page.obj, |dict| dict.set(b"Resources".to_vec(), Object::Dict(resources))).unwrap(); + appearance.set(ObjRef::new(5, 0), Object::Stream(Stream::from_raw(Dict::new(), b"0 0 20 20 re f".to_vec()))); + let ap = appearance.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Subtype".to_vec(), Object::name("Form")), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + (b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"Pattern".to_vec(), patterns)]))), + ]), + b"/Pattern cs /P0 scn 35 110 170 35 re f".to_vec(), + ))); + let annotation = appearance.add(Object::Dict(Dict::from_iter([ + (b"Type".to_vec(), Object::name("Annot")), + (b"Subtype".to_vec(), Object::name("Stamp")), + (b"Rect".to_vec(), Object::Array(vec![300.into(), 200.into(), 400.into(), 250.into()])), + (b"AP".to_vec(), Object::Dict(Dict::from_iter([(b"N".to_vec(), Object::Ref(ap))]))), + ]))); + appearance.update_dict(page.obj, |dict| dict.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(annotation)]))).unwrap(); + let mut implicit = appearance.clone(); + implicit + .update_dict(ap, |dict| { + dict.remove(b"Subtype"); + }) + .unwrap(); + assert_refused(implicit, PatternProblem::Content); + assert_refused(appearance, PatternProblem::Content); +} + +#[test] +fn pattern_image_xobjects_and_missing_resources_are_refused() { + let mut doc = fixture(b"/Picture Do", false, true, false); + let image = doc.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Type".to_vec(), Object::name("XObject")), + (b"Subtype".to_vec(), Object::name("Image")), + (b"Width".to_vec(), Object::Int(1)), + (b"Height".to_vec(), Object::Int(1)), + (b"ColorSpace".to_vec(), Object::name("DeviceGray")), + (b"BitsPerComponent".to_vec(), Object::Int(8)), + ]), + vec![0], + ))); + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([(b"XObject".to_vec(), Object::Dict(Dict::from_iter([(b"Picture".to_vec(), Object::Ref(image))])))])), + ); + }) + .unwrap(); + assert_refused(doc, PatternProblem::Content); + let mut missing = fixture(b"0 0 20 20 re f", false, true, false); + missing + .update_dict(ObjRef::new(6, 0), |dict| { + dict.remove(b"Resources"); + }) + .unwrap(); + assert_refused(missing, PatternProblem::Unreadable); +} + +#[test] +fn pattern_indirect_vector_filter_control_still_applies() { + let mut doc = fixture(b"40 120 15 15 re f", true, true, false); + let filter = doc.add(Object::name("FlateDecode")); + let parms = doc.add(Object::Dict(Dict::from_iter([(b"Predictor".to_vec(), Object::Int(1))]))); + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set(b"Filter".to_vec(), Object::Array(vec![Object::Ref(filter)])); + dict.set(b"DecodeParms".to_vec(), Object::Array(vec![Object::Ref(parms)])); + }) + .unwrap(); + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1); +} + +#[test] +fn pattern_depth_and_total_decoded_budgets_fail_closed() { + let mut deep = fixture(b"0 0 20 20 re f", false, true, false); + // Two frames a level (the array, then the reference): 300 levels run past the 512-frame + // depth budget long before the object budget, so it is the depth budget that refuses, and it + // does so without overflowing the stack. + let mut nested = Object::Int(1); + for _ in 0..300 { + nested = Object::Array(vec![Object::Ref(deep.add(nested))]); + } + deep.update_dict(ObjRef::new(6, 0), |dict| dict.set(b"Extra".to_vec(), nested)).unwrap(); + assert_refused(deep, PatternProblem::Limit); + + let mut total = fixture(b"0 0 20 20 re f", false, true, false); + let blank = vec![b' '; 1 << 20]; + let template = pattern_stream(&total).dict; + let mut patterns = Dict::new(); + for index in 0..9 { + let cell = total.add(Object::Stream(Stream::flate(template.clone(), &blank))); + patterns.set(format!("P{index}").into_bytes(), Object::Ref(cell)); + } + total + .update_dict(ObjRef::new(3, 0), |dict| { + dict.set(b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"Pattern".to_vec(), Object::Dict(patterns))]))); + }) + .unwrap(); + assert_refused(total, PatternProblem::Limit); +} + +#[test] +fn pattern_replacement_text_without_text_showing_is_refused() { + assert_refused(fixture(b"/Span << /ActualText (CELLSECRET) >> BDC 40 120 20 20 re f EMC", false, true, false), PatternProblem::Content); + let mut named = fixture(b"/Span /Description BDC 40 120 20 20 re f EMC", true, true, false); + named + .update_dict(ObjRef::new(6, 0), |dict| { + dict.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([( + b"Properties".to_vec(), + Object::Dict(Dict::from_iter([( + b"Description".to_vec(), + Object::Dict(Dict::from_iter([(b"Alt".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(b"CELLSECRET".to_vec())))])), + )])), + )])), + ); + }) + .unwrap(); + assert_refused(named, PatternProblem::Content); +} + +#[test] +fn pattern_resource_names_are_not_dictionary_categories_or_descriptions() { + for name in ["Pattern", "Alt", "E", "Resources"] { + for has_pattern in [false, true] { + let mut doc = fixture(b"40 120 15 15 re f", false, true, false); + let font = Object::Dict(Dict::from_iter([(name.as_bytes().to_vec(), Object::Ref(ObjRef::new(4, 0)))])); + if has_pattern { + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set(b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"Font".to_vec(), font)]))); + }) + .unwrap(); + } else { + doc.update_dict(ObjRef::new(3, 0), |dict| { + dict.set(b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"Font".to_vec(), font)]))); + }) + .unwrap(); + doc.set(ObjRef::new(5, 0), Object::Stream(Stream::from_raw(Dict::new(), b"35 110 400 35 re f".to_vec()))); + } + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1, "resource named {name}, pattern: {has_pattern}"); + } + } +} + +#[test] +fn pattern_opaque_and_external_xobjects_are_refused() { + for kind in ["PS", "Bogus", "missing", "Form-PS", "Form-Ref"] { + let mut doc = fixture(b"/X Do", false, true, false); + let mut dict = Dict::from_iter([(b"Type".to_vec(), Object::name("XObject"))]); + match kind { + "missing" => {} + "Form-PS" => { + dict.set(b"Subtype".to_vec(), Object::name("Form")); + dict.set(b"Subtype2".to_vec(), Object::name("PS")); + } + "Form-Ref" => { + dict.set(b"Subtype".to_vec(), Object::name("Form")); + dict.set( + b"Ref".to_vec(), + Object::Dict(Dict::from_iter([(b"F".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(b"external.pdf".to_vec())))])), + ); + } + subtype => { + dict.set(b"Subtype".to_vec(), Object::name(subtype)); + } + } + // Reference XObjects can have an empty fallback: that cannot establish safety of + // the external representation. No external file is read by this test or the guard. + let data = if kind == "Form-Ref" { b"".as_slice() } else { b"/Helvetica findfont 20 scalefont setfont (CELLSECRET) show" }; + let object = doc.add(Object::Stream(Stream::from_raw(dict, data.to_vec()))); + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([(b"XObject".to_vec(), Object::Dict(Dict::from_iter([(b"X".to_vec(), Object::Ref(object))])))])), + ); + }) + .unwrap(); + assert_refused(doc, PatternProblem::Unreadable); + } +} + +#[test] +fn pattern_named_form_resources_still_apply() { + for name in ["Pattern", "Alt", "E", "Resources"] { + for inside in [false, true] { + let content = format!("/{name} Do"); + let mut doc = fixture(content.as_bytes(), false, true, false); + let form = doc.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Type".to_vec(), Object::name("XObject")), + (b"Subtype".to_vec(), Object::name("Form")), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + ]), + b"35 110 400 35 re f".to_vec(), + ))); + let resources = Object::Dict(Dict::from_iter([( + b"XObject".to_vec(), + Object::Dict(Dict::from_iter([(name.as_bytes().to_vec(), Object::Ref(form))])), + )])); + doc.update_dict(ObjRef::new(if inside { 6 } else { 3 }, 0), |dict| dict.set(b"Resources".to_vec(), resources)).unwrap(); + if !inside { + doc.set(ObjRef::new(5, 0), Object::Stream(Stream::from_raw(Dict::new(), content.into_bytes()))); + } + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1, "form named {name}, inside: {inside}"); + } + } +} + +#[test] +fn pattern_soft_mask_groups_are_checked_by_resource_context() { + for subtype in [None, Some("Bogus"), Some("Form")] { + for text in [false, true] { + let mut doc = fixture(b"q /GS gs 40 120 20 20 re f Q", false, true, false); + let mut dict = Dict::from_iter([ + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + (b"Group".to_vec(), Object::Dict(Dict::from_iter([(b"S".to_vec(), Object::name("Transparency"))]))), + (b"Resources".to_vec(), Object::Dict(Dict::new())), + ]); + if let Some(subtype) = subtype { + dict.set(b"Subtype".to_vec(), Object::name(subtype)); + } + let group = doc.add(Object::Stream(Stream::from_raw(dict, if text { CELL.to_vec() } else { b"40 120 20 20 re f".to_vec() }))); + let state = Object::Dict(Dict::from_iter([( + b"SMask".to_vec(), + Object::Dict(Dict::from_iter([(b"S".to_vec(), Object::name("Luminosity")), (b"G".to_vec(), Object::Ref(group))])), + )])); + doc.update_dict(ObjRef::new(6, 0), |dict| { + dict.set( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([(b"ExtGState".to_vec(), Object::Dict(Dict::from_iter([(b"GS".to_vec(), state)])))])), + ); + }) + .unwrap(); + match (subtype, text) { + (Some("Form"), false) => { + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1); + } + (Some("Form"), true) => assert_refused(doc, PatternProblem::Content), + _ => assert_refused(doc, PatternProblem::Unreadable), + } + } + } +} + +#[test] +fn pattern_null_resource_categories_and_entries_are_absent() { + for null_entries in [false, true] { + let mut doc = fixture(b"40 120 15 15 re f", false, true, false); + let mut resources = Dict::new(); + for category in ["Pattern", "XObject", "Properties", "ExtGState"] { + let value = if null_entries { Object::Dict(Dict::from_iter([(b"Unused".to_vec(), Object::Null)])) } else { Object::Null }; + resources.set(category.as_bytes().to_vec(), value); + } + doc.update_dict(ObjRef::new(6, 0), |dict| dict.set(b"Resources".to_vec(), Object::Dict(resources))).unwrap(); + mark(&mut doc, 0); + assert_eq!(apply(&mut doc, None).unwrap().marks, 1); + } + let mut plain = fixture(b"", false, false, false); + plain + .update_dict(ObjRef::new(3, 0), |dict| { + dict.set(b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"Pattern".to_vec(), Object::Null)]))); + }) + .unwrap(); + plain.set(ObjRef::new(5, 0), Object::Stream(Stream::from_raw(Dict::new(), b"40 120 20 20 re f".to_vec()))); + mark(&mut plain, 0); + assert_eq!(apply(&mut plain, None).unwrap().marks, 1); +} + +#[test] +fn a_pattern_whose_only_paint_was_redacted_leaves_the_file_with_its_cell() { + // The mark hits the patterned path, which is removed whole (F-008). Nothing paints with the + // pattern any more, so it leaves the page's resources and its cell -- text included -- leaves + // the saved file: the redaction applies rather than refusing. + for compressed in [false, true] { + let mut doc = fixture(CELL, compressed, true, false); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(report.patterns_removed, 1, "compressed: {compressed}"); + assert!(!has_decoded_secret(saved(&doc)), "the cell's text is not in the saved file (compressed: {compressed})"); + } +} + +#[test] +fn a_form_without_resources_keeps_the_page_pattern_it_still_paints() { + // The page fills a box under the mark with /P0, then draws a form that has no /Resources of + // its own, so it paints with the page's: its /P0 fill lies outside the mark and survives. + // The page's own fill is removed whole, but the pattern must stay for the form's. + let cell = b"0.2 0.4 0.6 rg 40 120 15 15 re f\n"; + let bytes = Pdf::new() + .objs(vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 500 300] /Contents 5 0 R /Resources << /Font << /F1 4 0 R >> /Pattern << /P0 6 0 R >> /XObject << /Fm0 7 0 R >> >> >>".to_vec(), + b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(), + stream_obj("", b"/Pattern cs /P0 scn 35 110 170 35 re f\n0 g\n/Fm0 Do\n"), + stream_obj(PATTERN, cell), + stream_obj("/Type /XObject /Subtype /Form /BBox [0 0 500 300]", b"/Pattern cs /P0 scn 300 200 100 40 re f\n"), + ]) + .build(); + let mut doc = Document::open(Arc::new(bytes)).unwrap(); + mark(&mut doc, 0); + let report = apply(&mut doc, None).unwrap(); + assert_eq!((report.paths_removed, report.patterns_removed), (1, 0)); + let reopened = Document::open(Arc::new(saved(&doc))).unwrap(); + let page = &pdfcraft_model::pages(&reopened)[0]; + let res = reopened.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + let patterns = reopened.dict(res.get(b"Pattern").unwrap()).unwrap(); + assert!(patterns.get(b"P0").is_some(), "the form still paints with /P0 outside the mark"); +} + +#[test] +fn a_shared_appearance_of_a_reply_or_pop_up_removed_on_another_page_is_refused() { + // The marked comment on page 1 has no appearance; its reply (or pop-up) is listed on page 2, + // goes with it, and shares its appearance with a stamp that stays on page 2. That stamp keeps + // the text-bearing cell reachable, so the all-pages check must run and refuse. + for link in ["IRT", "Parent"] { + let mut doc = two_pages(); + let pages = pdfcraft_model::pages(&doc); + for page in &pages { + let mut resources = doc.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + resources.remove(b"Pattern"); + doc.update_dict(page.obj, |dict| dict.set(b"Resources".to_vec(), Object::Dict(resources.clone()))).unwrap(); + } + let ap = doc.add(Object::Stream(Stream::from_raw( + Dict::from_iter([ + (b"Subtype".to_vec(), Object::name("Form")), + (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 500.into(), 300.into()])), + ( + b"Resources".to_vec(), + Object::Dict(Dict::from_iter([( + b"Pattern".to_vec(), + Object::Dict(Dict::from_iter([(b"P0".to_vec(), Object::Ref(ObjRef::new(6, 0)))])), + )])), + ), + ]), + b"/Pattern cs /P0 scn 35 110 170 35 re f".to_vec(), + ))); + let comment = doc.add(Object::Dict(Dict::from_iter([ + (b"Type".to_vec(), Object::name("Annot")), + (b"Subtype".to_vec(), Object::name("Text")), + (b"Rect".to_vec(), Object::Array(vec![40.into(), 110.into(), 60.into(), 130.into()])), + ]))); + let follower = doc.add(Object::Dict(Dict::from_iter([ + (b"Type".to_vec(), Object::name("Annot")), + (b"Subtype".to_vec(), Object::name(if link == "IRT" { "Text" } else { "Popup" })), + (b"Rect".to_vec(), Object::Array(vec![300.into(), 200.into(), 400.into(), 260.into()])), + (link.as_bytes().to_vec(), Object::Ref(comment)), + (b"AP".to_vec(), Object::Dict(Dict::from_iter([(b"N".to_vec(), Object::Ref(ap))]))), + ]))); + let stamp = doc.add(Object::Dict(Dict::from_iter([ + (b"Type".to_vec(), Object::name("Annot")), + (b"Subtype".to_vec(), Object::name("Stamp")), + (b"Rect".to_vec(), Object::Array(vec![30.into(), 105.into(), 215.into(), 150.into()])), + (b"AP".to_vec(), Object::Dict(Dict::from_iter([(b"N".to_vec(), Object::Ref(ap))]))), + ]))); + doc.update_dict(pages[0].obj, |dict| dict.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(comment)]))).unwrap(); + doc.update_dict(pages[1].obj, |dict| dict.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(follower), Object::Ref(stamp)]))).unwrap(); + mark(&mut doc, 0); + let before = saved(&doc); + let error = apply(&mut doc, Some(&[0])).unwrap_err(); + assert_eq!(error, RedactError::Pattern { page: 2, reason: PatternProblem::Content }, "/{link}"); + assert_eq!(saved(&doc), before, "/{link}"); + assert_eq!(marks(&doc).len(), 1, "/{link}"); + } +} diff --git a/docs/roadmap.md b/docs/roadmap.md index 023688ddc4..736d53bb7a 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -245,7 +245,7 @@ total matches the parity estimate. | M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions.
Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA
Done since: Existing simple field-font encodings and Differences are respected when generating appearances; an unencodable value uses a separate Helvetica appearance font without changing the source font. Synthetic reopen regressions cover punctuation, subset fallback and multiline text. Exact font metrics/layout and broader Unicode fallback remain incomplete. | | M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions.
Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA
Done since: Tab and Shift-Tab wrapping to the same field or another widget of that field refresh the newly opened editor from the accepted value before Save. Synthetic UI regressions cover both directions, same-frame Save, validation refusal and undo/redo. Renaming a field renames it in the calculations that read it (field lists, notation, literal `getField` names), in one undo step, and says how many changed. Prepare a form ▸ Calculation order, a page of its own: each calculated field's formula and value, what is wrong (fields that calculate too early, loops, fields missing from `/CO`) and Fix order, Up/Down, Alt+↑/↓ and drag, and on the page each field's step and arrows to and from the field in focus (`form_calculation_order` headlessly). | | M7 | Content editing (text, images, header/footer, watermark) | 250–500 | 20% | 100–180 | Done: header & footer, watermarks, backgrounds, Bates; added text and images that stay editable (move, resize, format, rotate, flip, crop, replace); links (Link tool, Link Properties, create from URLs, remove all). Missing: editing existing text and images in place (the longest pole), image/PDF watermarks | -| M8 | Security + redaction | 100–180 | 68% | 15–30 | Done: opening protected documents, permissions, Protect Using Password, Remove security; redaction (mark text/areas/pages, Search & Redact with patterns, apply removing glyphs, image pixels, vectors, XObject content, annotations and fields, verification, full rewrite on save); Remove hidden information and Sanitize.
Done since: redaction codes (U.S. FOIA and Privacy Act sets as overlay text, in Redaction properties and `redact_mark`). Redaction cleans indirect and form structure references, preserving surviving form tags. Redaction retires inherited resource dictionaries, so a parent `/Pages` can't keep a removed XObject reachable; unredacted pages are left untouched. Image redaction removes alternate images and unsupported masks only on affected images. Changed marked-content spans lose replacement descriptions; untouched spans keep theirs. Hidden-text cleanup retires unused named descriptions across inherited/shared page and form resources (64 focused tests and scoped Rust 1.96 Clippy/wasm checks pass; publication CI pending). The redaction audit counts a filled cover that hides half of any one word of a long line (#2662), applied or not. Phone-pattern redaction also matches labelled Italian geographic numbers with `+39` and a slash after the area code (#3205 follow-up); unlabelled international slash numbers remain unsupported. Missing: certificate security, custom redaction code sets and pattern locales, DCT re-encoding | +| M8 | Security + redaction | 100–180 | 68% | 15–30 | Done: opening protected documents, permissions, Protect Using Password, Remove security; redaction (mark text/areas/pages, Search & Redact with patterns, apply removing glyphs, image pixels, vectors, XObject content, annotations and fields, verification, full rewrite on save); Remove hidden information and Sanitize.
Done since: redaction codes (U.S. FOIA and Privacy Act sets as overlay text, in Redaction properties and `redact_mark`). Redaction cleans indirect and form structure references, preserving surviving form tags. Redaction retires inherited resource dictionaries, so a parent `/Pages` can't keep a removed XObject reachable; unredacted pages are left untouched. Image redaction removes alternate images and unsupported masks only on affected images. Changed marked-content spans lose replacement descriptions; untouched spans keep theirs. Hidden-text cleanup retires unused named descriptions across inherited/shared page and form resources (64 focused tests and scoped Rust 1.96 Clippy/wasm checks pass; publication CI pending). Retained text/image-bearing or unreadable tiling-pattern cells now make Apply refuse atomically after a bounded check of rewritten resources; fully removed cells still apply, and cross-page retention is checked. The redaction audit counts a filled cover that hides half of any one word of a long line (#2662), applied or not. Phone-pattern redaction also matches labelled Italian geographic numbers with `+39` and a slash after the area code (#3205 follow-up); unlabelled international slash numbers remain unsupported. Missing: certificate security, custom redaction code sets and pattern locales, DCT re-encoding | | M9 | Signatures (PAdES, validation) | 160–280 | 60% | 40–70 | Done: new `sign` crate (DER, X.509, CMS, PKCS #12 on RustCrypto; aws-lc-rs for RSA private keys); PAdES B-B signing (visible, invisible, existing fields, certification with DocMDP); validation with trust store and changes-after-signing classification; self-signed digital IDs; Signatures panel, message bar, sign dialogs; agent tools. Checked with pdfsig and OpenSSL.
Done since: macOS Keychain and Windows Current User Personal store signing (software-backed CNG keys), certificate viewer, signed documents protected from rewrites, page validation overlays separate from original signature artwork (baked Not Verified captions remain; #79 open), snapshot tabs that reuse instead of multiplying. Missing: timestamps (B-T), LTV (DSS, OCSP, CRL), FieldMDP, smart cards, PKCS #11 | | M10 | OCR, create, export, print | 200–350 | 33% | 120–215 | Done: create from blank/text/PNG/JPEG/TIFF (multi-page)/GIF/BMP; export PNG/JPEG/TIFF and text; Print (Acrobat's sizing, n-up, booklet, poster, comments & forms, preview, the CUPS and Windows spoolers, print-ready PDF).
Done since: embedded/72/custom DPI choices for image imports; export all images; OCR (searchable image for pages, ranges and multiple files); single-sided cut-and-stack imposition with cut marks, through Print and doc_print; architectural/ANSI/large ISO sheet presets, Match page at 100% and PDF-scale preview (#599 follow-up); Create PDF from multiple files (PDFs, images and text, edited as pages in the grid); OCR can also read pictures on pages that have text, without adding that text twice (#1028). Model staging fetches SHA-256-verified ONNX detection/recognition files after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, then complete legacy RTEN pairs, without mixing formats or changing directory precedence. Done since: Word invoice export uses borderless inferred tables, floating positioned text/artwork only where text overlaps an image or is rotated, and per-page sizes, covered by synthetic invoice automation regressions (#773); printing on the web through the browser's print dialog, sheets drawn by PdfCraft (gap 4). Lossless image export applies per-component Decode, including Indexed mapping (#1201 partial). Existing 16-bit precision and color-management limits remain. #3532 partial: vector printing retains existing appearances with absent Subtype through a print-owned Form copy; source artwork stays unchanged. #3120: Office export reads form XObjects 50 levels deep, as the renderer does, and up to 50 000 a page; pages past that are listed in `truncated_pages`. Word table cells use the grid columns they span, including cells after a merged cell. Done since: text in a table slot covered by another cell's span remains as paragraph text in Word, HTML, RTF and PowerPoint. Missing: OCR languages beyond Latin, editable-text OCR output, full Office layout fidelity/import. Done since: dotted-leader contents entries stay in separate complete paragraphs (#1757) #2156 candidate: last selected print paper persists across sessions; other print preferences unchanged. Image export applies soft-mask Decode opacity intervals before resizing, including indirect entries and inverted ranges. Vector printing gives annotation appearances unoccupied XObject names so existing page artwork survives. Done since: Enhance scanned file cleans scanned pages in place (background removal, edge shadow removal, despeckle; the OCR text layer and comments stay; #4119 partial). | | M11 | Optimize, preflight, PDF/A/X/UA, print production | 200–350 | 18% | 80–140 | Done: new `optimize` crate: Reduce File Size and the PDF Optimizer (images measured where drawn, bicubic downsampling, JPEG/ZIP recompression only when smaller, discard objects and user data, Flate clean-up, resource merging, object streams). Target-version compatibility is partial: newer content is not detected or converted. Content stream boundaries preserve comments, split operands and graphics state when image fragments merge. Image downsampling includes page-local `/UserUnit` scaling when measuring physical resolution. Done since: reject invalid image and soft-mask dimensions with checked sizing (#2178); black-and-white images stored losslessly as CCITT Group 4 when smaller, averaged down to 300 ppi above 450 (halftones aside), Group 3 stored again as Group 4, and black-and-white RGB and indexed images that use only palette entries 0 and 1 packed to one bit. Missing: fonts and transparency panels, space audit, preflight, PDF/A/X/UA |