M11: PDF Optimizer can discard links to other documents (external cross-references)

PDF Optimizer > Discard User Data gets "Discard external cross-references",
off by default. It takes GoToR, GoToE, Launch and ImportData actions, and
Thread actions with an /F, out of the actions of link annotations and
bookmarks, and the actions chained after one through /Next take its place
in order. Other annotations, such as form fields, keep their actions. A
link left with no action and no /Dest is removed from its page; when the
link is an indirect object, that object is rewritten without the action
for anything else that refers to it. A bookmark keeps its title, place and
children, including a bookmark stored directly inside another one. Links
within the document and web links stay. An indirect action that changes
is written as a new indirect object, except the first action left at the
head of an /A, which is copied into the /A directly when other actions
follow it. Objects that use the old action keep it. An /A with more than
100,000 values to check, or one that opens another document and needs
more than 100,000 values to rewrite, stops the optimization with
OptimizeError::ExternalRefLimit, and so does an outline of more than
100,000 bookmarks.

doc_optimize takes discard_external_refs and reports external_links and
external_bookmarks.
This commit is contained in:
dotCooCoo
2026-10-24 04:10:32 -07:00
parent c894928e6f
commit 3d24d82635
9 changed files with 882 additions and 7 deletions
+3
View File
@@ -1258,6 +1258,7 @@ impl Automation {
("flate_unencoded", &mut settings.flate_unencoded),
("remove_invalid_links", &mut settings.remove_invalid_links),
("remove_unreferenced_dests", &mut settings.remove_unreferenced_dests),
("discard_external_refs", &mut settings.discard_external_refs),
] {
if let Some(b) = a.opt_bool(key)? {
*flag = b;
@@ -1297,6 +1298,8 @@ impl Automation {
"invalid_links": o.invalid_links,
"invalid_bookmarks": o.invalid_bookmarks,
"unreferenced_dests": o.unreferenced_dests,
"external_links": o.external_links,
"external_bookmarks": o.external_bookmarks,
"unused_xobjects": o.unused_xobjects,
"merged_objects": r.merged,
"discarded": r.discarded.iter().map(|(h, n)| json!({ "category": h.id(), "count": n })).collect::<Vec<_>>(),
+2 -1
View File
@@ -1375,7 +1375,7 @@ pub fn tools() -> Vec<ToolDef> {
.with(schema(json!({ "doc": doc() }), &["doc"])),
t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.")
.with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])),
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); remove_invalid_links and remove_unreferenced_dests (true); discard_external_refs takes GoToR, GoToE, Launch and ImportData actions, and Thread actions with /F, out of links and bookmarks, removes a link left with nothing to do, and keeps links within the document and web links (external_links, external_bookmarks); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
.cmd("optimize.advanced")
.with(schema(
json!({
@@ -1392,6 +1392,7 @@ pub fn tools() -> Vec<ToolDef> {
"flate_unencoded": { "type": "boolean" },
"remove_invalid_links": { "type": "boolean" },
"remove_unreferenced_dests": { "type": "boolean" },
"discard_external_refs": { "type": "boolean" },
"discard": { "type": "array", "items": { "type": "string" } },
}),
&["doc", "path"],
+16
View File
@@ -7352,6 +7352,22 @@ fn a_deleted_image_is_not_kept_by_reduce_or_optimize() {
assert_eq!(reopened["pages"], 1);
}
#[test]
fn optimizing_discards_links_to_other_documents_through_tools() {
let dir = workdir("optimize-external-refs");
std::fs::write(dir.join("remote.pdf"), remote_link_pdf("other.pdf", None)).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "remote.pdf" }))["doc"].as_u64().unwrap();
let kept = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "kept.pdf" }));
assert_eq!(kept["external_links"], 0, "{kept}");
let kept = ok(&mut a, "doc_open", json!({ "path": "kept.pdf" }))["doc"].as_u64().unwrap();
assert_eq!(ok(&mut a, "link_list", json!({ "doc": kept }))["count"], 1);
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "small.pdf", "discard_external_refs": true }));
assert_eq!((r["external_links"].as_u64(), r["external_bookmarks"].as_u64()), (Some(1), Some(0)), "{r}");
let small = ok(&mut a, "doc_open", json!({ "path": "small.pdf" }))["doc"].as_u64().unwrap();
assert_eq!(ok(&mut a, "link_list", json!({ "doc": small }))["count"], 0);
}
/// A two-page PDF with the same three-row table ("Name Qty / Apple 12 / Pear 7"); the second
/// page is turned with /Rotate 90.
fn table_pdf() -> Vec<u8> {
+7
View File
@@ -31,6 +31,13 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
- **Clean up:** Flate for streams with no filter; for streams with an outer ASCII85 or ASCIIHex
layer, that layer is removed when the result is smaller (the inner filters keep their encoded
bytes). Streams with `/DecodeParms`, images and object streams are left alone.
- **Discard external cross-references** (off by default): `GoToR`, `GoToE`, `Launch` and
`ImportData` actions, and `Thread` actions with an `/F`, are taken out of link annotations and
bookmarks, and the actions chained after one through `/Next` take its place. Other annotations,
such as form fields, keep their actions. A link left with no action and no `/Dest` is removed
from its page, and its object, when something else refers to it, keeps the link without the
action; a bookmark keeps its title and children. Links within the document and web links
(`URI`) stay. An outline of more than 100,000 bookmarks stops the optimization with an error.
The engine runs Remove Hidden Information (`pdfcraft-redact`) for the user-data categories,
then this, then merges identical objects and writes a full, compressed save
+427
View File
@@ -0,0 +1,427 @@
//! Discard User Data ▸ Discard external cross-references: links and bookmarks that open another
//! document.
//!
//! An action opens another document when its `/S` is `GoToR` (another PDF file), `GoToE` (a
//! document embedded in this one, or the one this is embedded in), `Launch` (a file or an
//! application), `ImportData` (form data read from another file) or `Thread` with an `/F` (an
//! article in another file). Such an action is taken out of the `/A` of a link annotation
//! (`/Subtype /Link`) or a bookmark, and the actions chained after it through `/Next` take its
//! place in order. Other annotations, such as form fields, keep their actions. A link left with no
//! action and no `/Dest` is removed from its page; when the link is stored as its own object, that
//! object keeps the link without its action for anything else that refers to it (the structure
//! tree, for example). A bookmark stays, with its title and children. Links that jump within the
//! document (`/Dest`, `GoTo`) and web links (`URI`) are kept. An `/A` that takes more than
//! [`MAX_STEPS`] values to check, or whose actions open another document and take more than that
//! to rewrite, stops the optimization with an error, and so does an outline of more than
//! [`MAX_OUTLINE`] items or with items stored inside each other more than [`MAX_DEPTH`] levels
//! deep.
use std::collections::{HashMap, HashSet};
use pdfcraft_cos::{Dict, Document, ObjRef, Object};
use crate::OptimizeError;
/// Action types that open or read another document (`Thread` does when it has an `/F`).
const EXTERNAL: [&[u8]; 4] = [b"GoToR", b"GoToE", b"Launch", b"ImportData"];
/// How many levels (actions, `/Next` arrays and references) a chain is rewritten. Past it, the rest
/// of the chain is kept when it opens no other document, and removed when it does.
const MAX_DEPTH: usize = 512;
/// How many values each pass over one `/A` looks at: the check for actions to other documents,
/// then the rewrite with its check past [`MAX_DEPTH`]. A chain that needs more in either pass
/// stops the optimization with [`OptimizeError::ExternalRefLimit`].
const MAX_STEPS: usize = 100_000;
/// How many bookmarks the outline walk looks at before it stops the optimization with
/// [`OptimizeError::ExternalRefLimit`].
const MAX_OUTLINE: usize = 100_000;
/// What is left of an action or a list of actions.
enum Kept {
/// Nothing changed: keep the object as it is stored.
Same,
/// These actions, in order, replace it.
Now(Vec<Object>),
}
#[derive(Default)]
struct Walk {
/// References on the current chain, to end a chain that comes back to itself.
path: Vec<ObjRef>,
/// Indirect objects from which an action to another document can be reached.
dirty: HashSet<ObjRef>,
/// Actions to another document taken out.
removed: usize,
/// A chain that opens another document past [`MAX_DEPTH`] was cut there.
cut: bool,
/// Values looked at so far, against [`MAX_STEPS`].
steps: usize,
/// The walk ran out of steps; its result is not used.
exhausted: bool,
}
fn is_external(doc: &Document, d: &Dict) -> bool {
let Some(s) = d.get(b"S").map(|s| doc.resolve(s)) else { return false };
match s.as_name() {
Some(b"Thread") => d.get(b"F").is_some_and(|f| !matches!(doc.resolve(f).as_ref(), Object::Null)),
Some(n) => EXTERNAL.contains(&n),
None => false,
}
}
/// The actions an unchanged `/Next` value stands for, in order.
fn listed(doc: &Document, next: &Object) -> Vec<Object> {
match doc.resolve(next).as_ref() {
Object::Array(a) => a.clone(),
Object::Dict(_) => vec![next.clone()],
_ => Vec::new(),
}
}
impl Walk {
/// Count one value; false once the walk has run out of steps.
fn step(&mut self) -> bool {
self.steps += 1;
if self.steps > MAX_STEPS {
self.exhausted = true;
}
!self.exhausted
}
/// Whether the actions stored directly in `o` include one to another document; the references
/// they hold are added to `refs`.
fn scan(&mut self, doc: &Document, o: &Object, refs: &mut Vec<ObjRef>) -> bool {
let mut stack = vec![o];
let mut external = false;
while let Some(o) = stack.pop() {
if !self.step() {
return false;
}
match o {
Object::Ref(r) => refs.push(*r),
Object::Array(items) => stack.extend(items),
Object::Dict(d) => {
external |= is_external(doc, d);
if let Some(n) = d.get(b"Next") {
stack.push(n);
}
}
_ => {}
}
}
external
}
/// Mark the indirect objects reachable from `a` that lead to an action to another document.
/// Each indirect object is looked at once.
fn reach(&mut self, doc: &Document, a: &Object) {
let mut todo = Vec::new();
self.scan(doc, a, &mut todo);
let mut seen = HashSet::new();
let mut parents: HashMap<ObjRef, Vec<ObjRef>> = HashMap::new();
let mut found = Vec::new();
while let Some(r) = todo.pop() {
if self.exhausted {
return;
}
if !seen.insert(r) {
continue;
}
let target = doc.get(r);
let mut refs = Vec::new();
if self.scan(doc, &target, &mut refs) {
found.push(r);
}
for c in refs {
parents.entry(c).or_default().push(r);
todo.push(c);
}
}
while let Some(r) = found.pop() {
if self.dirty.insert(r)
&& let Some(p) = parents.get(&r)
{
found.extend(p);
}
}
}
/// Whether `o` or an action chained after it opens another document. Needs [`Walk::reach`]
/// to have run on an `/A` that holds `o`.
fn holds_external(&mut self, doc: &Document, o: &Object) -> bool {
let mut refs = Vec::new();
let external = self.scan(doc, o, &mut refs);
external || refs.iter().any(|r| self.dirty.contains(r))
}
}
impl Walk {
/// Indirect actions that change are written as new indirect objects (the old ones stay for
/// whatever else uses them), so a rewritten chain nests no deeper than the original.
fn strip(&mut self, doc: &mut Document, o: &Object, depth: usize) -> Kept {
if !self.step() {
return Kept::Same;
}
if depth > MAX_DEPTH {
if !self.holds_external(doc, o) {
return Kept::Same;
}
self.cut = true;
return Kept::Now(Vec::new());
}
match o {
Object::Ref(r) => {
if !self.dirty.contains(r) {
return Kept::Same;
}
if self.path.contains(r) {
return Kept::Now(Vec::new());
}
self.path.push(*r);
let target = doc.get(*r);
let kept = self.strip(doc, &target, depth + 1);
self.path.pop();
match kept {
Kept::Now(mut v) if v.len() == 1 && matches!(v.first(), Some(Object::Dict(_))) => {
let d = v.remove(0);
Kept::Now(vec![Object::Ref(doc.add(d))])
}
other => other,
}
}
Object::Array(items) => {
let mut out = Vec::new();
let mut changed = false;
for item in items {
match self.strip(doc, item, depth + 1) {
Kept::Same => out.push(item.clone()),
Kept::Now(v) => {
changed = true;
out.extend(v);
}
}
}
if changed { Kept::Now(out) } else { Kept::Same }
}
Object::Dict(d) => {
let next = d.get(b"Next");
let rest = next.map(|n| self.strip(doc, n, depth + 1));
if is_external(doc, d) {
self.removed += 1;
return Kept::Now(match (rest, next) {
(Some(Kept::Now(v)), _) => v,
(Some(Kept::Same), Some(n)) => listed(doc, n),
_ => Vec::new(),
});
}
match rest {
Some(Kept::Now(v)) => {
let mut d = d.clone();
set_next(&mut d, v);
Kept::Now(vec![Object::Dict(d)])
}
_ => Kept::Same,
}
}
_ => Kept::Same,
}
}
}
fn set_next(d: &mut Dict, mut v: Vec<Object>) {
match v.len() {
0 => {
d.remove(b"Next");
}
1 => d.set(b"Next".to_vec(), v.remove(0)),
_ => d.set(b"Next".to_vec(), Object::Array(v)),
}
}
/// `/A` of `owner` with its actions to other documents taken out: `None` when nothing changed,
/// `Some(None)` when no action is left, `Some(Some(action))` otherwise (one action, the rest after
/// it).
fn strip_action(doc: &mut Document, a: &Object, owner: ObjRef) -> Result<Option<Option<Object>>, OptimizeError> {
let mut walk = Walk::default();
// A quick check first, which looks at each indirect object once: a chain with no action to
// another document is left alone however its actions are shared, unless it holds more than
// `MAX_STEPS` values.
walk.reach(doc, a);
let external = walk.holds_external(doc, a);
if walk.exhausted {
return Err(OptimizeError::ExternalRefLimit(owner));
}
if !external {
return Ok(None);
}
walk.steps = 0;
let kept = walk.strip(doc, a, 0);
if walk.exhausted {
return Err(OptimizeError::ExternalRefLimit(owner));
}
let Kept::Now(mut v) = kept else { return Ok(None) };
if walk.removed == 0 && !walk.cut {
return Ok(None);
}
if v.is_empty() {
return Ok(Some(None));
}
let first = v.remove(0);
if v.is_empty() {
return Ok(Some(Some(first)));
}
let Some(mut d) = doc.resolve(&first).as_dict().cloned() else { return Ok(Some(None)) };
let mut next = d.get(b"Next").map(|n| listed(doc, n)).unwrap_or_default();
next.extend(v);
set_next(&mut d, next);
Ok(Some(Some(Object::Dict(d))))
}
/// The outline walk: bookmarks stored as their own objects wait on `stack`; those stored inside
/// another bookmark are edited where they are.
#[derive(Default)]
struct Outline {
stack: Vec<ObjRef>,
items: usize,
changed: usize,
}
impl Outline {
/// Take actions to other documents out of the bookmark `d` (or, when `item` is false, the
/// outline dictionary) and of the bookmarks stored inside it. `owner` is the object that holds
/// `d`. Returns whether `d` changed.
fn visit(&mut self, doc: &mut Document, d: &mut Dict, owner: ObjRef, depth: usize, item: bool) -> Result<bool, OptimizeError> {
let mut changed = false;
if item {
self.items += 1;
if self.items > MAX_OUTLINE {
return Err(OptimizeError::ExternalRefLimit(owner));
}
let stripped = match d.get(b"A") {
Some(a) => strip_action(doc, a, owner)?,
None => None,
};
if let Some(action) = stripped {
match action {
Some(a) => d.set(b"A".to_vec(), a),
None => {
d.remove(b"A");
}
}
self.changed += 1;
changed = true;
}
}
let keys: &[&[u8]] = if item { &[b"Next", b"First"] } else { &[b"First"] };
for k in keys {
match d.get_mut(k) {
Some(Object::Ref(r)) => self.stack.push(*r),
Some(Object::Dict(c)) => {
if depth >= MAX_DEPTH {
return Err(OptimizeError::ExternalRefLimit(owner));
}
changed |= self.visit(doc, c, owner, depth + 1, true)?;
}
_ => {}
}
}
Ok(changed)
}
}
/// Take actions to other documents out of bookmarks, depth-first. Returns the bookmarks changed.
fn discard_bookmarks(doc: &mut Document) -> Result<usize, OptimizeError> {
let Some(root) = doc.root() else { return Ok(0) };
let catalog = doc.get(root);
let mut walk = Outline::default();
let mut seen = HashSet::new();
match catalog.as_dict().and_then(|c| c.get(b"Outlines")) {
Some(Object::Ref(o)) => {
seen.insert(*o);
if let Some(mut d) = doc.get(*o).as_dict().cloned()
&& walk.visit(doc, &mut d, *o, 0, false)?
{
doc.set(*o, Object::Dict(d));
}
}
Some(Object::Dict(o)) => {
let mut d = o.clone();
if walk.visit(doc, &mut d, root, 0, false)? {
doc.update_dict(root, |c| c.set(b"Outlines".to_vec(), Object::Dict(d)))?;
}
}
_ => {}
}
while let Some(r) = walk.stack.pop() {
if !seen.insert(r) {
continue;
}
let Some(mut d) = doc.get(r).as_dict().cloned() else { continue };
if walk.visit(doc, &mut d, r, 0, true)? {
doc.set(r, Object::Dict(d));
}
}
Ok(walk.changed)
}
/// Take actions to other documents out of links and bookmarks. Returns (links, bookmarks)
/// changed; a link counts whether it was removed or kept for its other actions.
pub(crate) fn discard(doc: &mut Document, pages: &[ObjRef]) -> Result<(usize, usize), OptimizeError> {
let mut links = 0;
for p in pages {
let Some(pd) = doc.get(*p).as_dict().cloned() else { continue };
let Some(annots) = pd.get(b"Annots").map(|a| doc.resolve(a)).and_then(|a| a.as_array().cloned()) else { continue };
let mut keep = Vec::with_capacity(annots.len());
let mut changed = false;
for x in &annots {
let Some(mut d) = doc.resolve(x).as_dict().cloned() else {
keep.push(x.clone());
continue;
};
let owner = match x {
Object::Ref(r) => *r,
_ => *p,
};
let link = d.get(b"Subtype").map(|s| doc.resolve(s)).is_some_and(|s| s.as_name().is_some_and(|n| n == b"Link"));
let stripped = match d.get(b"A") {
Some(a) if link => strip_action(doc, a, owner)?,
_ => None,
};
let Some(action) = stripped else {
keep.push(x.clone());
continue;
};
links += 1;
match action {
Some(a) => d.set(b"A".to_vec(), a),
None => {
d.remove(b"A");
}
}
let removed = d.get(b"A").is_none() && d.get(b"Dest").is_none();
match x {
Object::Ref(r) => {
doc.set(*r, Object::Dict(d));
if removed {
changed = true;
} else {
keep.push(x.clone());
}
}
_ => {
changed = true;
if !removed {
keep.push(Object::Dict(d));
}
}
}
}
if changed {
doc.update_dict(*p, |d| d.set(b"Annots".to_vec(), Object::Array(keep)))?;
}
}
let bookmarks = discard_bookmarks(doc)?;
Ok((links, bookmarks))
}
+19 -1
View File
@@ -18,11 +18,14 @@
//!
//! Metadata, attachments, comments, scripts, private data, hidden layers, bookmarks and form
//! fields are discarded by `pdfcraft-redact`'s Remove Hidden Information, which the engine
//! runs alongside.
//! runs alongside. Links that open another document, and bookmark actions that do, are discarded
//! here (Discard User Data ▸ Discard external cross-references, see
//! [`Settings::discard_external_refs`]).
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod audit;
mod external_refs;
pub mod fragments;
mod images;
mod links;
@@ -40,6 +43,8 @@ pub enum OptimizeError {
NoPages,
#[error(transparent)]
Cos(#[from] pdfcraft_cos::CosError),
#[error("object {0} holds too many actions or bookmarks to take out its links to other documents")]
ExternalRefLimit(ObjRef),
#[error("the optimization was cancelled")]
Cancelled,
}
@@ -104,6 +109,9 @@ pub struct Settings {
pub remove_invalid_links: bool,
/// Clean Up: remove named destinations nothing refers to.
pub remove_unreferenced_dests: bool,
/// Discard User Data: take `GoToR`, `GoToE` and `Launch` actions out of links and bookmarks;
/// a link left with nothing to do is removed. Links within the document and web links stay.
pub discard_external_refs: bool,
}
impl Default for Settings {
@@ -120,6 +128,7 @@ impl Default for Settings {
flate_unencoded: true,
remove_invalid_links: true,
remove_unreferenced_dests: true,
discard_external_refs: false,
}
}
}
@@ -182,6 +191,10 @@ pub struct Report {
pub invalid_links: usize,
pub invalid_bookmarks: usize,
pub unreferenced_dests: usize,
/// Links removed (or kept for their other actions) and bookmarks whose action was changed or
/// cleared, because they opened another document (Discard external cross-references).
pub external_links: usize,
pub external_bookmarks: usize,
/// Images and forms the pages listed but never drew, dropped (Clean Up).
pub unused_xobjects: usize,
}
@@ -276,6 +289,11 @@ pub fn optimize_with_progress(doc: &mut Document, settings: &Settings, progress:
report.invalid_links = l;
report.invalid_bookmarks = b;
}
if settings.discard_external_refs {
let (l, b) = external_refs::discard(doc, &pages)?;
report.external_links = l;
report.external_bookmarks = b;
}
if settings.remove_unreferenced_dests {
report.unreferenced_dests = links::remove_unreferenced_dests(doc)?;
}
+402
View File
@@ -393,6 +393,408 @@ fn invalid_links_and_unreferenced_destinations_go() {
assert_eq!(optimize(&mut again, &off).unwrap().invalid_links, 0);
}
/// An action dictionary of type `s`, followed by `next` when given.
fn action(s: &str, next: Option<Object>) -> Object {
let mut d = Dict::new();
d.set(b"S".to_vec(), Object::name(s));
match s {
"GoToR" | "Launch" => d.set(b"F".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(b"other.pdf".to_vec()))),
"URI" => d.set(b"URI".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(b"https://example.com".to_vec()))),
_ => {}
}
if let Some(n) = next {
d.set(b"Next".to_vec(), n);
}
Object::Dict(d)
}
fn link_with(a: Option<Object>, dest: Option<Object>) -> Object {
let mut d = Dict::new();
d.set(b"Type".to_vec(), Object::name("Annot"));
d.set(b"Subtype".to_vec(), Object::name("Link"));
d.set(b"Rect".to_vec(), Object::Array(vec![0.into(), 0.into(), 10.into(), 10.into()]));
if let Some(a) = a {
d.set(b"A".to_vec(), a);
}
if let Some(dest) = dest {
d.set(b"Dest".to_vec(), dest);
}
Object::Dict(d)
}
/// The action types an `/A` value runs, in order (depth-first through `/Next`), at most 1,000.
fn chain(doc: &Document, a: &Object) -> Vec<String> {
let mut out = Vec::new();
let mut stack = vec![a.clone()];
while let Some(o) = stack.pop() {
if out.len() >= 1_000 {
break;
}
match doc.resolve(&o).as_ref() {
Object::Array(items) => stack.extend(items.iter().rev().cloned()),
Object::Dict(d) => {
out.push(
d.get(b"S")
.map(|s| doc.resolve(s))
.and_then(|s| s.as_name().map(|n| String::from_utf8_lossy(n).into_owned()))
.unwrap_or_default(),
);
if let Some(n) = d.get(b"Next") {
stack.push(n.clone());
}
}
_ => {}
}
}
out
}
/// A one-page document whose page holds `annots`, with one bookmark per action in `bookmarks`.
fn linked_doc(annots: impl FnOnce(&mut Document, ObjRef) -> Vec<Object>, bookmarks: &[Object]) -> (Document, ObjRef, Vec<ObjRef>) {
let mut doc = Document::new_empty();
let p = page(&mut doc, &[], "BT ET");
let annots = annots(&mut doc, p);
doc.update_dict(p, |d| d.set(b"Annots".to_vec(), Object::Array(annots))).unwrap();
let outlines = doc.add(Object::Null);
let items: Vec<ObjRef> = bookmarks.iter().map(|_| doc.add(Object::Null)).collect();
for (i, (r, a)) in items.iter().zip(bookmarks).enumerate() {
let mut item = Dict::new();
item.set(b"Title".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(format!("Item {i}").into_bytes())));
item.set(b"Parent".to_vec(), Object::Ref(outlines));
item.set(b"A".to_vec(), a.clone());
if let Some(n) = items.get(i + 1) {
item.set(b"Next".to_vec(), Object::Ref(*n));
}
doc.set(*r, Object::Dict(item));
}
let mut o = Dict::new();
if let (Some(first), Some(last)) = (items.first(), items.last()) {
o.set(b"First".to_vec(), Object::Ref(*first));
o.set(b"Last".to_vec(), Object::Ref(*last));
}
doc.set(outlines, Object::Dict(o));
let root = doc.root().unwrap();
doc.update_dict(root, |c| c.set(b"Outlines".to_vec(), Object::Ref(outlines))).unwrap();
(doc, p, items)
}
fn annots_of(doc: &Document, p: ObjRef) -> Vec<Dict> {
let annots = doc.get(p).as_dict().unwrap().get(b"Annots").cloned().unwrap();
doc.resolve(&annots).as_array().unwrap().iter().map(|a| doc.resolve(a).as_dict().unwrap().clone()).collect()
}
#[test]
fn links_and_bookmarks_to_other_documents_are_discarded_and_links_within_stay() {
let (doc, p, items) = linked_doc(
|doc, p| {
let here = Object::Array(vec![Object::Ref(p), Object::name("Fit")]);
let mut goto = Dict::new();
goto.set(b"S".to_vec(), Object::name("GoTo"));
goto.set(b"D".to_vec(), here.clone());
let remote = doc.add(link_with(Some(action("GoToR", None)), None));
vec![
Object::Ref(remote),
link_with(Some(action("Launch", None)), None),
link_with(Some(action("GoToE", None)), None),
link_with(Some(action("URI", None)), None),
link_with(Some(Object::Dict(goto)), None),
link_with(None, Some(here)),
link_with(Some(action("GoToR", Some(action("URI", None)))), None),
]
},
&[action("GoToR", None), action("Named", None)],
);
let on = Settings { discard_external_refs: true, ..Settings::default() };
// Off by default: nothing changes.
let mut off = doc.clone();
let report = optimize(&mut off, &Settings::default()).unwrap();
assert_eq!((report.external_links, report.external_bookmarks), (0, 0));
assert_eq!(annots_of(&off, p).len(), 7);
let mut doc = doc;
let report = optimize(&mut doc, &on).unwrap();
assert_eq!((report.external_links, report.external_bookmarks, report.invalid_links), (4, 1, 0), "{report:?}");
let left: Vec<Vec<String>> = annots_of(&doc, p).iter().map(|d| d.get(b"A").map(|a| chain(&doc, a)).unwrap_or_default()).collect();
assert_eq!(
left,
[vec!["URI"], vec!["GoTo"], vec![], vec!["URI"]],
"the web link, the jump within and the /Dest link stay; the chained web link keeps its URI"
);
assert!(annots_of(&doc, p)[2].get(b"Dest").is_some());
let bookmark = |doc: &Document, r: ObjRef| doc.get(r).as_dict().unwrap().clone();
assert!(bookmark(&doc, items[0]).get(b"A").is_none(), "the bookmark stays, without its action");
assert_eq!(bookmark(&doc, items[0]).get(b"Next"), Some(&Object::Ref(items[1])), "and its place in the outline");
assert_eq!(chain(&doc, bookmark(&doc, items[1]).get(b"A").unwrap()), ["Named"]);
}
#[test]
fn an_action_to_another_document_hands_its_place_to_the_actions_after_it() {
let (doc, p, _) = linked_doc(
|doc, p| {
let here = Object::Array(vec![Object::Ref(p), Object::name("Fit")]);
// A shared Launch, stored on its own with an indirect /S, followed by a web link.
let launch_name = doc.add(Object::name("Launch"));
let Object::Dict(mut launch) = action("Launch", Some(action("URI", None))) else { panic!("action() builds a dictionary") };
launch.set(b"S".to_vec(), Object::Ref(launch_name));
let launch = doc.add(Object::Dict(launch));
let mut goto = Dict::new();
goto.set(b"S".to_vec(), Object::name("GoTo"));
goto.set(b"D".to_vec(), here);
goto.set(
b"Next".to_vec(),
Object::Array(vec![action("GoToR", Some(action("JavaScript", None))), Object::Ref(launch), action("Named", None)]),
);
// A remote go-to whose web link comes back to it.
let (a1, a2) = (doc.add(Object::Null), doc.add(Object::Null));
doc.set(a1, action("GoToR", Some(Object::Ref(a2))));
doc.set(a2, action("URI", Some(Object::Ref(a1))));
// A remote go-to that leads a list: the rest follow the first action left.
let list = action("GoToR", Some(Object::Array(vec![action("SubmitForm", Some(action("Named", None))), action("URI", None)])));
vec![
link_with(Some(Object::Dict(goto)), None),
link_with(Some(Object::Ref(a1)), None),
link_with(Some(list), None),
link_with(Some(Object::Ref(launch)), None),
]
},
&[],
);
let mut doc = doc;
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 4, "{report:?}");
let left: Vec<Vec<String>> = annots_of(&doc, p).iter().map(|d| chain(&doc, d.get(b"A").unwrap())).collect();
assert_eq!(
left,
[vec!["GoTo", "JavaScript", "URI", "Named"], vec!["URI"], vec!["SubmitForm", "Named", "URI"], vec!["URI"]],
"the actions after a removed one run in their order, and a chain that comes back ends there"
);
}
#[test]
fn a_chain_deeper_than_the_limit_loses_its_tail_and_one_without_a_remote_action_stays() {
let deep = |n: usize, last: &str| (0..n).fold(action(last, None), |next, _| action("URI", Some(next)));
let (doc, p, _) = linked_doc(|_, _| vec![link_with(Some(deep(600, "GoToR")), None), link_with(Some(deep(600, "Named")), None)], &[]);
let mut doc = doc;
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 1, "{report:?}");
let left: Vec<Vec<String>> = annots_of(&doc, p).iter().map(|d| chain(&doc, d.get(b"A").unwrap())).collect();
assert!(left[0].len() < 600 && left[0].iter().all(|s| s == "URI"), "cut before the remote go-to: {} actions", left[0].len());
assert_eq!(left[1].len(), 601, "a deep chain with nothing to remove is kept whole");
}
#[test]
fn a_long_chain_of_indirect_actions_stays_indirect_and_reads_back_after_saving() {
// 130 indirect web links, then a remote go-to: the parser reads at most 128 nested levels.
let mut tail = None;
let (doc, p, _) = linked_doc(
|doc, _| {
let mut r = doc.add(action("GoToR", None));
for _ in 0..130 {
r = doc.add(action("URI", Some(Object::Ref(r))));
}
tail = Some(r);
vec![link_with(Some(Object::Ref(r)), None)]
},
&[],
);
let mut doc = doc;
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 1, "{report:?}");
let a = annots_of(&doc, p)[0].get(b"A").cloned().unwrap();
assert!(matches!(a, Object::Ref(r) if Some(r) != tail), "a new indirect action: {a:?}");
let old = Object::Ref(tail.unwrap());
assert_eq!(chain(&doc, &old).last().map(String::as_str), Some("GoToR"), "the old chain is not edited");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
let reopened = Document::open(Arc::new(bytes)).unwrap();
let a = annots_of(&reopened, p)[0].get(b"A").cloned().unwrap();
assert_eq!(chain(&reopened, &a), vec!["URI"; 130], "every web link reads back");
}
#[test]
fn shared_actions_that_multiply_stop_the_optimization_when_one_opens_another_document() {
use pdfcraft_optimize::OptimizeError;
// Forty levels of indirect actions, each listing the next level twice: 2^40 paths.
let shared = |doc: &mut Document, last: &str| {
let mut r = doc.add(action(last, None));
for _ in 0..40 {
r = doc.add(action("URI", Some(Object::Array(vec![Object::Ref(r), Object::Ref(r)]))));
}
r
};
let mut link = None;
let (doc, _, _) = linked_doc(
|doc, _| {
let a = shared(doc, "GoToR");
let r = doc.add(link_with(Some(Object::Ref(a)), None));
link = Some(r);
vec![Object::Ref(r)]
},
&[],
);
let on = Settings { discard_external_refs: true, ..Settings::default() };
let r = optimize(&mut doc.clone(), &on);
assert!(matches!(r, Err(OptimizeError::ExternalRefLimit(o)) if Some(o) == link), "{r:?}");
let (doc, p, _) = linked_doc(
|doc, _| {
let a = shared(doc, "Named");
vec![link_with(Some(Object::Ref(a)), None)]
},
&[],
);
let before = annots_of(&doc, p);
let mut doc = doc;
let report = optimize(&mut doc, &on).unwrap();
assert_eq!(report.external_links, 0, "{report:?}");
assert_eq!(annots_of(&doc, p), before, "nothing to remove, however the actions are shared");
}
#[test]
fn a_shared_chain_without_remote_actions_is_looked_at_once() {
// 600 levels that each list a 2,000-action chain of web links sixty times, then a remote go-to.
let (doc, p, _) = linked_doc(
|doc, _| {
let mut g = doc.add(action("URI", None));
for _ in 0..2_000 {
g = doc.add(action("URI", Some(Object::Ref(g))));
}
let a = (0..600).fold(action("GoToR", None), |next, _| {
let mut list = vec![Object::Ref(g); 60];
list.push(next);
action("URI", Some(Object::Array(list)))
});
vec![link_with(Some(a), None)]
},
&[],
);
let mut doc = doc;
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 1, "{report:?}");
let left = annots_of(&doc, p);
assert!(!chain(&doc, left[0].get(b"A").unwrap()).iter().any(|s| s == "GoToR"));
}
#[test]
fn a_link_named_through_an_indirect_subtype_is_discarded_and_a_form_field_keeps_its_action() {
let (doc, p, _) = linked_doc(
|doc, _| {
let name = doc.add(Object::name("Link"));
let Object::Dict(mut indirect) = link_with(Some(action("GoToR", None)), None) else { panic!("link_with() builds a dictionary") };
indirect.set(b"Subtype".to_vec(), Object::Ref(name));
let Object::Dict(mut widget) = link_with(Some(action("GoToR", None)), None) else { panic!("link_with() builds a dictionary") };
widget.set(b"Subtype".to_vec(), Object::name("Widget"));
vec![Object::Dict(indirect), Object::Dict(widget)]
},
&[],
);
let mut doc = doc;
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 1, "{report:?}");
let left = annots_of(&doc, p);
assert_eq!(left.len(), 1, "the link goes");
assert_eq!(left[0].name(b"Subtype"), Some(&b"Widget"[..]));
assert_eq!(chain(&doc, left[0].get(b"A").unwrap()), ["GoToR"], "the form field keeps its action");
}
#[test]
fn thread_actions_into_another_file_and_data_imports_are_discarded_and_threads_here_stay() {
let with_file = |s: &str| {
let Object::Dict(mut d) = action(s, None) else { panic!("action() builds a dictionary") };
d.set(b"F".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(b"other.pdf".to_vec())));
d.set(b"D".to_vec(), Object::Int(0));
Object::Dict(d)
};
let here = {
let Object::Dict(mut d) = action("Thread", None) else { panic!("action() builds a dictionary") };
d.set(b"D".to_vec(), Object::Int(0));
Object::Dict(d)
};
let (doc, p, _) = linked_doc(
|_, _| vec![link_with(Some(with_file("Thread")), None), link_with(Some(here.clone()), None), link_with(Some(with_file("ImportData")), None)],
&[],
);
let mut doc = doc;
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 2, "{report:?}");
let left = annots_of(&doc, p);
assert_eq!(left.len(), 1);
assert_eq!(left[0].get(b"A"), Some(&here), "the thread in this document stays");
}
#[test]
fn an_outline_over_the_limit_stops_the_optimization() {
use pdfcraft_optimize::OptimizeError;
let bookmarks = vec![action("Named", None); 100_001];
let (doc, _, _) = linked_doc(|_, _| Vec::new(), &bookmarks);
let r = optimize(&mut doc.clone(), &Settings { discard_external_refs: true, ..Settings::default() });
assert!(matches!(r, Err(OptimizeError::ExternalRefLimit(_))), "{r:?}");
}
#[test]
fn bookmarks_stored_inside_other_bookmarks_lose_their_actions_too() {
let (mut doc, _, _) = linked_doc(|_, _| Vec::new(), &[]);
let bookmark = |title: &str, a: Object| {
let mut d = Dict::new();
d.set(b"Title".to_vec(), Object::String(pdfcraft_cos::PdfString::literal(title.as_bytes().to_vec())));
d.set(b"A".to_vec(), a);
d
};
let third = doc.add(Object::Dict(bookmark("C", action("GoToR", None))));
let mut second = bookmark("B", action("Launch", None));
second.set(b"First".to_vec(), Object::Ref(third));
let mut first = bookmark("A", action("GoToR", Some(action("Named", None))));
first.set(b"Next".to_vec(), Object::Dict(second));
let mut outlines = Dict::new();
outlines.set(b"First".to_vec(), Object::Dict(first));
let root = doc.root().unwrap();
doc.update_dict(root, |c| c.set(b"Outlines".to_vec(), Object::Dict(outlines))).unwrap();
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_bookmarks, 3, "{report:?}");
let catalog = doc.get(root).as_dict().cloned().unwrap();
let first = catalog.get(b"Outlines").and_then(Object::as_dict).and_then(|o| o.get(b"First")).and_then(Object::as_dict).cloned().unwrap();
assert_eq!(chain(&doc, first.get(b"A").unwrap()), ["Named"]);
let second = first.get(b"Next").and_then(Object::as_dict).cloned().unwrap();
assert!(second.get(b"A").is_none());
assert!(doc.get(third).as_dict().unwrap().get(b"A").is_none());
}
#[test]
fn a_removed_link_kept_by_the_structure_tree_reads_back_without_its_action() {
let mut link = None;
let (doc, p, _) = linked_doc(
|doc, _| {
let r = doc.add(link_with(Some(action("GoToR", None)), None));
link = Some(r);
vec![Object::Ref(r)]
},
&[],
);
let mut doc = doc;
let mut objr = Dict::new();
objr.set(b"Type".to_vec(), Object::name("OBJR"));
objr.set(b"Obj".to_vec(), Object::Ref(link.unwrap()));
objr.set(b"Pg".to_vec(), Object::Ref(p));
let mut element = Dict::new();
element.set(b"S".to_vec(), Object::name("Link"));
element.set(b"K".to_vec(), Object::Dict(objr));
let mut tree = Dict::new();
tree.set(b"Type".to_vec(), Object::name("StructTreeRoot"));
tree.set(b"K".to_vec(), Object::Dict(element));
let tree = doc.add(Object::Dict(tree));
let root = doc.root().unwrap();
doc.update_dict(root, |c| c.set(b"StructTreeRoot".to_vec(), Object::Ref(tree))).unwrap();
let report = optimize(&mut doc, &Settings { discard_external_refs: true, ..Settings::default() }).unwrap();
assert_eq!(report.external_links, 1, "{report:?}");
assert!(annots_of(&doc, p).is_empty(), "the link leaves its page");
let bytes = write_full(&doc, &SaveOptions::default()).unwrap();
let reopened = Document::open(Arc::new(bytes)).unwrap();
let catalog = reopened.get(reopened.root().unwrap()).as_dict().cloned().unwrap();
let tree = reopened.dict(catalog.get(b"StructTreeRoot").unwrap()).unwrap();
let element = reopened.dict(tree.get(b"K").unwrap()).unwrap();
let objr = reopened.dict(element.get(b"K").unwrap()).unwrap();
let kept = reopened.dict(objr.get(b"Obj").unwrap()).unwrap();
assert_eq!(kept.name(b"Subtype"), Some(&b"Link"[..]));
assert!(kept.get(b"A").is_none(), "the link's object holds no action to another document: {kept:?}");
}
#[test]
fn progress_is_reported_per_image_and_a_refusal_cancels() {
use pdfcraft_optimize::{OptimizeError, Stage, optimize_with_progress};
+1
View File
@@ -214,6 +214,7 @@ pub(crate) fn body(ui: &mut egui::Ui, d: &mut OptimizeDraft, t: &Tokens) -> (boo
discard_box(ui, &mut d.discard, Hidden::Comments, "Discard all comments, forms and multimedia");
discard_box(ui, &mut d.discard, Hidden::Metadata, "Discard document information and metadata");
discard_box(ui, &mut d.discard, Hidden::Attachments, "Discard all object data (file attachments)");
ui.add(widgets::checkbox(&mut s.discard_external_refs, tl!("Discard external cross-references")));
discard_box(ui, &mut d.discard, Hidden::PrivateData, "Discard private data of other applications");
discard_box(ui, &mut d.discard, Hidden::HiddenText, "Discard hidden text");
}
+5 -5
View File
@@ -7526,11 +7526,11 @@ title = "Discard external cross-references"
area = "K"
tier = "P2"
milestone = "M11"
status = "planned"
commands = []
tools = []
evidence = []
notes = ""
status = "shipped"
commands = ["optimize.advanced"]
tools = ["doc_optimize"]
evidence = ["crates/optimize/tests/optimize.rs::links_and_bookmarks_to_other_documents_are_discarded_and_links_within_stay", "crates/optimize/tests/optimize.rs::an_action_to_another_document_hands_its_place_to_the_actions_after_it", "crates/optimize/tests/optimize.rs::a_chain_deeper_than_the_limit_loses_its_tail_and_one_without_a_remote_action_stays", "crates/optimize/tests/optimize.rs::a_long_chain_of_indirect_actions_stays_indirect_and_reads_back_after_saving", "crates/optimize/tests/optimize.rs::shared_actions_that_multiply_stop_the_optimization_when_one_opens_another_document", "crates/optimize/tests/optimize.rs::a_shared_chain_without_remote_actions_is_looked_at_once", "crates/optimize/tests/optimize.rs::a_link_named_through_an_indirect_subtype_is_discarded_and_a_form_field_keeps_its_action", "crates/optimize/tests/optimize.rs::thread_actions_into_another_file_and_data_imports_are_discarded_and_threads_here_stay", "crates/optimize/tests/optimize.rs::an_outline_over_the_limit_stops_the_optimization", "crates/optimize/tests/optimize.rs::bookmarks_stored_inside_other_bookmarks_lose_their_actions_too", "crates/optimize/tests/optimize.rs::a_removed_link_kept_by_the_structure_tree_reads_back_without_its_action", "crates/automation/tests/it/automation.rs::optimizing_discards_links_to_other_documents_through_tools"]
notes = "PDF Optimizer ▸ Discard User Data, off by default. Takes GoToR, GoToE, Launch and ImportData actions, and Thread actions with an /F, out of the /A of link annotations (/Subtype /Link, stored directly or as an indirect name) and bookmarks, and the actions chained after one through /Next take its place in order. A link left with no action and no /Dest is removed from /Annots, and when the link is an indirect object its object is rewritten without the action for its other users (the structure tree, for example); a bookmark keeps its title and children. Bookmarks stored directly inside another bookmark or the outline dictionary are changed in place; an outline of more than 100,000 items, or with direct items nested more than 512 levels deep, stops the optimization with an error. Links within the document (/Dest, GoTo) and web links (URI) stay. An indirect action that changes is written as a new indirect object, except the first action left at the head of an /A, which is copied into the /A directly when other actions follow it; the old one stays for its other users. Past 512 nested levels (actions, /Next arrays and references), the rest of a chain is kept when it opens no other document and removed when it does. An /A with more than 100,000 values to check, or one that opens another document and needs more than 100,000 values to rewrite (actions shared so that their paths multiply), stops the optimization with an error. Not changed: /AA triggers, widget and other annotation actions, the catalog /OpenAction, and JavaScript."
[[feature]]
id = "optimize.discard-hidden-layers"