Commit Graph
37 Commits
Author SHA1 Message Date
Dylan M. Taylor 0a11922520 fix(create): preserve version-four PCX compatibility 2026-11-14 23:59:16 -05:00
Dylan M. Taylor e9f7aeaeee refactor(create): decode PCX through shared image codecs 2026-11-14 23:56:51 -05:00
Dylan M. Taylor 7d44e65412 fix(xfa): resolve clippy lints on main (#7354)
* fix(xfa): resolve clippy needless_update and field_reassign_with_default

* fix(create,xfdf): resolve remaining clippy lints

* fix(export): remove needless borrow in pptx and as_docx

* fix(engine): resolve clippy lints in signature_image

* fix(automation): resolve clippy lints in tests

* fix(create): avoid clippy::unreachable in tests

* chore(deny): allow IJG license for jpeg-encoder via craft-image-codecs
2026-11-14 16:37:43 -05:00
Dylan M. Taylor eead454105 Cut over remaining codecs, xfa XML, and export packaging to craft-libs (#7326)
* chore: cut over image, png, tiff, and zune-jpeg to craft-image-codecs

* chore: cut over xfa XML parsing from roxmltree to craft-xml

* chore: cut over export DOCX and PPTX packaging to craft-opc and craft-xml

* style: cargo fmt
2026-11-14 13:49:46 -05:00
Max Abramowitz 3a117c3c0e M10: open Word, Excel and PowerPoint files through LibreOffice
## What this changes

**File ▸ Open opens Word, Excel, PowerPoint, OpenDocument and RTF files** as new, unsaved PDFs (`Report.docx` → `Report.pdf`), the "LibreOffice sidecar" that `docs/gaps.md` (gap 8) and `docs/file-format-parity.md` planned for `create.from-office`.

- **Conversion** (`crates/create/src/office.rs`): an installed LibreOffice runs headless as a separate process, in a private temporary folder (the input is always `document.<ext>`, so the user's file name never reaches its command line). It gets a fresh profile with `DisableMacrosExecution`, `MacroSecurityLevel` 3 and `BlockUntrustedRefererLinks` set, and Writer's and Calc's `Content/Update/Link` set to *Never* (2 for Writer and 1 for Calc: the two number their values differently), no console window on Windows (`CREATE_NO_WINDOW`), and is killed after 120 s. The output is size-capped (512 MB), must start `%PDF-`, and must open as a PDF before it is used. LibreOffice is found through `PDFCRAFT_SOFFICE`, the `PATH`, then the usual install folders per OS. Without it, the user is told it's needed rather than "not a readable PDF".
- **Off the frame thread:** File ▸ Open converts on the existing open worker (`ResultKind::Converted`); the frame thread only opens the result.
- **Detection:** `SourceKind::Office` requires an Office extension *and* an OLE, ZIP or RTF signature, so `photos.zip`, or a PDF saved with a `.docx` name, are not misread.
- **Open dialog:** PDF (still the default), Images, Office documents, Text, All files. All files is left out on macOS, where rfd merges every filter into one allowed-types list, and on the web; Office documents are left out on the web. The extension lists are now shared constants in `pdfcraft-create` (`IMAGE_EXTENSIONS`, `TEXT_EXTENSIONS`, `OFFICE_EXTENSIONS`), used by both detection and the picker. The duplicate filter list in `open_dialog` is gone. The two new labels are in all 22 catalogues, with their `ATTRIBUTION.toml` hashes updated.
- **Automation:** `doc_create_multiple`, `doc_combine` and batch `convert` go through `convert_to_pdf`, so they take Office documents too; their descriptions say so.

**Deliberately not in this PR:** Insert pages, Combine files and Create ▸ Multiple files convert on the frame thread, so they now refuse Office documents with a pointer to File ▸ Open (`convert_to_pdf_inline` / `as_pdf_inline`) rather than freezing the window for seconds. Moving them onto a worker is the follow-up. `create.from-office` goes planned → **partial**, with that and the rest listed in its notes.

## How you verified it

On Windows 11 Pro with LibreOffice 26.8.1 installed, per crate:

- [x] `cargo fmt --check`
- [x] `cargo clippy -p pdfcraft-create -p pdfcraft-engine -p pdfcraft-ui-egui -p pdfcraft-automation -j 2 --all-targets -- -D warnings`
- [x] `cargo test -p pdfcraft-create -p pdfcraft-engine --lib` (42 and 276 pass); `-p pdfcraft-automation --test automation` (the combine/create tests); `-p pdfcraft-ui-egui --lib open_worker` and `--test editing --test create`
- [x] `cargo xtask layers`, `cargo xtask assets`, `cargo xtask parity --partial`
- [ ] wasm check: the `wasm32-unknown-unknown` target isn't installed here, so I'm leaving it to CI. The web paths are `cfg`'d and were reviewed by eye.

**Tests seen failing before the fix**, by reverting each part:
- Detection off → `office_documents_are_their_own_kind`, `source_kind_tells_pdfs_images_and_text_apart`, and both engine Office tests fail.
- Worker conversion off → `file_open_converts_an_office_document_on_the_worker` fails with the old user-facing message, *"the file is not a readable PDF: no readable page tree or cross-reference data was found"*.

New tests also cover a converter that hangs (it's stopped; the test binary stands in for LibreOffice), one that writes no PDF, a missing converter, temp-folder cleanup, `file://` URL escaping (spaces, drive letters, UTF-8), and the search order.

**Real conversions, end to end:** `crates/automation/tests/it/office.rs` builds a minimal DOCX, an XLSX (with a formula, `=6*7`), a PPTX and an RTF inside the test, so no binary fixtures are committed. It runs them through `doc_create_multiple` in `separate` mode (each output reopened and its text checked, including `42`) and in `combine` mode (page order and bookmarks), and through `doc_combine` beside a PDF, with a page range on one file. These tests, `libreoffice_converts_a_document_when_installed` and the process-tree cleanup test (5 out of 5 reruns) pass with LibreOffice 26.8.1. The Office tests skip where LibreOffice isn't installed, which includes CI.

Three existing tests used a 4-byte `report.docx` as their "file that can't be converted" example. That's no longer true, so they now use `archive.zip`, which keeps what they test.

## Never-crash

- [x] No new `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!` outside tests
- [x] No input-derived indexing or slicing (`rsplit_once` for the extension); the PDF read back is size-capped
- [x] No `unsafe`. The Windows flag uses `std::os::windows::process::CommandExt`
- [x] A crashing, hanging or misbehaving LibreOffice is an error, never a panic or a stuck app: the worker's `catch_unwind`, a 120 s kill, and output checks

## Provenance

- [x] No Adobe assets; no fonts; no copied code
- [x] LibreOffice (MPL-2.0) is only *run* as an external process, as the docs planned; nothing of it is linked or copied. Its command-line options and the profile keys come from LibreOffice's public documentation.
- [x] The translations are contributor-original, written from the plain meaning of "Office documents" and "All files" ("Office" kept as a product name); hashes updated, `cargo xtask assets` passes

## Security

- [x] `--root` confinement unchanged: automation still resolves and reads paths before converting; only the private temp folder is written outside it
- [x] MCP and the control channel untouched
- [ ] **No new outbound network: please decide.** PdfCraft itself opens no connection. The profile blocks untrusted referer links and sets Writer and Calc never to update linked content on load. I converted eight synthetic files that link to a listener on 127.0.0.1: RTF `INCLUDETEXT` and `INCLUDEPICTURE`, a DOCX external picture, an ODT linked section and linked picture, an ODS table source and an external cell reference, and an ODP linked picture. None made a connection, with or without the link settings. Two things remain open: Impress has no link-update setting, and the process's network access isn't sandboxed (documented in `crates/create/README.md` and `docs/gaps.md`).

## Parity and reach

- [x] `create.from-office` planned → partial, with the evidence tests and what's still missing
- [x] Reachable headlessly (`doc_create_multiple`, `doc_combine`, batch `convert`)
- [x] Docs: `docs/file-format-parity.md` (Word/Excel/PowerPoint and RTF rows), `docs/gaps.md` gap 8, `docs/target-app-parity.md` row J (numbers not re-measured), `docs/roadmap.md` M10, the `ROADMAP.md` log, and the `crates/create` README

## Anything a reviewer should look at first

1. **The network question** above. It's the owner's call.
2. **The frame-thread restraint:** Insert, Combine and Create Multiple refuse Office documents rather than block. Is a worker for those wanted in this PR, or as a follow-up?
2026-11-11 11:06:01 +01:00
oracola91 12a9b23394 Create PDF from text takes Cyrillic and Greek, and reports no longer escape them (#5403) 2026-11-09 18:31:29 +08:00
Dylan M. Taylor b3793789f6 fix(create): bound a PNG's frame buffer by what its bytes can hold; no truncating JP2 box lengths (#6007) 2026-11-09 18:18:11 +08:00
1d71d6039f Add PCX image support to Create PDF conversion (#6396)
* Add PCX image support to Create PDF conversion

PCX is listed as an accepted Create source elsewhere but PdfCraft rejects it, so users cannot convert PCX images to PDF.

Detect PCX by its header in is_image and decode it in embed, adding pcx to CONVERTIBLE so existing from_images and source_kind paths handle it.

Test: pcx_images_become_pages failed before the change and passes after.

* Create: PCX scanlines hold every plane's row, 8-bit indexed images, older versions, standard test files

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 13:16:36 +01:00
9f04acccef M10: OCR engines, confidence, preprocessing and a review workflow (#977)
* M10.20: plan Unicode text for OCR output in Type3 faces (fonts, edit)

OCR output used to flatten every character beyond WinAnsi to a silent `?`. This
adds the two pieces that let the OCR layers write the recognized words as they
are; the OCR code that uses them follows in M10.21.

pdfcraft-fonts
- `TextPlan::for_words` maps each word to the standard Helvetica when WinAnsi
  covers it, else to Type3 fonts drawn from the craft-fonts faces (the Japanese
  document face first, then the rest of the build input). Characters no face
  has a glyph for are counted (`missing`), never dropped silently. At most 240
  characters per font and 2400 per plan, so one hostile word cannot balloon.
- Which face has which character is asked once per (face, character) for a plan,
  not once per word.
- `face_has`, `craft_glyph`, `win_ansi_char`, `fits_win_ansi` are public for it.

pdfcraft-edit
- `add_type3_fonts` / `stamp_with_fonts` register a plan's fonts on a page under
  a free `PCOcr`, `PCOcr1`, ... name, build each font once and share the object
  with later pages. A shared face is registered under a free name on a later
  page that already has a font of the name it was given first, instead of
  replacing that page's own font.
- The fonts carry real advances and a ToUnicode CMap, so extraction reads the
  text back.

pdfcraft-redact
- A test shows a hand-built Type3 text layer (the shape the OCR output writes) is
  redacted like any other text and the verifier passes.

Tests: pdfcraft-fonts 62 passed, pdfcraft-edit 129 passed, pdfcraft-redact 33
passed; clippy -D warnings clean on the three crates. The name-reuse and the
redaction tests build their fonts by hand, so they run without the craft-fonts
build input; the tests that plan real faces skip without it and were not run in
this commit's own check.

* M12: give the hostile-XFA open test more room on a loaded machine

The test opens a form whose calculate script sets a value 50 000 times, against
the one-second budget of an unasked script and a five-second wall-clock bound
on the open. Both sit close to what a machine running several suites at once
can take, so the test can fail without anything being wrong.

It now runs 20 000 iterations (the total is 19999) and allows 30 seconds, which
still fails for a script that hangs or is not capped. Test-only; unrelated to
the OCR work in this series, so it is a commit of its own.

Tests: both hostile_xfa tests in pdfcraft-engine pass.

* M10.21: pluggable OCR engines, preprocessing, review sessions and the OCR tools (ocr, engine, create, automation)

Scan & OCR gets a second engine, optional raster clean-up, a review step before
anything is applied, editable output and a batch mode, headlessly first: the
interface for them follows in M10.22.

pdfcraft-ocr
- A `Recognizer` trait with two engines: the built-in ocrs (moved out of lib.rs)
  and the installed tesseract program, run as a fenced external process: fixed
  argv, allow-listed language codes, a private temp directory (mode 0700 on
  Unix), 120 s timeout, 64 MiB output cap, 50 000-word reading cap. A location
  from PDFCRAFT_TESSERACT is validated (absolute, existing, executable) and an
  invalid one fails closed instead of falling back to the PATH. `--list-langs`
  and `--version` are read from stdout or, for older versions, stderr.
- Strategies: PrimaryOnly, ConfidenceWeighted (the second engine joins when the
  first reading's mean is under 70 or unknown) and a ROVER merge (higher
  confidence wins per word; against an ocrs word of unknown confidence a
  tesseract word wins only at 70 or more; capped at 100 M comparisons; a failing
  partner degrades to the primary with a note).
- Preprocessing: opt-in auto-rotate, deskew, denoise and binarize on the RGBA
  raster, each with an exact inverse so word boxes map back to the scan. The
  300 dpi normalization the steps are calibrated for runs only when a step is
  on: with every step off the render is read as rendered, as before. (Enlarging
  a 72 dpi render 4x on its own made the models read "due: €250" as "ue: ?250".)
  `OcrImage` is validated by construction (side and 64 MP area caps, checked
  before any buffer is allocated, including by the resize).
- Confidence bands (High >= 90, Medium 70-89, Low < 70) drive review.

pdfcraft-engine, pdfcraft-create
- `OcrSettings` gains engine, strategy, the four steps, output mode, region, the
  skip flags and the low-confidence threshold; Automatic reads confidence
  weighted unless a strategy is chosen.
- A recognition is a session: the document guard refuses a stale result with its
  reason, pages can be applied one at a time (the guard is taken again after each
  applied page), and applying is one undoable step. A per-language dictionary
  (a plain-text file: words added from any document stay on disk after it closes).
- Output is searchable (an invisible layer) or a NEW editable document of visible
  text at the recognized boxes (`create::from_contents`); words beyond WinAnsi use
  the Type3 faces of M10.20 in both, with the shared plan across pages.
- Engine discovery never blocks the interface: results are cached, a stale
  answer is shown while a background probe retries, and the models search and
  `--version` are cached. Batch: pages or files that already have text are
  skipped unless forced, and inputs sharing a file name are found up front.

pdfcraft-automation
- ocr_recognize (engine, strategy, steps, region, output mode), ocr_words (with a
  retained review handle), ocr_review_apply, ocr_dictionary, ocr_status and
  ocr_recognize_files. The editable `out` must be a new file inside --root and is
  refused for a searchable run (it used to be ignored), two batch inputs of one
  file name are refused before anything is read, and the batch folder is checked
  for existing outputs.

pdfcraft-ui-egui
- Only what the new API needs: the Recognize Text dialog reads with
  `recognizers(&settings)`. `ocr.correct` is registered with its label in every
  catalog; the screen itself is M10.22.

Tests: pdfcraft-ocr 78 passed; pdfcraft-create passed; pdfcraft-engine and
pdfcraft-automation 450 passed and 1 failed (the automation lib 17, automation.rs
157, hostile_args 2 among them); pdfcraft-ui-egui lib + ocr + control tests 391
passed; clippy -D warnings clean on the five crates. The one failure is in a
test this commit does not change:
recognize_text_makes_a_scanned_page_searchable fails where the models are
installed (they read "quick" as "auick" at that render size); the roadmap on main
already records it. The real tesseract binary ran in
tesseract::tests::recognize_runs_the_installed_binary (tesseract 5.4.0, PATH).

* M10.22: OCR Verify screen, the Recognize Text options and the interface catalogs (ui-egui)

Scan & OCR ▸ Correct recognized text: a full-window screen where a recognition
is reviewed before anything is applied, plus the dialog and batch options for
the engines of M10.21.

OCR Verify screen
- Four panes: the page list, the source page with confidence-coloured word boxes
  (click to select, drag to draw a region), the recognized text, and a zoom plus
  word inspector. Corrections commit on Enter (an empty commit removes the word),
  Remove/Restore, "+ Dictionary" (a per-language file in the preferences folder,
  so a word added from a document stays on disk after it closes), per-word and
  per-page Verified, F8 / Shift+F8 walk the low-confidence words, Ctrl+T marks the
  page reviewed, Ctrl+Tab moves the pane focus, and review edits undo within the
  panel.
- Re-OCR of the whole page or a drawn region. A region re-read moves the review
  marks of the words it keeps to their new positions and selects the first new
  word; a whole-page re-read clears that page's marks (and the undo brings them
  back). The region outline is scaled onto the pane, not mapped through user
  space again, so it is right on rotated and cropped pages.
- Accept / Reject per page or whole. Pages accepted one at a time each apply once
  (the document guard is taken again after each applied page; an edit made
  elsewhere in between is still refused with its reason), and Accept in
  searchable mode is one undoable step. Editable mode writes a NEW
  `<name>_ocr.pdf` through a save dialog and never the source. A real Cancel
  keeps the pages already read.
- The engine and strategy menus disable a choice that cannot run and say why;
  the check never waits on a process (a background probe fills a cache), and
  choosing Automatic moves "Primary only" to Confidence weighted.

Recognize Text dialog and batch
- Engine, strategy, the four clean-up steps and the skip flags are remembered with
  the other choices (a saved Automatic without a strategy reads confidence
  weighted). The batch refuses two inputs of one file name and existing outputs
  before reading anything, notes low-confidence pages at the settings' threshold,
  and its toast has a Cancel (the file in flight finishes).

Catalogs and attribution
- The 90 new interface strings are in all 21 catalogs (the Lithuanian ones were
  written without native review, and the Bulgarian ones avoid U+045D, which the
  bundled monospace face lacks). ATTRIBUTION.toml carries the new hashes, a
  note on the Lithuanian entries and the README screenshot; ATTRIBUTION.md is
  regenerated.
- README: a screenshot of the Verify screen, taken with
  `cargo xtask screenshots` from a synthetic scan (`shot --ocr-verify on`) built
  from allowed assets; the page reads it with the real engines.

Tests: pdfcraft-ui-egui lib 373 passed (1 ignored: the Windows hinting
measurement probe), tests ocr 6, control 46, fonts 9; clippy -D warnings clean on
the engine, ui-egui, pdfcraft and xtask packages; `cargo xtask assets` ok. The
review state machine has unit tests for every fix above; each was seen failing
with its fix reverted.

* M10.23: parity entries, roadmap and localization notes for the OCR workflow

Records what M10.20-M10.22 ship, and what they do not.

parity/acrobat-features.toml
- Status changes against main: ocr.rotation-detection and ocr.deskew planned ->
  shipped (the opt-in steps of M10.21, with their tests); ocr.correct-recognized-
  text and ocr.ocr-engines planned -> partial. Other Area I entries get their
  evidence and notes brought up to date.
- ocr.correct-recognized-text: the evidence includes the one-page-at-a-time accept
  and the region re-read tests; the notes say what the headless tools cover
  (ocr_words with a retained review, ocr_review_apply through the same stale-
  document guard, ocr_dictionary) and what is still UI-only (suggestions,
  per-word Verified marks, region re-reads).
- ocr.ocr-engines stays partial rather than shipped: part of its evidence is
  conditional (the real-binary test runs only where tesseract is installed and
  skips elsewhere, and the process-level tests are Unix-only). It cites the tests
  that always run (location validation and the fail-closed rule, --list-langs
  parsing from stdout or stderr, the merge rules, the default strategy) and says
  which part is not proven everywhere.
- `cargo xtask parity`: area I 32.7% -> 44.2% weighted (5 -> 7 shipped, 7 -> 9
  partial); overall 52.8% -> 53.0% shipped and 58.2% -> 58.5% weighted.

ROADMAP.md
- One progress-log entry (2026-10-13, newest first) and the banner line. It says
  the catalogs are 21 with the Lithuanian additions not natively reviewed, that
  the end-to-end redaction test of the Type3 layer needs the craft-fonts build
  input, and that recognize_text_makes_a_scanned_page_searchable still fails where
  the models are installed, as main's log already records.

docs/localization.md, docs/localization-parity.md
- The Lithuanian entry count (2,578 -> 2,668) and the counts of the other
  catalogs that carry entry counts are re-measured with the 90 new entries; the
  Lithuanian OCR entries are marked as not natively reviewed.

Checked: `cargo xtask parity` (numbers above, evidence tests exist) and
`cargo xtask assets` pass on this tree.

* M10.24: ocr: apply a late result only to the document it was read from

The Recognize Text dialog closes on Go, so the document can be edited, or have
text redacted, while a long run is still reading. The result was applied
without the stale check the tools and the Verify screen use, so words read
before a redaction could be written back as invisible text. The run now keeps
its guard and applies through the checked path.

Accept with the editable output in the Verify screen had the same gap (Exit
keeps the review open): it now refuses when the document changed since the
review started.

Both new tests fail with their guard switched off.

* M10.25: ocr: ocr_dictionary takes dictionary file names only

Updating sorts, lower-cases and rewrites the file it is given, so pointed at
.env or a README it would corrupt it. The tool now accepts only
ocr-dictionary-<language>.txt style names, for reading as well as writing.

The size-limit test used a file name the rule now refuses first, so it renamed
its fixture to keep testing the size limit. The new test fails with the rule
switched off.

* M10.26: pt-BR: keep (s) on the word counts and say mecanismo for engine

Two review fixes from a native speaker: the two word(s) lines had lost their
(s), so a count of 1 read as 1 palavras, and engine was translated both as
motor and mecanismo. Mecanismo is the Brazilian word for a software engine and
two lines already used it, so the other six now match.

* M10: fit the rebased OCR workflow to current main's structure

* Fix OCR workflow issues from the review

Honor the Recognize dialog's editable-output choice: its result opens in
the review screen (whose Accept writes the editable document) instead of
an invisible layer. Make page acceptance idempotent: a second Accept
entire page never appends the OCR layer twice and keeps later
corrections. Move page and region re-OCR onto a worker thread with a
status line and a cancel that discards the reading. Skip the Tesseract
probe for jobs that name their engines. Bound Tesseract's stderr while
the child runs, not only after it exits. Surface the editable output's
substitution notes in the tools' and the run's results.

Also repair the OCR batch tool against the current structure: inputs are
held to the read root again, and a batch whose inputs share an output
name is refused before the output folder is created, compared the way
the default filesystem compares names; and the editable writer test's
word stays inside the fixture's pages.

* Apply pre-review fixes to the OCR workflow

Hold the automation OCR tools to the document's copying permission, as
the other text tools are: the recognized text is left out of the
results when copying is forbidden, and the paths that carry the words
out of the document (the editable output, the review word list) are
refused. Mark the ocr_dictionary tool as overwriting its path. Close
the searchable layer's graphics state around its invisible text so the
rendering mode cannot leak into later content. Read the OCR user
dictionary folder from the portable data folder in portable mode, like
the recovery store does. Resolve the attribution manifest's leftover
conflict markers and regenerate the generated attribution. Describe the
review flow, the searchable layer's Type3 fonts and the parity evidence
references accurately, and drop an undeclared copy of the recovery
module.

* OCR: reject readings that exceed the word limit

Assisted-by: OpenAI Codex

* OCR: new documents go through the page tree builder main added

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OCR: review notes and lints

The editable output refuses a file created during recognition. ocr_words takes
skip_text_pages and force_ocr. Automatic OCR checks the engine it will use. The
stderr test stays below the kill cap, and a failed --ocr-verify shot exits with
an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OCR: restore the Verify screenshot attribution, follow the moved constants

The merge dropped the screenshot row with the translation rows. The model
filename constants live in ocrs.rs now, so the license check reads them there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Attribution: the Verify screenshot entry goes after the header comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Attribution: the screenshot row in the table follows the manifest order

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 23:39:55 +01:00
Dylan M. Taylor 97260514e6 fix(create): write a new document's page tree once (#5865)
Create PDF added each page to the page tree through Document::update_dict, which copies the tree's dictionary and its whole /Kids array, so a text of many form feeds took quadratic time: 40,000 pages took 108 s in a debug build. Pages are now collected and /Kids and /Count are written once, for blank, image and text documents.
2026-11-02 23:36:51 -05:00
Grzegorz MartensandClaude Opus 5.5 0ed339901e M2.6: the JBIG2 images of an operation or a page share one allowance of decoding
One image has its own limit, about 6 s to 17 s, but nothing bounded a file of thousands of images
made to reach it: an operation took that long for each, and an image sharing /JBIG2Globals decodes
them again.

Image::decode_within (hayro-jbig2 patch 11) spends from what a run may still do, starting with its
page's pixels, and charge_reading charges reading a stream and its globals, 4 steps a byte, before
anything is parsed, so an image that fails has paid too. pdfcraft_filters::decode_jbig2_within takes
it, and ImageBudget in cos is that allowance: 2^34 steps, about 1,000 scanned pages, and 2^13 more for
each byte of the file, so a long scan is read whole. An image past it spends all that's left, and
later ones aren't read, globals included.

Optimize leaves such an image as it is and counts it in Report::images_refused. Image export leaves it
out, with the reason, and stops at 2 GiB of files held. Office export shares one allowance, leaves the
picture out and says the page was truncated. Redaction removes it whole, as any image it can't read.

Rendering gives a page's images 2^30 steps between them, inside the watchdog, and each page starts
afresh in Context::new as its other budgets do (hayro-syntax patch 31, hayro-interpret patch 73).
It's a thread-local, unbounded until a page sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
2026-10-31 13:21:45 +01:00
Grzegorz MartensandClaude Opus 5.5 9883da3bc3 M11: Create PDF stores black-and-white images at one bit, as Group 4 when smaller
A scan saved as a 1-bit PNG decodes to eight-bit grey and was stored
that way, larger than the PNG it came from; a bilevel TIFF was stored at
one bit but with Flate. Now any image of only pure black and white
(1-bit, eight-bit grey, a two-colour palette, RGB without a profile) is
stored at one bit per pixel, as CCITT Group 4 or Flate, whichever is
smaller as stored. A grey profile stays; RGB with a profile keeps its
colours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
2026-10-29 20:48:02 +01:00
KenM76andClaude Opus 5.5 38c9482191 fix(xfa, create): read a BMP's pixels from after its headers when its offset points into them
LiveCycle Designer wrote BMPs whose file header gives bfOffBits = 0. image
(0.25) seeks to that offset as given, so the pixels were read from the file
header on and the picture was drawn shifted and wrapped around (#4937).

`pdfcraft_xfa::image::bmp_with_pixel_offset` recomputes the offset from the
info header (its size, BI_BITFIELDS masks in a 40-byte header, the palette)
when the declared one lies inside the headers and the computed one fits in
the file; anything else is passed through unchanged and not copied. XFA
pictures (size and decode) and `create::from_images` both go through it, so
pdfcraft-create gains a dependency on pdfcraft-xfa (L4 -> L3).

Fixes #4937

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
2026-10-27 05:00:44 -04:00
a047c6ba09 chore: resolve SonarCloud maintainability issues (#4356)
* chore: resolve SonarCloud maintainability issues

* fix(packaging): use verify_sha256 helper function

* style: cargo fmt

* fix: restore wildcard imports and fix method-reference type errors

Revert the glob-import expansions (they missed SomIndices and left unused
imports), revert char::is_ascii_digit where the iterator yields char by
value, and pass the content slice to prune_doomed_xobjects as &[Vec<u8>].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep XObject decodability check without binding content in interp

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan Taylor <dylan@craftmaker.dev>
2026-10-23 23:28:18 -04:00
aiongg c704c1089a fix(create): keep the TIFF total budget in range on 32-bit targets 2026-10-22 21:55:12 +00:00
aiongg cad0fe4526 fix(create): refuse a GIF screen past 4096x4096 and convert its canvas in place
A one-pixel GIF on a huge logical screen decodes to a canvas that size,
which the conversion then copied before splitting it into colour and alpha,
so a few bytes could hold over a gigabyte. Screens past 16 Mpx are refused
before decoding, and the decoded canvas is converted without a copy.
2026-10-22 21:17:15 +00:00
aiongg 7e9f5a4435 fix(create): bound a TIFF's decoded pages together, and read one when one is wanted
A TIFF's pages were each capped at 2 GiB but all kept in memory until
embedded, so many small directories sharing one compressed strip could ask
for many times that; embedding one image also decoded every page. The pages
of one TIFF are now refused past 4 GiB together, and image_xobject reads
only the first page.
2026-10-22 21:17:15 +00:00
NetterSilverandClaude Haiku 5.5 43f8ddf264 Apply JPEG EXIF orientation when creating PDFs from images (#2951)
Fixes #2944

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-10-18 03:14:25 -04:00
dfa43beb9f M10: BigTIFF and strict GIF detection in Create PDF; XDP data in format parity (#1453)
- Create PDF detects BigTIFF (II+/MM+) as well as classic TIFF; the tiff
  decoder already reads it, so these scans now convert instead of being
  refused.
- Image detection accepts only GIF87a/GIF89a, the headers conversion
  decodes, so a malformed GIF header is no longer called an image and then
  rejected with a misleading error.
- file-format-parity.md: XDP split into XFA data import/export (shipped
  under form.xfa-data) and opening .xdp as a document (none).

Tests: bigtiff_images_become_pages; malformed GIF headers in
source_kind_tells_pdfs_images_and_text_apart. Both fail without the fix.


Claude-Session: https://claude.ai/code/session_01F8DFYWkDyTnEcin9PNkUW5

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-13 01:19:48 -04:00
Zeyu LiandBrandon Thomas b91d38c073 M14: text the fonts can't draw is kept and reported, never written as ? silently (#1112)
* M14: text the fonts can't draw is kept and reported, never written as ? silently

Form values and captions, text boxes, callouts, Fill & Sign text, measurement labels and stamp by-lines keep text the standard fonts can't draw, draw ? for it, mark the annotation (PCUndrawn) and report the edit (interface notice, tools' warnings); forms set /NeedAppearances (not in PDF/A). Flattening, redaction overlay text, typed and visible signatures and create-from-text refuse instead. Reports escape such characters as [U+XXXX].

* Catalogs: the two new can't-draw messages in Czech, Slovene, Kazakh, Turkish and Greek; refresh hashes

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-11 17:53:55 -04:00
b642e327e7 Security: fix 10 verified audit findings in redaction, signature validation, and untrusted-input handling (#1099)
* fix(create): cap decoded size of imported PNGs

A PNG's declared dimensions sized an infallible allocation, so a few-byte file claiming huge dimensions aborted the process when the allocation failed. Cap the decoded size at the TIFF ceiling and reserve every input-sized buffer fallibly, returning CreateError::Image instead. (Security audit run-2: v3-024.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(xfa): bound total picture bytes kept during layout

Each laid-out picture kept its own copy, so one picture in a repeating subform multiplied without bound and could exhaust memory on open. Add a per-layout aggregate cap (256 MiB) that stops with XfaError::TooLarge. (Security audit run-2: v3-103.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(xfdf): bound XML nesting and protect redaction marks on import

Deeply nested XFDF/XML overflowed roxmltree's recursive parser and aborted the process; a non-recursive depth scan now rejects it before parsing. Imported comments could also replace a locked or redaction mark by /NM or add a hidden one; place() no longer replaces locked/Redact annotations and clears the hidden/invisible flags on imported redaction marks. (Security audit run-2: v3-105, v3-111.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(sign): classify post-signing changes by the signed revision's roles

A signed object's modification was classified from labels in the attacker-written new version, so relabelling a signed page /Type /Sig (or giving it /ModDate), or relabelling a drawn stream /ObjStm or /XRef, let a forged page validate as Valid even under DocMDP P=1. Decide an object's role (page content, signature, metadata, file plumbing) from what it was in the signed revision, not its new dictionary. (Security audit run-2: v3-082, v3-083.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(redact): remove redacted content from the saved file, fail closed otherwise

Apply Redactions and Remove Hidden Information left the originals behind: dropped or cleaned image/form XObjects kept their resource names (so the full rewrite saved the originals), an XFA form kept the redacted value in its datasets packet, and vertical-writing text under a mark was never hit by the horizontal removal geometry yet reported success. Prune replaced XObject names the page no longer draws, drop /XFA and /NeedsRendering when redacting an XFA form, and fail closed (RedactError::VerticalText) rather than silently leave vertical text. (Security audit run-2: v3-064, v3-067, v3-062.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(ui-egui): trust the displayed signer, not the document's embedded root

The Signatures panel's one-click 'Add to trusted certificates' trusted the top of the chain the document supplied, installing the author's self-signed root as an anchor instead of the signer shown. Trust the displayed signer certificate. (Security audit run-2: v3-098.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Use main's decoded-size ceiling for PNGs too; keep the depth tests on main's checker

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-11 15:09:39 -04:00
dd065b39bb M4, #569: Combine files takes images and text files (#1032)
* M10: refuse a PNG whose header claims more than 2 GiB of pixels before allocating

The png crate's limits don't cover the caller's frame buffer, which png_image sized from the
header alone: a few-KB PNG claiming 60000x60000 RGBA made Create allocate and fill 14.4 GB.
Apply the 2 GiB ceiling TIFF pages already have. Reached from Create, Insert pages and (with
#569) Combine files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M4, #569: Combine files takes images and text files

PNG, JPEG, TIFF, GIF, BMP, JPEG 2000 and .txt files become PDFs as they join the Combine files
list (Add files…, drops, folder scans), so each is an ordinary row or card with its pages,
thumbnails and page range. Session::as_pdf does the conversion; PDFs pass through unread
(Combine reads them itself, with their passwords), as does a file named .pdf without a header,
which the reader may still open. doc_combine accepts the same file types, and a converted
file's bookmark drops its extension.

A file that can't be read (a damaged image or PDF, a type that can't be converted) no longer
is refused at the door: it joins the list with why in its Warnings (a red card in the grid).
Combine then asks "Leave out files that can't be read?", naming them; Combine the rest goes on
without them, Cancel changes nothing. Password-protected files still block Combine until
unlocked.

The picker offers "PDF, images and text", and the empty state, folder tooltips and folder
notices say what is accepted; translated in every full catalog (catalog hashes updated).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-11 12:04:10 -04:00
snipeTRandBrandon Thomas 9035ae3c82 fix(create): read UTF-16, byte-order-marked and Windows code page text files (#903)
Text files were decoded with String::from_utf8_lossy wherever they are
read: Create PDF and Combine from a .txt file, the automation `create`
tool's text `path`, and Search & Redact's word list file. So
- a UTF-16 file (Windows Notepad's "Unicode") became NULs and replacement
  characters ("??G?r?????e?" for "Grüße"),
- a UTF-8 byte order mark became a stray character at the start, and
- a Windows-1252 or 1254 file lost every accented letter ("caf�", "K��").
For a word list that means the words were never found and stayed
unredacted.

pdfcraft_create::decode_text picks UTF-8 or UTF-16 (LE/BE) by the byte
order mark and drops it, keeps valid UTF-8, and otherwise decodes the
Windows code page: 1254 when the bytes 1254 uses for Ğ İ Ş ğ ı ş appear
without á í ó ú (which Turkish never writes but Icelandic and Faroese,
the only Western languages using those bytes, always do), else 1252.
encoding_rs is already in the build through lopdf.

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-10 17:07:47 -04:00
7c386fa4ac M10.2: scan JPEG metadata beyond frame headers (#764)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-10 15:46:22 -04:00
06a7b9a7ca Issue #666: images made into a PDF keep their ICC profile (#729)
Create from images (doc_create, and image_xobject for stamps, signatures and
inserted images) dropped a JPEG's or PNG's embedded ICC profile, so a CMYK
JPEG became untagged /DeviceCMYK.

- JPEG: the APP2 ICC_PROFILE chunks are joined by sequence number; a profile
  with a missing or repeated chunk is ignored.
- PNG: the iCCP profile is read.
- A profile whose header is sound (acsp) and whose colour space matches the
  image (GRAY/RGB/CMYK for 1/3/4 components) becomes [/ICCBased <stream>]
  with /N and the device space as /Alternate. Anything else keeps the device
  space, as before. JPEG data and an Adobe CMYK /Decode are unchanged.
- Images in one document that carry the same profile share one profile
  object.

TIFF profiles are not read yet.

Co-authored-by: lloyd <lloyd@stackworx.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:14:51 -04:00
627d822709 create: lift TIFF decoding buffer limit to allow print-sized images (#665) (#681)
* create: lift TIFF decoding buffer limit to allow print-sized images (#665)

By default, the tiff crate limits decoded buffer size to 256 MiB.
Set Decoder::with_limits(Limits::unlimited()) to match PNG decoding
behavior and prevent 'decoder limits exceeded' on large images.

* create: cap the TIFF decode buffer at 2 GiB instead of lifting it entirely (#665)

A few bytes claiming enormous dimensions must still be refused, not allocated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:12:10 -04:00
f88a59f762 M10.2: Create PDF from multiple files, edited as pages in the Organize grid (#361)
* M10.2: create PDF from multiple files

Create a PDF ▸ Multiple files takes PDFs, images and .txt files, lets the
user reorder them, and makes one combined PDF (a bookmark and an optional
page range per file) or one PDF per file in a chosen folder, never
overwriting what is there.

- create: source_kind() tells PDFs, images and text apart (moved from the shell)
- engine: Session::convert_to_pdf, shared by Open, the dialog and the tool
- automation: doc_create_multiple (combine or separate)
- ui: Multiple files dialog reusing the Combine files list; translations
- parity entry and ROADMAP updated; catalog hashes refreshed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M10.2: multiple files open in the page grid; insert files at any gap

Create a PDF ▸ Multiple files now opens the picked files as one unsaved
document in the Organize page grid instead of a file-list dialog: pages
are removed and reordered there, and Save pages writes what is shown.

- grid: a "+" at every gap inserts files right there; Save pages button
- insert from file: several files in order, images and text converted,
  inserted pages selected
- the file-list dialog is removed; one PDF per file stays in the
  doc_create_multiple tool
- translations, catalog hashes, parity notes and ROADMAP updated

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M4: insert files by dropping them on the page grid

On desktop, PDFs, images and .txt files dropped on the Organize page grid
are inserted at the gap under the pointer, in order, and selected; a
dropped file still opens as a document everywhere else.

winit does not report the pointer while files are dragged over the
window on every platform, so a drop waits (up to a second) for the
pointer to show up and otherwise appends at the end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M4: files dragged over the page grid follow the pointer (macOS)

winit sends no pointer moves while files are dragged over the window, so
egui kept the place the pointer entered at: the insertion bar froze there
and the files were inserted there. During a file drag the pointer is now
read from AppKit (safe objc2 calls, crates already in the tree) and fed
to egui, so the bar and a "+" badge follow the pointer and the files go
where they are released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-08 17:42:13 -04:00
607d3d1c0e #190: choose DPI when creating PDFs from images (#206)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-07 23:14:22 -04:00
Brandon ThomasandClaude Opus 5.5 d3db99818f Rename PrintCraft to PdfCraft (#174)
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now
PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the
PrintCraft settings and crash-recovery folders to the new name, and both apps read settings
saved under the old storage key, so upgrades keep recent files, preferences and unsaved work.
The signing test fixtures keep their PrintCraft subject names (they are binary).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 16:03:57 -04:00
Brandon ThomasandClaude Opus 5.5 7632d8b179 Never crash: create has no expect calls
`add_page` returns the crate's error when the new document has no page tree, and text pages are
filled with `if let`. The crate now denies the panic lints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:12:21 -04:00
Brandon ThomasandClaude Opus 5.5 ab47bf4b9a CI: as_chunks instead of constant-size chunks_exact (clippy 1.99 chunks_exact_to_as_chunks)
Stable clippy 1.99 denies chunks_exact/chunks_exact_mut with a constant
size (32 sites), which failed the test job on main and every open PR.
Behaviour unchanged; MSRV 1.90 has as_chunks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:48:43 -04:00
Brandon ThomasandClaude Opus 5.5 c2f53d5595 M7: Edit text & images: select existing page images to move, resize, rotate, flip, replace, save or delete; page_images, image_edit, image_save tools
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 02:55:08 -04:00
Brandon ThomasandClaude Opus 5.5 9372b25d0b M10: Create PDF from JPEG 2000 images (JP2 or codestream, embedded as JPXDecode); M1: looping /Prev chains test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 08:07:53 -04:00
Brandon ThomasandClaude Opus 5.5 1e69a92c41 M10: Export a PDF ▸ Export all images: JPEGs unchanged, other images as PNG with alpha; skipped images reported; doc_export_all_images
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 04:47:45 -04:00
Brandon ThomasandClaude Opus 5.5 57980200e2 M7.4: Edit a PDF ▸ Add content: text (click and type; standard fonts, size, bold/italic, colour, alignment, wrapping) and images (PNG/JPEG/TIFF/GIF/BMP, centred at natural size) as page content that stays editable: each item is its own tagged content stream with its parameters, drawn in display space; select, move, resize (images keep proportions), double-click to retype, Format text panel, Delete; Bates numbering ▸ Add opens Header & Footer with a Bates number. content_list/page_add_text/page_add_image/content_update/content_delete tools (66 tools). Lucide align-center/right icons
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 00:58:03 -04:00
Brandon ThomasandClaude Opus 5.5 81d0af6c7f M10.3/M10.4: Create from TIFF (every page; 1-bit scans stay 1-bit; resolution from the file), GIF and BMP; opening those converts them. Export pages as JPEG (quality) and TIFF besides PNG, in the dialog and doc_export_images
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 00:46:58 -04:00
Brandon ThomasandClaude Opus 5.5 adb644c5e1 M10.2/M11.1: Create a PDF (blank, from PNG/JPEG images at their resolution with transparency, from plain text) in a new create crate (L4); opening an image or text file converts it to a new unsaved PDF; Reduce file size (resource merge across the document, GC, object streams) writes a compact copy; Create group Ready; doc_create/doc_reduce tools (49 tools). Fix: the Hand tool did not pan (the page widget took the drag)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 23:29:17 -04:00