M2.6: say what zlib-rs reads differently, and test which backend inflates

The Consumers review of the commit before this one: zlib-rs refuses a back-reference to before
the start of the output, which miniz_oxide filled with zeros and went on
from. Read strictly such a stream is now refused, and read tolerantly what
came before it is kept, as zlib has it. flate.rs's quirks and the filters
README say so, and a_copy_from_before_the_start_is_damage checks it, which
also fails under miniz_oxide, so it shows which backend runs: the read price
is set for zlib-rs.

The same review: other dependencies still build flate2's default miniz_oxide
backend, and zlib-rs runs because flate2 prefers it when both are built, not
because it is the only one. The Cargo.toml comment says so. The miniz_oxide
measured was 0.9.1, flate2's (0.8.9 is png's); FLATE_READ_WORK's doc says so.
The filters README named miniz_oxide as the backend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

Cargo.toml

@@ -64,8 +64,10 @@ pdfcraft-automation = { path = "crates/automation", version = "0.17.0", default-
 hayro = "0.7.1"
 hayro-syntax = "0.7.2"
 lopdf = "0.45"
-# zlib-rs, which the Windows build already got through craft-update: miniz_oxide spends about
-# 1.7 µs on every deflate block, empty ones included (`pdfcraft_filters::FLATE_READ_WORK`).
+# zlib-rs: miniz_oxide spends about 1.7 µs on every deflate block, empty ones included
+# (`pdfcraft_filters::FLATE_READ_WORK`). Other dependencies still build flate2's default
+# miniz_oxide backend; flate2 prefers zlib-rs when both are built, as the Windows build already did
+# through craft-update. `flate::tests::a_copy_from_before_the_start_is_damage` shows which runs.
 flate2 = { version = "1.1", default-features = false, features = ["zlib-rs"] }
 weezl = "0.2"
 # CCITT Group 4 encoding (pdfcraft-filters); already in the tree through tiff.

crates/filters/README.md

@@ -3,7 +3,7 @@
 PDF stream filters: the decoders and encoders behind `/Filter` and `/DecodeParms`.
 
 - **Layer:** L0, standalone. No dependency on any other PdfCraft crate. `cos` depends on it (architecture §3 rule 3).
-- **Licence:** MIT OR Apache-2.0. Clean-room: written from ISO 32000-2 §7.4, RFC 1950/1951 (through `flate2` with the pure-Rust `miniz_oxide` backend), TIFF 6.0 §14 (predictor 2) and RFC 2083 §6 (PNG filters). CCITT Group 4 encoding (ITU-T T.6) is the `fax` crate's (MIT, already in the tree through `tiff`), checked here by round trips through an independent decoder; CCITT decoding (T.4 and T.6) is the `hayro-ccitt` crate's, and JBIG2 decoding the `hayro-jbig2` crate's (both MIT OR Apache-2.0, already in the tree through hayro), bounded here. JBIG2 generic region encoding is written from the public ISO/IEC 14492 Final Committee Draft (WG1 N1359, 1999); Annex H.2 checks the arithmetic coder byte for byte.
+- **Licence:** MIT OR Apache-2.0. Clean-room: written from ISO 32000-2 §7.4, RFC 1950/1951 (through `flate2` with the pure-Rust `zlib-rs` backend), TIFF 6.0 §14 (predictor 2) and RFC 2083 §6 (PNG filters). CCITT Group 4 encoding (ITU-T T.6) is the `fax` crate's (MIT, already in the tree through `tiff`), checked here by round trips through an independent decoder; CCITT decoding (T.4 and T.6) is the `hayro-ccitt` crate's, and JBIG2 decoding the `hayro-jbig2` crate's (both MIT OR Apache-2.0, already in the tree through hayro), bounded here. JBIG2 generic region encoding is written from the public ISO/IEC 14492 Final Committee Draft (WG1 N1359, 1999); Annex H.2 checks the arithmetic coder byte for byte.
 
 - `#![forbid(unsafe_code)]`. Builds for `wasm32-unknown-unknown`.
 
@@ -37,14 +37,14 @@ encode_jbig2(data, columns, rows) -> Result<Vec<u8>, FilterError>   // 1-bit row
 - Image codecs (DCT, JPX, JBIG2, CCITT), `Crypt` and unknown filters anywhere in a chain give `Unsupported`: callers keep such data encoded. So does a chain longer than `MAX_CHAIN` (8) filters, before any stage runs. Decryption belongs to `crypt`.
 - `max_output` bounds every intermediate buffer (including the pre-predictor buffer) and gives `LimitExceeded(max_output)`. This also holds when Flate retries raw deflate after a misleading zlib header, in both strict and tolerant decoding.
 - `decode_counted` and `decode_tolerant_counted` are for a caller charging a budget (`pdfcraft_cos::DecodeBudget`). `max_output` bounds all stages together (each stage gets what the earlier ones left), and they give the bytes every stage that ran decoded, on success and on failure. A stage's output counts as its decoder produced it, before a predictor or a later stage shrinks it, and a failed decode pays for what it decoded first.
-- `decode_tolerant_within` is for a run of images (`pdfcraft_cos::ImageBudget`), as `decode_within` (its strict form), `decode_jbig2_within` and `decode_ccitt_within` are, all spending from one `work`, in the JBIG2 decoder's steps. `max_output` bounds each stage, as in `decode_tolerant`, and no stage decodes past what `work` has left. It pays a step a byte of `data` first and a step a byte each stage decodes, so a chain that fails has paid for what it did: 0.2 to 3 ns a byte in or out, up to 19 ns through a predictor on samples of a bit, against the 6 to 16 ns of a JBIG2 step. A Flate stage also pays `FLATE_READ_WORK` (2) steps a byte it reads, before it runs, and again if it reads them a second time as raw deflate: inflating does work for every block it reads, and a stream of empty dynamic blocks took 21 ns a byte to make nothing, about 7 ns a step at that price. Inflating is flate2's zlib-rs backend on every platform; miniz_oxide, before, took 1.4 µs a byte on empty fixed blocks. What `work` can't pay for spends all of it, and gives `Spent`; with nothing left nothing is read. A chain that can't be decoded at all costs nothing and gives its usual error.
+- `decode_tolerant_within` is for a run of images (`pdfcraft_cos::ImageBudget`), as `decode_within` (its strict form), `decode_jbig2_within` and `decode_ccitt_within` are, all spending from one `work`, in the JBIG2 decoder's steps. `max_output` bounds each stage, as in `decode_tolerant`, and no stage decodes past what `work` has left. It pays a step a byte of `data` first and a step a byte each stage decodes, so a chain that fails has paid for what it did: 0.2 to 3 ns a byte in or out, up to 19 ns through a predictor on samples of a bit, against the 6 to 16 ns of a JBIG2 step. A Flate stage also pays `FLATE_READ_WORK` (2) steps a byte it reads, before it runs, and again if it reads them a second time as raw deflate: inflating does work for every block it reads, and a stream of empty dynamic blocks took 21 ns a byte to make nothing, about 7 ns a step at that price. flate2 inflates with its zlib-rs backend, which it prefers to miniz_oxide when other dependencies build that too; miniz_oxide, its backend here before, took 1.4 µs a byte on empty fixed blocks. What `work` can't pay for spends all of it, and gives `Spent`; with nothing left nothing is read. A chain that can't be decoded at all costs nothing and gives its usual error.
 - `decode_tolerant` turns `Corrupt` into "what was decoded so far" plus `partial = true`, then runs the rest of the chain on it (predictors are applied to partial output too). `LimitExceeded` and `Unsupported` are still errors.
 
 ## Behaviour
 
 | Filter | Decode | Encode |
 |---|---|---|
-| FlateDecode | zlib header checked, then inflated raw: a missing or wrong Adler-32 is tolerated; no valid header → raw deflate; trailing bytes ignored; preset dictionary → Corrupt; truncated → Corrupt (partial in tolerant mode) | zlib level 6 |
+| FlateDecode | zlib header checked, then inflated raw: a missing or wrong Adler-32 is tolerated; no valid header → raw deflate; trailing bytes ignored; preset dictionary → Corrupt; truncated, or a back-reference to before the start of the output → Corrupt (partial in tolerant mode) | zlib level 6 |
 | LZWDecode | 9–12-bit MSB-first codes, clear (256), EOD (257), `EarlyChange` 0/1; missing EOD/initial clear tolerated; a full table stops growing | greedy, leading clear, clear-table when the table fills, EOD |
 | Predictors (Flate/LZW) | 2 = TIFF (1/2/4/8/16 bpc), 10–15 = PNG with per-row tags 0–4; partial final rows decoded; bad params or tags → Corrupt | 2, 10 None, 11 Sub, 12 Up, 13 Average, 14 Paeth, 15 per-row optimum (min sum of signed residuals) |
 | ASCIIHexDecode | white space ignored, odd digit padded with 0, `>` EOD (optional), other characters → Corrupt | upper case, newline every 64 digits, `>` |

crates/filters/src/flate.rs

@@ -5,6 +5,10 @@
 //!   deflate stream is inflated raw after the 2-byte zlib header;
 //! - streams without a valid zlib header are inflated as raw deflate;
 //! - data after the end of the deflate stream is ignored.
+//!
+//! A back-reference to before the start of the output points at nothing, and is damage, as zlib
+//! has it: read strictly the stream is refused, and read tolerantly what came before it is kept.
+//! (miniz_oxide, flate2's backend here before zlib-rs, copied zeros and went on.)
 
 use std::io::Write;
 
@@ -195,4 +199,67 @@ mod tests {
     fn dec_limit(d: &[u8], max: usize) -> Result<Vec<u8>, FilterError> {
         decode(d, max, Some(max)).0.map_err(|f| f.error)
     }
+
+    /// Deflate's bits: header fields and extra bits least significant first, Huffman codes most
+    /// significant first (RFC 1951 §3.1.1).
+    #[derive(Default)]
+    struct Bits {
+        out: Vec<u8>,
+        acc: u32,
+        n: u32,
+    }
+
+    impl Bits {
+        fn put(&mut self, value: u32, len: u32) {
+            for i in 0..len {
+                self.acc |= ((value >> i) & 1) << self.n;
+                self.n += 1;
+                if self.n == 8 {
+                    self.out.push(self.acc as u8);
+                    (self.acc, self.n) = (0, 0);
+                }
+            }
+        }
+
+        fn code(&mut self, code: u32, len: u32) {
+            for i in (0..len).rev() {
+                self.put((code >> i) & 1, 1);
+            }
+        }
+
+        fn finish(mut self) -> Vec<u8> {
+            if self.n > 0 {
+                self.out.push(self.acc as u8);
+            }
+            self.out
+        }
+    }
+
+    /// A back-reference to before the start of the output points at nothing: read strictly the
+    /// stream is refused, and what came before it is what a tolerant reading keeps. miniz_oxide,
+    /// flate2's backend here before zlib-rs, copied zeros and went on, so this also shows which
+    /// backend inflates (the price of reading Flate, `FLATE_READ_WORK`, is set for zlib-rs).
+    #[test]
+    fn a_copy_from_before_the_start_is_damage() {
+        let literal = |bits: &mut Bits, b: u8| bits.code(0x30 + u32::from(b), 8);
+        let mut bits = Bits::default();
+        bits.put(1, 1); // the final block,
+        bits.put(1, 2); // fixed Huffman codes (§3.2.6)
+        literal(&mut bits, b'A');
+        literal(&mut bits, b'B');
+        bits.code(1, 7); // length 3
+        bits.code(4, 5); // distance 5 (code 4 and an extra bit of 0), three bytes before the start
+        bits.put(0, 1);
+        literal(&mut bits, b'C');
+        bits.code(0, 7); // end of block
+        let data = bits.finish();
+        assert!(zlib_header(&data).is_none(), "read as raw deflate");
+        match decode(&data, 1 << 20, Some(1 << 20)).0 {
+            Err(f) => {
+                assert!(matches!(f.error, FilterError::Corrupt { .. }), "{}", f.error);
+                assert_eq!(f.partial, b"AB");
+            }
+            Ok(out) => panic!("read as {out:?}"),
+        }
+    }
 }

crates/filters/src/lib.rs

@@ -264,7 +264,7 @@ pub const MAX_CHAIN: usize = 8;
 /// 0.6.8, release build) the slowest blocks measured, empty dynamic ones, took 21 ns a byte read
 /// to make nothing, empty fixed ones 9 and fixed ones of a literal each 6, so at this price the
 /// worst comes to about 7 ns a step, inside what a step of JBIG2 decoding takes. (miniz_oxide
-/// 0.8.9, flate2's backend here before, builds a block's tables for every block, and took 1,383 ns
+/// 0.9.1, flate2's backend here before, builds a block's tables for every block, and took 1,383 ns
 /// a byte read on empty fixed blocks.) Data a real encoder makes took 0.2 to 2.3 ns a byte made, its
 /// reading included, so an image pays twice its stored length more than it would unpriced.
 pub const FLATE_READ_WORK: u64 = 2;