Add craft-zip: bounded ZIP reading and a deterministic writer for the package crates (#941)

* craft-zip: ZIP preflight, bounded entry read and Budget (from craft-archive, #129)

The ZIP half of the craft-archive proposal (#126, #129): an allocation-free check of the end
records and central directory that runs before a ZIP reader indexes the archive, a bounded
entry read that distrusts the declared size, and the named Budget both charge. The XML half
of #129 is not carried: craft-xml bounds XML now.

* craft-zip: Archive (entry-count guard, duplicate names, budgeted reads) and Writer

check_entry_count is craft-opc's guard: the declared entry count, ZIP64 included, checked on
the raw bytes before zip parses the central directory. Archive reads entries within a per-entry
limit and a total Budget through read_entry. Duplicate names are found by counting the central
directory headers against the entries zip kept. Writer is craft-opc's ZipOut: stored or deflated
entries, ZIP64 for large ones, one fixed timestamp.

---------

Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
This commit is contained in:
Dylan M. Taylor
2026-11-11 18:53:53 -05:00
committed by GitHub
co-authored by skylar taylor-barrick
parent 77591e5efb
commit 17e55c96db
13 changed files with 1894 additions and 0 deletions
Generated
+10
View File
@@ -1749,6 +1749,16 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "craft-zip"
version = "0.1.0"
dependencies = [
"craft-zip",
"flate2",
"thiserror 2.0.21",
"zip",
]
[[package]]
name = "crc32fast"
version = "1.5.2"
+1
View File
@@ -99,6 +99,7 @@ craft-vectorcraft-format = { path = "crates/vectorcraft-format" }
craft-vectordrawable = { path = "crates/vectordrawable" }
craft-xls = { path = "crates/xls" }
craft-xml = { path = "crates/xml" }
craft-zip = { path = "crates/zip" }
craft-y4m = { path = "crates/y4m" }
craft-epub = { path = "crates/epub" }
craft-video = { path = "crates/video" }
+1
View File
@@ -276,6 +276,7 @@ craft-<name> = { git = "https://github.com/storytold/craft-libs", rev = "<commit
loading the file, and plane access. Bounded on hostile input.
- [`craft-kra`](crates/kra/README.md): Krita documents (`.kra`, `.krz`), read: the ZIP archive, the `maindoc.xml` node tree as written, and the tiled pixel files with their LZF compression, a tile at a time. From PhotoCraft.
- [`craft-xml`](crates/xml/README.md): bounded XML for hostile input: an owned element tree with limits on size, depth, nodes and attributes, streaming of repeated elements, namespace handling (as written, by local name, or canonical prefixes by URI), escaping and a writer. No DTDs and no entity expansion.
- [`craft-zip`](crates/zip/README.md): ZIP archives from untrusted files: an entry-count check before the archive is indexed (ZIP64 included), entry reads within per-entry and total budgets that count real bytes, duplicate-name detection, a strict structure preflight, and a deterministic stored/deflate writer. The shared ZIP layer for the package formats.
- [Shared UI component audit](docs/ui-components-audit.md): source inventory, extraction priorities
and compatibility boundaries across the Craft suite.
- [Theme infrastructure](docs/theme-infrastructure.md): eight-app comparison and the small shared
+26
View File
@@ -0,0 +1,26 @@
[package]
name = "craft-zip"
description = "ZIP archives from untrusted files: a preflight of the end records and central directory before anything is indexed, entry reads that don't trust declared sizes, duplicate-name detection, and a deterministic writer."
version.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
repository.workspace = true
[lints]
workspace = true
[dependencies]
thiserror = { workspace = true }
# `deflate-flate2` with flate2's default miniz_oxide backend, as craft-update pins it (zip's own
# `deflate` feature would switch a whole app's flate2 to zlib-rs).
zip = { version = "9", default-features = false, features = ["deflate-flate2"] }
# The backend of zip's deflate: `deflate-flate2` leaves the choice to the build.
flate2 = { version = "1", default-features = false, features = ["rust_backend"] }
[features]
# `zip64_entry_count_sample`, the regression input for a reader's own entry-count test.
test-support = []
[dev-dependencies]
craft-zip = { path = ".", features = ["test-support"] }
+68
View File
@@ -0,0 +1,68 @@
# craft-zip
ZIP archives from untrusted files, read within limits and written. No `unsafe`.
The [`zip`](https://crates.io/crates/zip) crate does the decoding. This crate is what every
reader of a ZIP-based format (OOXML, OpenDocument, EPUB, IDML, design-tool archives) otherwise
writes again around it. Written from the ZIP specification (PKWARE APPNOTE.TXT).
Every limit is the caller's number: the crate has no defaults. Entry-name rules (which names
are safe, which spellings are one part) stay with the format that knows them.
## Reading
```rust
use craft_zip::{Archive, Compression, Limits, Writer};
# let mut w = Writer::new();
# w.add("mimetype", b"application/epub+zip", Compression::Stored)?;
# w.add("content.xml", b"<a/>", Compression::Deflated)?;
# let bytes = w.finish()?;
let mut zip = Archive::open(&bytes, Limits { max_entries: 20_000, max_total_bytes: 512 << 20 })?;
if zip.has_duplicate_names() {
// Two entries share a name; a format that must not guess which is real refuses here.
}
let part = zip.read("content.xml", 64 << 20)?; // Ok(None) when there is no such entry
assert_eq!(part.as_deref(), Some(&b"<a/>"[..]));
assert_eq!(zip.budget().used(), 4);
# Ok::<(), craft_zip::Error>(())
```
- `Archive::open` runs `check_entry_count` first. `zip` reserves room for, and parses, as many
central directory headers as the end record declares before its caller can count them, and a
ZIP64 end record can declare millions. The check finds the end records the way `zip` does and
refuses on the raw bytes.
- `Archive::read` refuses an entry that declares more than `max`, that expands past what it
declares, or that does not fit in what is left of `max_total_bytes`. At most 64 KiB is
reserved before bytes arrive, whatever the entry declares. Only a read that succeeds is
charged.
- `has_duplicate_names`: `zip` keeps one entry per name, so a central directory with more
headers than `zip` has entries repeats a name.
- `budget_mut` is for a caller with its own charging rule (charging a failed read, say).
- `Error::is_limit` tells a limit from a broken archive.
## Writing
`Writer::add(name, data, Compression::Stored | Deflated | Level(n))`, then `finish()`. Entries
keep their order (a stored `mimetype` goes first by being added first). Every entry has the same
1980 timestamp, so equal input gives equal bytes. An entry of 2 GiB or more is written in the
ZIP64 form. A repeated name is an error.
## Strict preflight
`preflight::preflight(bytes, limits)` checks the whole structure on the raw bytes without
allocating: exactly one end record may span the end of the file, the ZIP64 pair must agree with
it, the directory must sit exactly before the end records and be filled exactly by its entries,
single disk only, with limits on entries, directory bytes and name length. It is stricter than
`zip` (it refuses archives with data before them and archives with two plausible end records),
so `Archive::open` does not run it: a caller that wants nothing ambiguous to reach a ZIP reader
calls it first.
`preflight::read_entry(reader, declared, max, &mut budget)` is the bounded read on any
`std::io::Read`; `Budget` is a named limit with a running total.
## Where it came from
`preflight`, `read_entry` and `Budget`: the ZIP half of the craft-archive proposal (#126, #129).
`check_entry_count`: craft-opc's `guard`. Duplicate detection: craft-penpot's rule, made exact
by walking the directory. The writer: craft-opc's `ZipOut`.
+118
View File
@@ -0,0 +1,118 @@
//! An archive open for reading: entry-count guard, duplicate names, budgeted entry reads.
use std::borrow::Cow;
use std::io::Cursor;
use crate::preflight::{ReadError, read_entry};
use crate::{Budget, Error, check_entry_count};
/// What an [`Archive`] refuses to exceed. Both numbers are the caller's.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Limits {
/// Most entries the archive may declare.
pub max_entries: u64,
/// Most bytes all the entries read from the archive may expand to together.
pub max_total_bytes: u64,
}
/// A ZIP archive in memory. Nothing is decompressed until an entry is read.
pub struct Archive<'a> {
zip: zip::ZipArchive<Cursor<&'a [u8]>>,
budget: Budget,
duplicates: bool,
}
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
const CENTRAL_HEADER_LEN: usize = 46;
/// The central directory headers that follow one another from `start`, counted up to `stop`.
fn count_central_headers(bytes: &[u8], start: u64, stop: usize) -> usize {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| usize::from(u16::from_le_bytes([b[0], b[1]])));
let Ok(mut at) = usize::try_from(start) else { return 0 };
let mut count = 0;
while count < stop && bytes.get(at..).is_some_and(|b| b.starts_with(CENTRAL_HEADER)) {
let (Some(name), Some(extra), Some(comment)) = (u16_at(at + 28), u16_at(at + 30), u16_at(at + 32)) else { break };
count += 1;
at = at.saturating_add(CENTRAL_HEADER_LEN + name + extra + comment);
}
count
}
impl<'a> Archive<'a> {
/// Indexes `bytes`. The declared entry count is checked against
/// [`Limits::max_entries`] before `zip` parses the central directory, and again after.
pub fn open(bytes: &'a [u8], limits: Limits) -> Result<Self, Error> {
check_entry_count(bytes, limits.max_entries)?;
let zip = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| Error::NotZip(e.to_string()))?;
let found = zip.len() as u64;
if found > limits.max_entries {
return Err(Error::TooManyEntries { declared: found, limit: limits.max_entries });
}
// `zip` keeps one entry per name, so a directory with more headers than `zip` has entries
// repeats a name.
let duplicates = count_central_headers(bytes, zip.central_directory_start(), zip.len().saturating_add(1)) > zip.len();
Ok(Archive { zip, budget: Budget::new("archive", limits.max_total_bytes), duplicates })
}
/// The number of distinct entry names.
pub fn len(&self) -> usize {
self.zip.len()
}
pub fn is_empty(&self) -> bool {
self.zip.is_empty()
}
/// Do two entries share a name? `zip` then keeps one of them, and which one a writer meant
/// is not known: a format that must not guess refuses the archive.
pub fn has_duplicate_names(&self) -> bool {
self.duplicates
}
/// The entry names in the archive's order, folders included. A name that is not text is
/// left out.
pub fn names(&self) -> impl Iterator<Item = Cow<'_, str>> {
self.zip.file_names().filter_map(Result::ok)
}
/// Is there an entry `name` (exactly as stored)?
pub fn contains(&self, name: &str) -> bool {
self.zip.index_for_name(name).is_some()
}
/// The size entry `name` declares. It is a claim, not a measurement.
pub fn declared_size(&mut self, name: &str) -> Option<u64> {
let index = self.zip.index_for_name(name)?;
self.zip.by_index_raw(index).ok().map(|f| f.size())
}
/// The bytes of entry `name`, or `None` when there is no such entry.
///
/// The entry may declare, and expand to, at most `max` bytes, and at most what is left of
/// [`Limits::max_total_bytes`]. An entry that expands past the size it declares is refused.
/// Only the bytes of a read that succeeds are charged to the total.
pub fn read(&mut self, name: &str, max: u64) -> Result<Option<Vec<u8>>, Error> {
let Some(index) = self.zip.index_for_name(name) else { return Ok(None) };
let entry = |msg: String| Error::Entry { name: name.to_string(), msg };
let f = self.zip.by_index(index).map_err(|e| entry(e.to_string()))?;
let declared = f.size();
match read_entry(f, declared, max, &mut self.budget) {
Ok(data) => Ok(Some(data)),
Err(ReadError::DeclaredTooLarge { limit, .. }) | Err(ReadError::Overrun { limit }) => {
Err(Error::EntryTooLarge { name: name.to_string(), limit })
}
Err(ReadError::Budget(e)) => Err(Error::BudgetSpent { name: name.to_string(), limit: e.limit }),
Err(ReadError::Io(e)) => Err(entry(e.to_string())),
}
}
/// The total budget: what the entries read so far have used, and what is left.
pub fn budget(&self) -> &Budget {
&self.budget
}
/// The total budget, for a caller with its own charging rule (charging a failed read, say).
pub fn budget_mut(&mut self) -> &mut Budget {
&mut self.budget
}
}
+192
View File
@@ -0,0 +1,192 @@
//! A bytes-only check of a ZIP archive's declared entry count before the `zip` crate opens it.
//!
//! `zip::ZipArchive::new` reserves room for, and parses, as many central directory headers as the
//! end record declares before its caller can look at `len()`. A ZIP64 end record can declare
//! millions of entries for a file of a few hundred megabytes (only 46 bytes each have to fit), and
//! each parsed entry costs several times that in memory. Here the end records are found the way
//! `zip` finds them (APPNOTE.TXT §4.3.14 to §4.3.16), and a package declaring more than the
//! reader's limit is refused before `zip` parses anything. Anything else that is wrong is left for
//! `zip` to report.
use crate::Error;
const EOCD: &[u8; 4] = b"PK\x05\x06";
const EOCD64_LOCATOR: &[u8; 4] = b"PK\x06\x07";
const EOCD64: &[u8; 4] = b"PK\x06\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// End records looked at, at most. A package has one; a few more can turn up inside stored data.
const MAX_END_RECORDS: usize = 16;
/// Refuse `bytes` when the end record `zip` would use declares more than `max_entries` entries.
///
/// `zip` searches the file backwards for an end of central directory record. A ZIP64 one points
/// through its locator at the ZIP64 end record, which `zip` looks for from the locator's offset
/// on: every such record there is checked, and as `zip` may fall back to an earlier end record
/// when one doesn't parse, the search goes on. A plain one is used when it declares no entries, or
/// a central directory header lies between its directory offset and itself: it is the last one
/// looked at.
pub fn check_entry_count(bytes: &[u8], max_entries: u64) -> Result<(), Error> {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]));
let u32_at = |at: usize| bytes.get(at..at.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]));
let u64_at = |at: usize| bytes.get(at..at.checked_add(8)?).and_then(|b| <[u8; 8]>::try_from(b).ok()).map(u64::from_le_bytes);
let refuse =
|n: u64| -> Result<(), Error> { if n > max_entries { Err(Error::TooManyEntries { declared: n, limit: max_entries }) } else { Ok(()) } };
let mut end = bytes.len();
let mut seen = 0;
while let Some(at) = bytes.get(..end).and_then(|b| b.windows(4).rposition(|w| w == EOCD)) {
end = at;
seen += 1;
if seen > MAX_END_RECORDS {
return Err(Error::TooManyEndRecords(MAX_END_RECORDS));
}
// `zip` skips a record whose comment runs past the end of the file.
let Some(comment) = u16_at(at + 20) else { continue };
if at + 22 + usize::from(comment) > bytes.len() {
continue;
}
let (Some(on_disk), Some(total), Some(cd_offset)) = (u16_at(at + 8), u16_at(at + 10), u32_at(at + 16)) else { continue };
let zip64 = total == u16::MAX || cd_offset == u32::MAX;
if zip64
&& let Some(locator) = at.checked_sub(20)
&& bytes.get(locator..locator + 4) == Some(EOCD64_LOCATOR.as_slice())
{
let (Some(eocd64), Some(disks)) = (u64_at(locator + 8), u32_at(locator + 16)) else { continue };
let Some(from) = usize::try_from(eocd64).ok().filter(|&f| f < locator) else { continue };
if disks > 1 {
continue;
}
let records = bytes.get(from..locator).unwrap_or_default().windows(4).enumerate().filter(|(_, w)| *w == EOCD64);
for (i, _) in records {
let p = from + i;
refuse(u64_at(p + 24).unwrap_or(0).max(u64_at(p + 32).unwrap_or(0)))?;
}
continue;
}
if total == 0 {
return Ok(());
}
let cd = cd_offset as usize;
if cd < at && bytes.get(cd..at).is_some_and(|b| b.windows(4).any(|w| w == CENTRAL_HEADER)) {
return refuse(u64::from(on_disk.max(total)));
}
}
Ok(())
}
/// A package of one empty stored file, `a`, whose central directory lists it `entries` times (at
/// most as many as fit in 64 MiB) under a ZIP64 end record that declares them all: `zip`
/// parses and keeps every one of them before its caller can count them. Regression input for the
/// readers' tests: built only for tests, this crate's and (through the `test-support` feature,
/// which they turn on as a dev-dependency) the readers', never into an app.
#[cfg(any(test, feature = "test-support"))]
pub fn zip64_entry_count_sample(entries: u64) -> Vec<u8> {
// Local file header (APPNOTE.TXT §4.3.7): version 10, stored, no data, name "a".
let mut f = b"PK\x03\x04".to_vec();
f.extend_from_slice(&[10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // version, flags, method, time, date
f.extend_from_slice(&[0; 12]); // crc, sizes
f.extend_from_slice(&[1, 0, 0, 0]); // name and extra lengths
f.push(b'a');
let cd_at = f.len() as u64;
// Its central header (§4.3.12), 47 bytes.
let mut header = CENTRAL_HEADER.to_vec();
header.extend_from_slice(&[10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // versions, flags, method, time, date
header.extend_from_slice(&[0; 12]); // crc, sizes
header.extend_from_slice(&[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); // name, extra, comment, disk, attributes
header.extend_from_slice(&[0; 4]); // local header offset
header.push(b'a');
let count = entries.min((64 << 20) / header.len() as u64);
for _ in 0..count {
f.extend_from_slice(&header);
}
let cd_len = f.len() as u64 - cd_at;
let eocd64_at = f.len() as u64;
// ZIP64 end of central directory record (§4.3.14), 56 bytes.
f.extend_from_slice(EOCD64);
f.extend_from_slice(&44u64.to_le_bytes()); // size of the rest of the record
f.extend_from_slice(&[45, 0, 45, 0]); // versions
f.extend_from_slice(&[0; 8]); // this disk, disk with the central directory
f.extend_from_slice(&entries.to_le_bytes()); // entries on this disk
f.extend_from_slice(&entries.to_le_bytes()); // entries
f.extend_from_slice(&cd_len.to_le_bytes());
f.extend_from_slice(&cd_at.to_le_bytes());
// Its locator (§4.3.15), 20 bytes.
f.extend_from_slice(EOCD64_LOCATOR);
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&eocd64_at.to_le_bytes());
f.extend_from_slice(&1u32.to_le_bytes());
// End of central directory record (§4.3.16), 22 bytes, each count and offset "see ZIP64".
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0; 4]);
f.extend_from_slice(&[0xFF; 12]);
f.extend_from_slice(&[0; 2]);
f
}
#[cfg(test)]
mod tests {
use super::{CENTRAL_HEADER, EOCD, check_entry_count as check, zip64_entry_count_sample};
use crate::Error;
/// A plain package's tail: one central header at 0, then an end record declaring `entries`.
fn plain(entries: u16, comment: &[u8]) -> Vec<u8> {
let mut f = CENTRAL_HEADER.to_vec();
f.resize(46, 0);
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&entries.to_le_bytes());
f.extend_from_slice(&entries.to_le_bytes());
f.extend_from_slice(&46u32.to_le_bytes());
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&(comment.len() as u16).to_le_bytes());
f.extend_from_slice(comment);
f
}
#[test]
fn a_zip64_end_record_declaring_too_many_entries_is_refused() {
// `zip` would reserve and parse them all before the reader's cap.
let sample = zip64_entry_count_sample(20_001);
assert_eq!(check(&sample, 20_000), Err(Error::TooManyEntries { declared: 20_001, limit: 20_000 }));
assert!(check(&zip64_entry_count_sample(5), 20_000).is_ok());
// Junk before the package (a self-extracting stub) doesn't hide it.
let mut prefixed = vec![0x55u8; 4096];
prefixed.extend_from_slice(&sample);
assert!(check(&prefixed, 20_000).is_err());
}
#[test]
fn a_plain_end_record_is_checked_as_zip_reads_it() {
assert!(check(&plain(3, b""), 20_000).is_ok());
assert_eq!(check(&plain(30_000, b"a comment"), 20_000), Err(Error::TooManyEntries { declared: 30_000, limit: 20_000 }));
// A record whose comment runs past the end is not the one `zip` uses.
let mut cut = plain(30_000, b"");
cut.truncate(cut.len() - 2);
cut.extend_from_slice(&500u16.to_le_bytes());
assert!(check(&cut, 20_000).is_ok());
assert!(check(b"not a package", 20_000).is_ok());
assert!(check(b"", 20_000).is_ok());
}
#[test]
fn a_fake_end_record_after_the_real_one_does_not_hide_it() {
// `zip` skips a last record with no central header before it and uses the one before.
let mut f = zip64_entry_count_sample(1_000_000);
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&7u16.to_le_bytes());
f.extend_from_slice(&7u16.to_le_bytes());
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&((f.len() - 12) as u32).to_le_bytes());
f.extend_from_slice(&0u16.to_le_bytes());
assert!(check(&f, 20_000).is_err());
}
#[test]
fn many_end_records_are_refused() {
// Each declares one entry and has no central header before it, so `zip` would try them all.
let record = [0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0];
let f: Vec<u8> = (0..20).flat_map(|_| EOCD.iter().copied().chain(record)).collect();
assert_eq!(check(&f, 20_000), Err(Error::TooManyEndRecords(16)));
}
}
+187
View File
@@ -0,0 +1,187 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
//! ZIP archives from untrusted files, read within limits and written.
//!
//! The `zip` crate does the decoding. This crate is what every reader of a ZIP-based format
//! otherwise writes again around it:
//!
//! - [`Archive::open`] refuses an archive that declares more entries than the caller allows
//! before `zip` parses them ([`check_entry_count`]), and tells whether two entries share a name.
//! - [`Archive::read`] reads an entry within a per-entry limit and the archive's total
//! [`Budget`], counting the bytes that really arrive, not the size the archive declares.
//! - [`Writer`] writes stored and deflated entries, ZIP64 where an entry needs it, with the same
//! timestamp on every entry so equal input gives equal bytes.
//! - [`preflight`] is a stricter check of the whole end-record and central-directory structure
//! on the raw bytes, for a caller that wants nothing ambiguous to reach a ZIP reader at all.
//!
//! Every limit is the caller's number. Entry-name rules (which names are safe, which spellings
//! are one part) stay with the format that knows them.
#[cfg(doctest)]
#[doc = include_str!("../README.md")]
struct ReadmeExamples;
mod archive;
mod guard;
pub mod preflight;
mod write;
pub use archive::{Archive, Limits};
pub use guard::check_entry_count;
#[cfg(any(test, feature = "test-support"))]
pub use guard::zip64_entry_count_sample;
pub use write::{Compression, Writer};
/// Why an archive was not read or not written.
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
/// The bytes are not a ZIP archive the `zip` crate can index.
#[error("not a ZIP archive: {0}")]
NotZip(String),
/// The archive declares more entries than [`Limits::max_entries`].
#[error("the archive declares {declared} entries, more than {limit}")]
TooManyEntries { declared: u64, limit: u64 },
/// More end of central directory records than an archive can have.
#[error("more than {0} end of central directory records")]
TooManyEndRecords(usize),
/// An entry declares, or expands to, more than the limit given for it.
#[error("{name} is larger than {limit} bytes")]
EntryTooLarge { name: String, limit: u64 },
/// The entries read so far have used up [`Limits::max_total_bytes`].
#[error("{name} does not fit in what is left of the {limit} bytes the archive may expand to")]
BudgetSpent { name: String, limit: u64 },
/// An entry could not be decoded (a failed checksum, an unknown method, encryption).
#[error("{name}: {msg}")]
Entry { name: String, msg: String },
/// The archive could not be written.
#[error("the ZIP archive could not be written: {0}")]
Write(String),
}
impl Error {
/// Was a limit exceeded (as opposed to the archive being broken)?
pub fn is_limit(&self) -> bool {
matches!(self, Error::TooManyEntries { .. } | Error::TooManyEndRecords(_) | Error::EntryTooLarge { .. } | Error::BudgetSpent { .. })
}
}
use std::fmt;
/// A named limit and the total charged against it so far.
///
/// The name is what a breach message should mention ("document", "part", "images"), and the limit
/// is the application's own number. The crate never invents either, so one budget can cover a
/// whole import and be reported in the application's terms.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Budget {
what: &'static str,
limit: u64,
used: u64,
}
impl Budget {
/// A budget named `what` that admits up to `limit` units in total.
pub const fn new(what: &'static str, limit: u64) -> Self {
Self { what, limit, used: 0 }
}
/// Charge `n` units, or refuse without changing the total.
///
/// A refused charge leaves the total untouched, so a caller may carry on with a smaller
/// charge, and the amount refused is reported in [`Exceeded::requested`].
pub fn charge(&mut self, n: u64) -> Result<(), Exceeded> {
if n > self.remaining() {
return Err(Exceeded { what: self.what, limit: self.limit, requested: n });
}
self.used = self.used.saturating_add(n);
Ok(())
}
/// The units still available; zero when the budget is exhausted.
pub const fn remaining(&self) -> u64 {
self.limit.saturating_sub(self.used)
}
/// The units charged so far.
pub const fn used(&self) -> u64 {
self.used
}
/// The limit this budget was created with.
pub const fn limit(&self) -> u64 {
self.limit
}
/// The name this budget was created with.
pub const fn what(&self) -> &'static str {
self.what
}
}
/// A charge that a [`Budget`] refused: what was limited, the limit, and the amount asked for.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Exceeded {
/// The budget's name, so a message can say which limit was hit.
pub what: &'static str,
/// The limit that was already reached.
pub limit: u64,
/// The amount that did not fit.
pub requested: u64,
}
impl fmt::Display for Exceeded {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "the {} limit is {}; another {} was requested", self.what, self.limit, self.requested)
}
}
impl std::error::Error for Exceeded {}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn budgets_charge_and_report_the_remainder() {
let mut budget = Budget::new("document", 10);
assert_eq!(budget.what(), "document");
assert_eq!(budget.limit(), 10);
assert_eq!(budget.remaining(), 10);
assert_eq!(budget.used(), 0);
assert_eq!(budget.charge(4), Ok(()));
assert_eq!(budget.used(), 4);
assert_eq!(budget.remaining(), 6);
assert_eq!(budget.charge(6), Ok(()));
assert_eq!(budget.remaining(), 0);
}
#[test]
fn a_refused_charge_leaves_the_total_untouched() {
let mut budget = Budget::new("part", 100);
assert_eq!(budget.charge(60), Ok(()));
let exceeded = budget.charge(41).err();
assert_eq!(exceeded, Some(Exceeded { what: "part", limit: 100, requested: 41 }));
assert_eq!(budget.used(), 60);
// The refused amount is not charged, so the budget stays usable.
assert_eq!(budget.charge(40), Ok(()));
assert_eq!(budget.used(), 100);
assert!(budget.charge(1).is_err());
}
#[test]
fn a_zero_budget_admits_nothing_and_the_message_names_the_limit() {
let mut budget = Budget::new("entries", 0);
assert_eq!(budget.remaining(), 0);
assert_eq!(budget.charge(0), Ok(()));
let message = budget.charge(1).err().map(|e| e.to_string());
assert_eq!(message.as_deref(), Some("the entries limit is 0; another 1 was requested"));
}
#[test]
fn a_limit_at_the_maximum_does_not_overflow() {
let mut budget = Budget::new("total", u64::MAX);
assert_eq!(budget.charge(u64::MAX), Ok(()));
assert_eq!(budget.remaining(), 0);
assert!(budget.charge(u64::MAX).is_err());
}
}
+429
View File
@@ -0,0 +1,429 @@
//! ZIP structure checks that run before a ZIP reader indexes the archive, and bounded entry reads.
//!
//! A ZIP file ends with an end-of-central-directory record (EOCD) naming the offset and size of a
//! central directory that lists every entry, optionally preceded by a ZIP64 pair when the archive
//! is too large for the classic fields. [`preflight`] validates those records and walks the
//! directory on the raw bytes, so a hostile entry count or directory size is refused before a
//! reader can spend memory and time on it. [`read_entry`] then bounds one entry's inflation.
//!
//! Only single-disk archives are accepted: split archives and self-extracting prefixed archives
//! are rejected, because their relative offsets would make this check disagree with the bytes a
//! later reader indexes. The algorithm follows the ZIP specification (PKWARE APPNOTE, "ZIP File
//! Format"); no third-party code is involved.
use std::fmt;
use std::io::Read;
use crate::{Budget, Exceeded};
const EOCD_SIGNATURE: &[u8; 4] = b"PK\x05\x06";
const ZIP64_EOCD_SIGNATURE: &[u8; 4] = b"PK\x06\x06";
const ZIP64_LOCATOR_SIGNATURE: &[u8; 4] = b"PK\x06\x07";
const CENTRAL_FILE_SIGNATURE: &[u8; 4] = b"PK\x01\x02";
const CENTRAL_DIGITAL_SIGNATURE: &[u8; 4] = b"PK\x05\x05";
/// The classic end-of-central-directory record, fixed part.
const EOCD_LEN: usize = 22;
/// The ZIP64 end-of-central-directory locator.
const ZIP64_LOCATOR_LEN: usize = 20;
/// The fixed part of a ZIP64 end-of-central-directory record: 12 bytes of signature and record
/// size, then 44 bytes of fixed fields.
const ZIP64_EOCD_FIXED_LEN: usize = 44;
const ZIP64_EOCD_HEADER_LEN: u64 = 12;
/// The fixed part of a central-directory file header, before the name, extra field and comment.
const CENTRAL_FILE_LEN: usize = 46;
/// The largest archive comment the classic end record can describe.
const MAX_COMMENT_LEN: usize = u16::MAX as usize;
/// Central-directory file header field offsets.
const NAME_LEN_AT: usize = 28;
const EXTRA_LEN_AT: usize = 30;
const COMMENT_LEN_AT: usize = 32;
const ENTRY_DISK_AT: usize = 34;
/// Application-owned limits for [`preflight`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Limits {
/// The most central-directory entries the archive may declare.
pub max_entries: u64,
/// The most bytes the central directory may occupy.
pub max_central_directory_bytes: u64,
/// The most bytes a central-directory entry name may occupy.
pub max_name_bytes: u16,
}
/// What [`preflight`] found: the directory's declared totals.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Directory {
/// The number of central-directory entries, counted by walking them.
pub entries: u64,
/// The declared size of the central directory in bytes.
pub central_directory_bytes: u64,
}
/// Why [`preflight`] refused an archive.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Error {
/// No end-of-central-directory record spans the end of the file.
MissingEndRecord,
/// More than one end record could claim the end of the file, so which one a reader would use
/// is not determined; none is trusted.
AmbiguousEndRecords,
/// The structure is malformed or unsupported. The message names what was wrong.
Invalid(&'static str),
/// The archive declares more entries than the limit admits.
TooManyEntries {
/// The declared entry count.
actual: u64,
/// The caller's limit.
limit: u64,
},
/// The central directory is larger than the limit admits.
CentralDirectoryTooLarge {
/// The declared directory size in bytes.
actual: u64,
/// The caller's limit.
limit: u64,
},
/// A central-directory entry name is longer than the limit admits.
NameTooLong {
/// The declared name length in bytes.
actual: u64,
/// The caller's limit.
limit: u64,
},
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::MissingEndRecord => f.write_str("the ZIP end record is missing"),
Self::AmbiguousEndRecords => f.write_str("the ZIP has more than one plausible end record"),
Self::Invalid(message) => write!(f, "invalid ZIP structure: {message}"),
Self::TooManyEntries { actual, limit } => write!(f, "the ZIP declares {actual} entries; the limit is {limit}"),
Self::CentralDirectoryTooLarge { actual, limit } => write!(f, "the ZIP central directory is {actual} bytes; the limit is {limit}"),
Self::NameTooLong { actual, limit } => write!(f, "a ZIP entry name is {actual} bytes; the limit is {limit}"),
}
}
}
impl std::error::Error for Error {}
/// Why [`read_entry`] refused an entry.
///
/// Not `PartialEq`: an [`std::io::Error`] is not comparable, so tests compare the displayed
/// message instead.
#[derive(Debug)]
pub enum ReadError {
/// The entry declares more bytes than the per-entry limit admits.
DeclaredTooLarge {
/// The declared size.
declared: u64,
/// The caller's per-entry limit.
limit: u64,
},
/// The entry produced more bytes than the smaller of its declared size and the per-entry
/// limit: the declared size lied, or the data is a decompression bomb.
Overrun {
/// The per-entry bound that was passed.
limit: u64,
},
/// The entry needed more bytes than the total budget had left.
Budget(Exceeded),
/// The reader failed.
Io(std::io::Error),
}
impl fmt::Display for ReadError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::DeclaredTooLarge { declared, limit } => write!(f, "the entry declares {declared} bytes; the limit is {limit}"),
Self::Overrun { limit } => write!(f, "the entry produced more than the {limit} bytes it is allowed"),
Self::Budget(exceeded) => exceeded.fmt(f),
Self::Io(error) => error.fmt(f),
}
}
}
impl std::error::Error for ReadError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
Self::Budget(exceeded) => Some(exceeded),
Self::Io(error) => Some(error),
_ => None,
}
}
}
impl From<std::io::Error> for ReadError {
fn from(error: std::io::Error) -> Self {
Self::Io(error)
}
}
impl From<Exceeded> for ReadError {
fn from(exceeded: Exceeded) -> Self {
Self::Budget(exceeded)
}
}
/// The most bytes [`read_entry`] reserves up front, however large the entry declares itself.
///
/// A declared size is attacker-controlled, so reserving it would let a ten-byte archive claim a
/// gigabyte; the buffer grows as real bytes arrive instead.
pub const MAX_PREALLOCATION: u64 = 64 * 1024;
/// Validate the end records and walk the declared central directory without allocating.
///
/// Checks, in order:
///
/// * the end-of-central-directory record: exactly one candidate must span the end of the file, so
/// an archive that hides a second plausible record is refused rather than guessed at;
/// * the ZIP64 pair when the classic record uses a sentinel or a locator is present: a single disk,
/// a locator immediately before the end record, the ZIP64 record ending at that locator, and no
/// disagreement between the classic and ZIP64 fields;
/// * the declared entry count, directory size and entry-name lengths against `limits`;
/// * the directory arithmetic: its offset and size must not overflow and must reach the end
/// records, so a lying offset or size cannot point the reader elsewhere;
/// * every entry header in the directory: its signature, its disk number, its variable-length
/// fields, and that the entries exactly fill the declared directory, allowing one optional
/// central-directory digital signature at its end.
///
/// Returns the declared entry count and directory size. Malformed input is an error, never a panic.
pub fn preflight(bytes: &[u8], limits: Limits) -> Result<Directory, Error> {
let eocd_at = find_end_record(bytes)?;
let directory = end_record_totals(bytes, eocd_at)?;
if directory.entries > limits.max_entries {
return Err(Error::TooManyEntries { actual: directory.entries, limit: limits.max_entries });
}
if directory.size > limits.max_central_directory_bytes {
return Err(Error::CentralDirectoryTooLarge { actual: directory.size, limit: limits.max_central_directory_bytes });
}
let directory_end = directory.offset.checked_add(directory.size).ok_or(Error::Invalid("central-directory offset overflow"))?;
let structure_at = u64::try_from(directory.structure_at).map_err(|_| Error::Invalid("end-record offset does not fit"))?;
if directory_end != structure_at {
return Err(Error::Invalid("central-directory offset and size do not reach the end records"));
}
let start = usize::try_from(directory.offset).map_err(|_| Error::Invalid("central-directory offset does not fit this platform"))?;
let end = usize::try_from(directory_end).map_err(|_| Error::Invalid("central-directory end does not fit this platform"))?;
walk_directory(bytes, start, end, directory.entries, limits.max_name_bytes)?;
Ok(Directory { entries: directory.entries, central_directory_bytes: directory.size })
}
/// Read one entry up to the smallest of `max`, the total budget's remainder and `declared`.
///
/// The declared size is refused outright when it exceeds `max`. At most [`MAX_PREALLOCATION`] bytes
/// are reserved however large the entry claims to be. The bytes are read up to the bound and only
/// the bytes actually read are charged to `total`, so a declaration that overstates the entry
/// costs nothing extra.
///
/// More bytes arriving than the bound admits is the hostile case: [`ReadError::Overrun`] when the
/// per-entry bound or the declaration was what stopped the read (a lying size or a decompression
/// bomb), and [`ReadError::Budget`] when the total budget was already the tighter bound.
///
/// Works with any [`Read`], including a `zip` crate entry reader. An I/O failure is
/// [`ReadError::Io`]; the bytes read before it are not charged.
pub fn read_entry(r: impl Read, declared: u64, max: u64, total: &mut Budget) -> Result<Vec<u8>, ReadError> {
if declared > max {
return Err(ReadError::DeclaredTooLarge { declared, limit: max });
}
let per_entry = declared.min(max);
let budget_left = total.remaining();
let bound = per_entry.min(budget_left);
// One byte past the bound has to be readable to tell "exactly the bound" from "more".
let mut out = Vec::with_capacity(usize::try_from(bound.min(MAX_PREALLOCATION)).unwrap_or(0));
let mut reader = r.take(bound.saturating_add(1));
reader.read_to_end(&mut out)?;
let read = u64::try_from(out.len()).unwrap_or(u64::MAX);
if read > bound {
// The budget was the tighter bound, so the total limit is what refused the entry.
if per_entry > budget_left {
return Err(ReadError::Budget(Exceeded { what: total.what(), limit: total.limit(), requested: read }));
}
return Err(ReadError::Overrun { limit: bound });
}
total.charge(read)?;
Ok(out)
}
/// The declared entry count, directory size and offset, and where the end records begin.
struct EndRecord {
entries: u64,
size: u64,
offset: u64,
/// The offset of the first end record: the ZIP64 record when there is one, else the EOCD.
structure_at: usize,
}
/// Find the single end-of-central-directory record that ends the file.
///
/// A record is a candidate when its signature is present, its declared comment length puts its end
/// exactly at the end of the file, and it lies in the last 64 KiB plus 22 bytes — the only place
/// the format allows one. The search runs backwards, so the record nearest the end is considered
/// first; finding a second candidate means the archive is ambiguous and is refused.
fn find_end_record(bytes: &[u8]) -> Result<usize, Error> {
let last = bytes.len().checked_sub(EOCD_LEN).ok_or(Error::MissingEndRecord)?;
let first = bytes.len().saturating_sub(EOCD_LEN.saturating_add(MAX_COMMENT_LEN));
let mut found = None;
for at in (first..=last).rev() {
if !signature_at(bytes, at, EOCD_SIGNATURE) {
continue;
}
let Some(comment_len) = u16_at(bytes, at + 20).map(usize::from) else { continue };
let Some(end) = at.checked_add(EOCD_LEN).and_then(|value| value.checked_add(comment_len)) else { continue };
if end != bytes.len() {
continue;
}
if found.replace(at).is_some() {
return Err(Error::AmbiguousEndRecords);
}
}
found.ok_or(Error::MissingEndRecord)
}
/// Read the totals the end records declare, using the ZIP64 pair when one is present.
fn end_record_totals(bytes: &[u8], eocd_at: usize) -> Result<EndRecord, Error> {
let disk = u16_at(bytes, eocd_at + 4).ok_or(Error::Invalid("truncated end record"))?;
let directory_disk = u16_at(bytes, eocd_at + 6).ok_or(Error::Invalid("truncated end record"))?;
let disk_entries = u16_at(bytes, eocd_at + 8).ok_or(Error::Invalid("truncated end record"))?;
let entries = u16_at(bytes, eocd_at + 10).ok_or(Error::Invalid("truncated end record"))?;
let directory_size = u32_at(bytes, eocd_at + 12).ok_or(Error::Invalid("truncated end record"))?;
let directory_offset = u32_at(bytes, eocd_at + 16).ok_or(Error::Invalid("truncated end record"))?;
let sentinel = disk == u16::MAX
|| directory_disk == u16::MAX
|| disk_entries == u16::MAX
|| entries == u16::MAX
|| directory_size == u32::MAX
|| directory_offset == u32::MAX;
let locator_at = eocd_at.checked_sub(ZIP64_LOCATOR_LEN).filter(|at| signature_at(bytes, *at, ZIP64_LOCATOR_SIGNATURE));
let Some(locator_at) = locator_at else {
if sentinel {
return Err(Error::Invalid("ZIP64 sentinel without a ZIP64 end record"));
}
if disk != 0 || directory_disk != 0 || disk_entries != entries {
return Err(Error::Invalid("multi-disk ZIP archive"));
}
return Ok(EndRecord {
entries: u64::from(entries),
size: u64::from(directory_size),
offset: u64::from(directory_offset),
structure_at: eocd_at,
});
};
let zip64_disk = u32_at(bytes, locator_at + 4).ok_or(Error::Invalid("truncated ZIP64 locator"))?;
let zip64_offset = u64_at(bytes, locator_at + 8).ok_or(Error::Invalid("truncated ZIP64 locator"))?;
let zip64_disks = u32_at(bytes, locator_at + 16).ok_or(Error::Invalid("truncated ZIP64 locator"))?;
if zip64_disk != 0 || zip64_disks != 1 {
return Err(Error::Invalid("multi-disk ZIP64 archive"));
}
let zip64_at = usize::try_from(zip64_offset).map_err(|_| Error::Invalid("ZIP64 end-record offset does not fit this platform"))?;
if !signature_at(bytes, zip64_at, ZIP64_EOCD_SIGNATURE) {
return Err(Error::Invalid("the ZIP64 end record is missing at its declared offset"));
}
let zip64_disk_number = u32_at(bytes, zip64_at + 16).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
let zip64_directory_disk = u32_at(bytes, zip64_at + 20).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
let zip64_disk_entries = u64_at(bytes, zip64_at + 24).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
let zip64_entries = u64_at(bytes, zip64_at + 32).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
let zip64_directory_size = u64_at(bytes, zip64_at + 40).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
let zip64_directory_offset = u64_at(bytes, zip64_at + 48).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
let zip64_record_size = u64_at(bytes, zip64_at + 4).ok_or(Error::Invalid("truncated ZIP64 end record"))?;
if zip64_record_size < u64::try_from(ZIP64_EOCD_FIXED_LEN).unwrap_or(44) {
return Err(Error::Invalid("the ZIP64 end record is too short"));
}
// The ZIP64 record ends exactly where its locator begins; anything else means the locator
// points somewhere other than this record.
let record_end = u64::try_from(zip64_at)
.ok()
.and_then(|at| at.checked_add(ZIP64_EOCD_HEADER_LEN))
.and_then(|at| at.checked_add(zip64_record_size))
.ok_or(Error::Invalid("ZIP64 end-record length overflow"))?;
if record_end != u64::try_from(locator_at).unwrap_or(u64::MAX) {
return Err(Error::Invalid("the ZIP64 end record does not end at its locator"));
}
if zip64_disk_number != 0 || zip64_directory_disk != 0 || zip64_disk_entries != zip64_entries {
return Err(Error::Invalid("multi-disk ZIP64 archive"));
}
if (disk != u16::MAX && disk != 0)
|| (directory_disk != u16::MAX && directory_disk != 0)
|| (disk_entries != u16::MAX && u64::from(disk_entries) != zip64_entries)
|| (entries != u16::MAX && u64::from(entries) != zip64_entries)
|| (directory_size != u32::MAX && u64::from(directory_size) != zip64_directory_size)
|| (directory_offset != u32::MAX && u64::from(directory_offset) != zip64_directory_offset)
{
return Err(Error::Invalid("the classic and ZIP64 end records disagree"));
}
Ok(EndRecord { entries: zip64_entries, size: zip64_directory_size, offset: zip64_directory_offset, structure_at: zip64_at })
}
/// Walk the declared directory, checking each entry header and that the entries fill it exactly.
fn walk_directory(bytes: &[u8], start: usize, end: usize, entries: u64, max_name_bytes: u16) -> Result<(), Error> {
if end > bytes.len() || start > end {
return Err(Error::Invalid("the central directory is outside the archive"));
}
let mut at = start;
for _ in 0..entries {
if !signature_at(bytes, at, CENTRAL_FILE_SIGNATURE) {
return Err(Error::Invalid("a central-directory entry signature is missing"));
}
let name = u16_at(bytes, at + NAME_LEN_AT).ok_or(Error::Invalid("truncated central-directory entry"))?;
let extra = usize::from(u16_at(bytes, at + EXTRA_LEN_AT).ok_or(Error::Invalid("truncated central-directory entry"))?);
let comment = usize::from(u16_at(bytes, at + COMMENT_LEN_AT).ok_or(Error::Invalid("truncated central-directory entry"))?);
let disk = u16_at(bytes, at + ENTRY_DISK_AT).ok_or(Error::Invalid("truncated central-directory entry"))?;
if disk != 0 {
return Err(Error::Invalid("a central-directory entry refers to another disk"));
}
if name == 0 || name > max_name_bytes {
return Err(Error::NameTooLong { actual: u64::from(name), limit: u64::from(max_name_bytes) });
}
at = at
.checked_add(CENTRAL_FILE_LEN)
.and_then(|value| value.checked_add(usize::from(name)))
.and_then(|value| value.checked_add(extra))
.and_then(|value| value.checked_add(comment))
.ok_or(Error::Invalid("central-directory entry length overflow"))?;
if at > end {
return Err(Error::Invalid("a central-directory entry exceeds the declared directory size"));
}
}
// The directory normally ends where its last entry does, optionally followed by one
// central-directory digital signature record.
if at < end && signature_at(bytes, at, CENTRAL_DIGITAL_SIGNATURE) {
let signature_len = usize::from(u16_at(bytes, at + 4).ok_or(Error::Invalid("truncated central-directory signature"))?);
at = at
.checked_add(6)
.and_then(|value| value.checked_add(signature_len))
.ok_or(Error::Invalid("central-directory signature length overflow"))?;
}
if at != end {
return Err(Error::Invalid("the central-directory entry count and size disagree"));
}
Ok(())
}
fn bytes_at(bytes: &[u8], at: usize, len: usize) -> Option<&[u8]> {
bytes.get(at..at.checked_add(len)?)
}
fn u16_at(bytes: &[u8], at: usize) -> Option<u16> {
Some(u16::from_le_bytes(bytes_at(bytes, at, 2)?.try_into().ok()?))
}
fn u32_at(bytes: &[u8], at: usize) -> Option<u32> {
Some(u32::from_le_bytes(bytes_at(bytes, at, 4)?.try_into().ok()?))
}
fn u64_at(bytes: &[u8], at: usize) -> Option<u64> {
Some(u64::from_le_bytes(bytes_at(bytes, at, 8)?.try_into().ok()?))
}
fn signature_at(bytes: &[u8], at: usize, signature: &[u8; 4]) -> bool {
bytes_at(bytes, at, signature.len()) == Some(signature.as_slice())
}
#[cfg(test)]
mod tests;
+580
View File
@@ -0,0 +1,580 @@
//! Tests for the ZIP preflight and the bounded entry read.
//!
//! Most cases build their bytes here rather than carrying archive fixtures, so the hostile shapes
//! (lying counts, offsets past the end, ZIP64 variants, overflow attempts) are visible in the test
//! and can be adjusted a byte at a time.
use crate::Budget;
use crate::preflight::{self as zip, Directory, Error, Limits, MAX_PREALLOCATION, ReadError};
/// Limits wide enough that only the case under test can trip them.
fn roomy() -> Limits {
Limits { max_entries: 100_000, max_central_directory_bytes: 32 * 1024 * 1024, max_name_bytes: 4_096 }
}
const EOCD_SIGNATURE: &[u8; 4] = b"PK\x05\x06";
const ZIP64_EOCD_SIGNATURE: &[u8; 4] = b"PK\x06\x06";
const ZIP64_LOCATOR_SIGNATURE: &[u8; 4] = b"PK\x06\x07";
const CENTRAL_FILE_SIGNATURE: &[u8; 4] = b"PK\x01\x02";
/// One central-directory file header carrying `name`, with no extra field or comment.
fn central_entry(name: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(CENTRAL_FILE_SIGNATURE);
out.extend_from_slice(&20u16.to_le_bytes()); // version made by
out.extend_from_slice(&20u16.to_le_bytes()); // version needed
out.extend_from_slice(&0u16.to_le_bytes()); // flags
out.extend_from_slice(&0u16.to_le_bytes()); // method: stored
out.extend_from_slice(&0u16.to_le_bytes()); // time
out.extend_from_slice(&0u16.to_le_bytes()); // date
out.extend_from_slice(&0u32.to_le_bytes()); // crc32
out.extend_from_slice(&0u32.to_le_bytes()); // compressed size
out.extend_from_slice(&0u32.to_le_bytes()); // uncompressed size
out.extend_from_slice(&u16::try_from(name.len()).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes()); // extra length
out.extend_from_slice(&0u16.to_le_bytes()); // comment length
out.extend_from_slice(&0u16.to_le_bytes()); // disk
out.extend_from_slice(&0u16.to_le_bytes()); // internal attributes
out.extend_from_slice(&0u32.to_le_bytes()); // external attributes
out.extend_from_slice(&0u32.to_le_bytes()); // local header offset
out.extend_from_slice(name);
assert_eq!(out.len(), 46 + name.len());
out
}
/// A minimal, well-formed archive holding `names`, with `comment` after its end record. The
/// returned offset is where that end record starts.
fn archive(names: &[&[u8]], comment: &[u8]) -> (Vec<u8>, usize) {
assert!(comment.len() <= usize::from(u16::MAX));
let mut out = Vec::new();
for name in names {
out.extend_from_slice(&central_entry(name));
}
let directory_size = out.len();
out.extend_from_slice(EOCD_SIGNATURE);
out.extend_from_slice(&0u16.to_le_bytes()); // disk
out.extend_from_slice(&0u16.to_le_bytes()); // directory disk
out.extend_from_slice(&u16::try_from(names.len()).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&u16::try_from(names.len()).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&u32::try_from(directory_size).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&0u32.to_le_bytes()); // directory offset: the directory starts at 0
out.extend_from_slice(&u16::try_from(comment.len()).unwrap_or(0).to_le_bytes());
out.extend_from_slice(comment);
let eocd_at = out.len() - comment.len() - 22;
(out, eocd_at)
}
/// An end-record-shaped run declaring `comment_len` bytes after itself. It is a candidate end
/// record only when it also ends exactly at the end of the file.
fn forged_end_record(comment_len: u16) -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(EOCD_SIGNATURE);
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(&0u32.to_le_bytes());
out.extend_from_slice(&0u32.to_le_bytes());
out.extend_from_slice(&comment_len.to_le_bytes());
assert_eq!(out.len(), 22);
out
}
/// A single-entry archive whose central directory is preceded by `pad` bytes of file data, so the
/// directory does not start at zero.
fn archive_with_padding(pad: usize) -> Vec<u8> {
let mut out = vec![0u8; pad];
let directory_at = out.len();
out.extend_from_slice(&central_entry(b"word/document.xml"));
let directory_size = out.len() - directory_at;
out.extend_from_slice(EOCD_SIGNATURE);
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(&1u16.to_le_bytes());
out.extend_from_slice(&1u16.to_le_bytes());
out.extend_from_slice(&u32::try_from(directory_size).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&u32::try_from(directory_at).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out
}
/// A ZIP64 archive holding `names`: the entries, then the ZIP64 end record, its locator and the
/// classic end record with sentinel fields.
fn zip64_archive(names: &[&[u8]]) -> Vec<u8> {
let mut out = Vec::new();
for name in names {
out.extend_from_slice(&central_entry(name));
}
let directory_size = out.len();
let zip64_at = out.len();
out.extend_from_slice(ZIP64_EOCD_SIGNATURE);
out.extend_from_slice(&44u64.to_le_bytes()); // the 44 fixed bytes that follow
out.extend_from_slice(&45u16.to_le_bytes());
out.extend_from_slice(&45u16.to_le_bytes());
out.extend_from_slice(&0u32.to_le_bytes()); // this disk
out.extend_from_slice(&0u32.to_le_bytes()); // directory disk
out.extend_from_slice(&u64::from(names.len() as u32).to_le_bytes());
out.extend_from_slice(&u64::from(names.len() as u32).to_le_bytes());
out.extend_from_slice(&u64::try_from(directory_size).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&0u64.to_le_bytes()); // directory offset
let locator_at = out.len();
out.extend_from_slice(ZIP64_LOCATOR_SIGNATURE);
out.extend_from_slice(&0u32.to_le_bytes());
out.extend_from_slice(&u64::try_from(zip64_at).unwrap_or(0).to_le_bytes());
out.extend_from_slice(&1u32.to_le_bytes());
assert_eq!(out.len() - locator_at, 20);
out.extend_from_slice(EOCD_SIGNATURE);
out.extend_from_slice(&u16::MAX.to_le_bytes());
out.extend_from_slice(&u16::MAX.to_le_bytes());
out.extend_from_slice(&u16::MAX.to_le_bytes());
out.extend_from_slice(&u16::MAX.to_le_bytes());
out.extend_from_slice(&u32::MAX.to_le_bytes());
out.extend_from_slice(&u32::MAX.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out
}
/// Overwrite the bytes at `at` in `bytes`; every offset here comes from the fixture, not a file.
fn put(bytes: &mut [u8], at: usize, value: &[u8]) {
let end = at + value.len();
bytes.get_mut(at..end).unwrap().copy_from_slice(value);
}
/// Overwrite the little-endian `u16` at `at`.
fn put_u16(bytes: &mut [u8], at: usize, value: u16) {
put(bytes, at, &value.to_le_bytes());
}
/// Overwrite the little-endian `u32` at `at`.
fn put_u32(bytes: &mut [u8], at: usize, value: u32) {
put(bytes, at, &value.to_le_bytes());
}
/// An entry error's message. `ReadError` carries a `std::io::Error`, which is not comparable, so
/// tests compare what the caller would see.
fn message(error: Option<ReadError>) -> Option<String> {
error.map(|error| error.to_string())
}
#[test]
fn a_plain_archive_is_counted_and_its_offsets_are_honoured() {
let (bytes, _) = archive(&[b"a.txt", b"dir/b.xml"], b"");
assert_eq!(zip::preflight(&bytes, roomy()).unwrap(), Directory { entries: 2, central_directory_bytes: 106 });
// A directory that does not start at zero, because local file data precedes it.
let padded = archive_with_padding(1_000);
assert_eq!(zip::preflight(&padded, roomy()).unwrap(), Directory { entries: 1, central_directory_bytes: 46 + 17 });
}
#[test]
fn an_archive_comment_ending_exactly_at_eof_is_accepted() {
let (bytes, _) = archive(&[b"a.txt"], b"a comment with PK\x05\x06 inside");
assert_eq!(zip::preflight(&bytes, roomy()).unwrap(), Directory { entries: 1, central_directory_bytes: 51 });
}
#[test]
fn a_file_too_short_for_an_end_record_is_refused() {
for length in 0..22 {
assert_eq!(zip::preflight(&vec![0u8; length], roomy()), Err(Error::MissingEndRecord));
}
// A signature whose comment length runs past the end of the file is not an end record.
let mut bytes = vec![0u8; 22];
put(&mut bytes, 0, EOCD_SIGNATURE);
put_u16(&mut bytes, 20, 7);
assert_eq!(zip::preflight(&bytes, roomy()), Err(Error::MissingEndRecord));
}
#[test]
fn a_second_end_record_inside_the_comment_is_ambiguous() {
// The real end record's comment contains another end record that also ends exactly at EOF.
let trailing = b"trailing";
let mut comment = forged_end_record(u16::try_from(trailing.len()).unwrap());
comment.extend_from_slice(trailing);
let (bytes, _) = archive(&[b"a.txt"], &comment);
assert_eq!(bytes.len() - comment.len() + 22 + trailing.len(), bytes.len(), "the forged record must reach EOF");
assert_eq!(zip::preflight(&bytes, roomy()), Err(Error::AmbiguousEndRecords));
}
#[test]
fn a_truncated_archive_is_refused() {
let (bytes, _) = archive(&[b"a.txt", b"b.txt"], b"");
for keep in [0, 1, 10, 21, 22, bytes.len() - 23, bytes.len() - 1] {
let truncated = bytes.get(..keep).unwrap_or_default().to_vec();
assert!(zip::preflight(&truncated, roomy()).is_err(), "a truncation to {keep} bytes was accepted");
}
// A directory size that no longer reaches the end records.
let (mut short, eocd_at) = archive(&[b"a.txt"], b"");
put_u32(&mut short, eocd_at + 12, 1);
assert_eq!(zip::preflight(&short, roomy()), Err(Error::Invalid("central-directory offset and size do not reach the end records")));
}
#[test]
fn a_directory_offset_past_the_end_of_the_file_is_refused() {
let (mut bytes, eocd_at) = archive(&[b"a.txt"], b"");
put_u32(&mut bytes, eocd_at + 16, 1_000_000);
assert_eq!(zip::preflight(&bytes, roomy()), Err(Error::Invalid("central-directory offset and size do not reach the end records")));
// An offset that does not match the declared size.
let (mut shifted, eocd_at) = archive(&[b"a.txt"], b"");
put_u32(&mut shifted, eocd_at + 16, 1);
assert_eq!(zip::preflight(&shifted, roomy()), Err(Error::Invalid("central-directory offset and size do not reach the end records")));
// Offset and size whose sum is far past the end of the file. The limits are lifted so this
// exercises the offset arithmetic rather than the size limit catching it first. The classic
// maximum on either field means "look at the ZIP64 records" in the format, so the lying values
// stop one short of it and the ZIP64 case below uses it.
let unlimited = Limits { max_entries: u64::MAX, max_central_directory_bytes: u64::MAX, ..roomy() };
let (mut far, eocd_at) = archive(&[b"a.txt"], b"");
put_u32(&mut far, eocd_at + 12, u32::MAX - 1);
put_u32(&mut far, eocd_at + 16, u32::MAX - 1);
assert_eq!(zip::preflight(&far, unlimited), Err(Error::Invalid("central-directory offset and size do not reach the end records")));
// The same on the ZIP64 path, where the sum cannot even be represented.
let mut zip64 = zip64_archive(&[b"a.txt"]);
let zip64_at = zip64.len() - 22 - 20 - 56;
put(&mut zip64, zip64_at + 48, &u64::MAX.to_le_bytes());
put(&mut zip64, zip64_at + 40, &u64::MAX.to_le_bytes());
assert_eq!(zip::preflight(&zip64, unlimited), Err(Error::Invalid("central-directory offset overflow")));
}
#[test]
fn the_entry_count_limit_is_enforced() {
let (bytes, _) = archive(&[b"a.txt", b"b.txt", b"c.txt"], b"");
assert_eq!(zip::preflight(&bytes, Limits { max_entries: 3, ..roomy() }).unwrap().entries, 3);
assert_eq!(zip::preflight(&bytes, Limits { max_entries: 2, ..roomy() }), Err(Error::TooManyEntries { actual: 3, limit: 2 }));
assert_eq!(zip::preflight(&bytes, Limits { max_entries: 0, ..roomy() }), Err(Error::TooManyEntries { actual: 3, limit: 0 }));
}
#[test]
fn the_directory_size_limit_is_enforced() {
let (bytes, _) = archive(&[b"a.txt"], b"");
let size = 51u64;
assert_eq!(zip::preflight(&bytes, Limits { max_central_directory_bytes: size, ..roomy() }).unwrap().central_directory_bytes, size);
assert_eq!(
zip::preflight(&bytes, Limits { max_central_directory_bytes: size - 1, ..roomy() }),
Err(Error::CentralDirectoryTooLarge { actual: size, limit: size - 1 })
);
}
#[test]
fn the_entry_name_limit_is_enforced() {
let long = vec![b'n'; 200];
let (bytes, _) = archive(&[&long], b"");
assert_eq!(zip::preflight(&bytes, Limits { max_name_bytes: 200, ..roomy() }).unwrap().entries, 1);
assert_eq!(zip::preflight(&bytes, Limits { max_name_bytes: 199, ..roomy() }), Err(Error::NameTooLong { actual: 200, limit: 199 }));
// An empty name is malformed whatever the limit says.
let (empty, _) = archive(&[b""], b"");
assert_eq!(zip::preflight(&empty, roomy()), Err(Error::NameTooLong { actual: 0, limit: 4_096 }));
// A name length that would run past the declared directory.
let (mut lying, _) = archive(&[b"a.txt"], b"");
put_u16(&mut lying, 28, u16::MAX);
assert!(zip::preflight(&lying, roomy()).is_err());
}
#[test]
fn end_record_counts_that_disagree_with_the_directory_are_caught() {
// Far more entries than the directory holds: the signatures run out inside the directory.
let (mut too_many, eocd_at) = archive(&[b"a.txt"], b"");
put_u16(&mut too_many, eocd_at + 8, 1_000);
put_u16(&mut too_many, eocd_at + 10, 1_000);
assert!(matches!(zip::preflight(&too_many, roomy()), Err(Error::Invalid(_))));
// Fewer entries than the directory holds: the entries no longer fill the declared size.
let (mut too_few, eocd_at) = archive(&[b"a.txt", b"b.txt"], b"");
put_u16(&mut too_few, eocd_at + 8, 1);
put_u16(&mut too_few, eocd_at + 10, 1);
assert_eq!(zip::preflight(&too_few, roomy()), Err(Error::Invalid("the central-directory entry count and size disagree")));
// A huge declared count is refused by the walk once the limit is lifted, and by the limit
// before the walk when the limit is the tighter bound.
let (mut huge, eocd_at) = archive(&[b"a.txt"], b"");
put_u16(&mut huge, eocd_at + 8, u16::MAX - 1);
put_u16(&mut huge, eocd_at + 10, u16::MAX - 1);
assert!(matches!(zip::preflight(&huge, roomy()), Err(Error::Invalid(_))));
assert_eq!(zip::preflight(&huge, Limits { max_entries: 10, ..roomy() }), Err(Error::TooManyEntries { actual: 65_534, limit: 10 }));
}
#[test]
fn entries_or_end_records_that_name_another_disk_are_refused() {
let (mut entry, _) = archive(&[b"a.txt"], b"");
put_u16(&mut entry, 34, 1);
assert_eq!(zip::preflight(&entry, roomy()), Err(Error::Invalid("a central-directory entry refers to another disk")));
let (mut directory_disk, eocd_at) = archive(&[b"a.txt"], b"");
put_u16(&mut directory_disk, eocd_at + 6, 1);
assert_eq!(zip::preflight(&directory_disk, roomy()), Err(Error::Invalid("multi-disk ZIP archive")));
let (mut disk, eocd_at) = archive(&[b"a.txt"], b"");
put_u16(&mut disk, eocd_at + 4, 1);
assert_eq!(zip::preflight(&disk, roomy()), Err(Error::Invalid("multi-disk ZIP archive")));
let (mut disk_entries, eocd_at) = archive(&[b"a.txt"], b"");
put_u16(&mut disk_entries, eocd_at + 8, 2);
assert_eq!(zip::preflight(&disk_entries, roomy()), Err(Error::Invalid("multi-disk ZIP archive")));
}
#[test]
fn one_trailing_digital_signature_is_allowed_and_other_trailing_data_is_not() {
// A one-entry archive whose directory is preceded by 1,000 bytes, so offsets are not zero and
// the directory size can be adjusted without moving the end record.
let mut bytes = vec![0u8; 1_000];
bytes.extend_from_slice(&central_entry(b"a.txt"));
let directory_size = bytes.len() - 1_000;
bytes.extend_from_slice(EOCD_SIGNATURE);
bytes.extend_from_slice(&0u16.to_le_bytes());
bytes.extend_from_slice(&0u16.to_le_bytes());
bytes.extend_from_slice(&1u16.to_le_bytes());
bytes.extend_from_slice(&1u16.to_le_bytes());
bytes.extend_from_slice(&u32::try_from(directory_size).unwrap_or(0).to_le_bytes());
bytes.extend_from_slice(&1_000u32.to_le_bytes());
bytes.extend_from_slice(&0u16.to_le_bytes());
let eocd_at = bytes.len() - 22;
assert_eq!(zip::preflight(&bytes, roomy()).unwrap(), Directory { entries: 1, central_directory_bytes: 51 });
// A central-directory digital signature record appended to the directory is legal.
let mut signed = bytes.get(..eocd_at).unwrap_or_default().to_vec();
signed.extend_from_slice(b"PK\x05\x05");
signed.extend_from_slice(&3u16.to_le_bytes());
signed.extend_from_slice(b"xyz");
signed.extend_from_slice(bytes.get(eocd_at..).unwrap_or_default());
let signed_eocd_at = signed.len() - 22;
put_u32(&mut signed, signed_eocd_at + 12, directory_size as u32 + 6 + 3);
assert_eq!(zip::preflight(&signed, roomy()).unwrap(), Directory { entries: 1, central_directory_bytes: 60 });
// Any other trailing data makes the entry count and the declared size disagree.
let mut junk = bytes.get(..eocd_at).unwrap_or_default().to_vec();
junk.extend_from_slice(b"junk");
junk.extend_from_slice(bytes.get(eocd_at..).unwrap_or_default());
let junk_eocd_at = junk.len() - 22;
put_u32(&mut junk, junk_eocd_at + 12, directory_size as u32 + 4);
assert_eq!(zip::preflight(&junk, roomy()), Err(Error::Invalid("the central-directory entry count and size disagree")));
}
#[test]
fn zip64_end_records_are_walked_and_their_totals_are_limited() {
let bytes = zip64_archive(&[b"a.txt", b"b.txt"]);
assert_eq!(zip::preflight(&bytes, roomy()).unwrap(), Directory { entries: 2, central_directory_bytes: 2 * 46 + 4 + 6 });
assert_eq!(zip::preflight(&bytes, Limits { max_entries: 1, ..roomy() }), Err(Error::TooManyEntries { actual: 2, limit: 1 }));
}
#[test]
fn zip64_sentinels_without_a_zip64_end_record_are_refused() {
let (mut bytes, eocd_at) = archive(&[b"a.txt"], b"");
put_u16(&mut bytes, eocd_at + 10, u16::MAX);
assert_eq!(zip::preflight(&bytes, roomy()), Err(Error::Invalid("ZIP64 sentinel without a ZIP64 end record")));
}
#[test]
fn zip64_records_are_checked_against_their_locator_and_each_other() {
let good = zip64_archive(&[b"a.txt", b"b.txt"]);
let locator_at = good.len() - 22 - 20;
let zip64_at = locator_at - 56;
// A locator pointing where there is no ZIP64 end record.
let mut missing = good.clone();
put(&mut missing, locator_at + 8, &7u64.to_le_bytes());
assert_eq!(zip::preflight(&missing, roomy()), Err(Error::Invalid("the ZIP64 end record is missing at its declared offset")));
// A ZIP64 record whose length does not put its end at the locator.
let mut bad_size = good.clone();
put(&mut bad_size, zip64_at + 4, &45u64.to_le_bytes());
assert_eq!(zip::preflight(&bad_size, roomy()), Err(Error::Invalid("the ZIP64 end record does not end at its locator")));
// A ZIP64 record too short to hold the fields it must.
let mut too_short = good.clone();
put(&mut too_short, zip64_at + 4, &43u64.to_le_bytes());
assert_eq!(zip::preflight(&too_short, roomy()), Err(Error::Invalid("the ZIP64 end record is too short")));
// A locator that describes more than one disk.
let mut multi_disk = good.clone();
put(&mut multi_disk, locator_at + 16, &2u32.to_le_bytes());
assert_eq!(zip::preflight(&multi_disk, roomy()), Err(Error::Invalid("multi-disk ZIP64 archive")));
// Classic fields that contradict the ZIP64 record.
let mut lying = good.clone();
let eocd_at = lying.len() - 22;
put_u16(&mut lying, eocd_at + 10, 1);
assert_eq!(zip::preflight(&lying, roomy()), Err(Error::Invalid("the classic and ZIP64 end records disagree")));
// A ZIP64 offset that cannot be a platform offset.
let mut huge = good.clone();
put(&mut huge, locator_at + 8, &u64::MAX.to_le_bytes());
assert!(matches!(zip::preflight(&huge, roomy()), Err(Error::Invalid(_))));
}
#[test]
fn corrupting_any_single_byte_of_an_archive_never_panics() {
let (bytes, _) = archive(&[b"a.txt", b"b.xml"], b"comment");
for at in 0..bytes.len() {
for value in [0x00u8, 0x01, 0x41, 0xff] {
let mut corrupted = bytes.clone();
if let Some(byte) = corrupted.get_mut(at) {
*byte = value;
}
let _ = zip::preflight(&corrupted, roomy());
}
}
}
#[test]
fn an_entry_exactly_at_its_declared_size_is_read_and_charged() {
let mut total = Budget::new("document", 1_000);
let data = vec![7u8; 100];
let out = zip::read_entry(std::io::Cursor::new(data.as_slice()), 100, 1_000, &mut total).unwrap();
assert_eq!(out.len(), 100);
assert_eq!(total.used(), 100);
assert_eq!(total.remaining(), 900);
}
#[test]
fn a_declared_size_over_the_per_entry_limit_is_refused_before_reading() {
let mut total = Budget::new("document", 1_000_000);
let data = vec![0u8; 10];
assert_eq!(
message(zip::read_entry(std::io::Cursor::new(data.as_slice()), 5_000, 4_096, &mut total).err()),
Some("the entry declares 5000 bytes; the limit is 4096".to_string())
);
assert_eq!(total.used(), 0, "a refused entry must not be charged");
}
#[test]
fn a_lying_declared_size_is_caught_and_an_overstated_one_only_bounds_the_read() {
let mut total = Budget::new("document", 1_000_000);
let data = vec![7u8; 100];
// A hundred bytes arrive from an entry that declared ten.
assert_eq!(
message(zip::read_entry(std::io::Cursor::new(data.as_slice()), 10, 1_000, &mut total).err()),
Some("the entry produced more than the 10 bytes it is allowed".to_string())
);
assert_eq!(total.used(), 0, "an overrun must not be charged");
// A hundred bytes arrive from an entry that declared a thousand: bounded, and only the real
// bytes are charged.
let out = zip::read_entry(std::io::Cursor::new(data.as_slice()), 1_000, 4_096, &mut total).unwrap();
assert_eq!(out.len(), 100);
assert_eq!(total.used(), 100);
}
#[test]
fn a_decompression_bomb_is_rejected_at_the_smallest_bound() {
// The entry declares a megabyte and inflates past it: the per-entry bound stops the read.
let mut total = Budget::new("document", 64 * 1024 * 1024);
let bomb = Bomb { remaining: 128 * 1024 * 1024, chunk: 64 * 1024 };
let error = zip::read_entry(bomb, 1_000_000, 4 * 1024 * 1024, &mut total).err();
assert_eq!(message(error), Some("the entry produced more than the 1000000 bytes it is allowed".to_string()));
assert_eq!(total.used(), 0, "a bomb must not be charged anything");
// The same bomb under a per-entry limit smaller than the declaration: still refused.
let mut total = Budget::new("document", 64 * 1024 * 1024);
let bomb = Bomb { remaining: 128 * 1024 * 1024, chunk: 64 * 1024 };
let error = zip::read_entry(bomb, 100 * 1024 * 1024, 4_096, &mut total).err();
assert_eq!(message(error), Some("the entry declares 104857600 bytes; the limit is 4096".to_string()));
}
#[test]
fn a_bomb_is_rejected_at_the_total_budget_when_that_is_the_tighter_bound() {
let mut total = Budget::new("document", 8 * 1024);
let bomb = Bomb { remaining: 16 * 1024 * 1024, chunk: 1_024 };
let error = zip::read_entry(bomb, 1024 * 1024, 1024 * 1024, &mut total).err();
assert!(matches!(error, Some(ReadError::Budget(_))), "{error:?}");
assert_eq!(total.used(), 0);
}
#[test]
fn the_total_budget_bounds_a_series_of_entries() {
let mut total = Budget::new("document", 250);
let first = vec![1u8; 100];
let second = vec![2u8; 100];
let third = vec![3u8; 100];
assert_eq!(zip::read_entry(std::io::Cursor::new(first.as_slice()), 100, 1_000, &mut total).unwrap().len(), 100);
assert_eq!(zip::read_entry(std::io::Cursor::new(second.as_slice()), 100, 1_000, &mut total).unwrap().len(), 100);
assert_eq!(total.remaining(), 50);
// Only 50 bytes fit and the third entry wants 100.
assert!(matches!(zip::read_entry(std::io::Cursor::new(third.as_slice()), 100, 1_000, &mut total).err(), Some(ReadError::Budget(_))));
assert_eq!(total.used(), 200);
}
#[test]
fn a_declared_size_does_not_preallocate_more_than_the_fixed_cap() {
let mut total = Budget::new("document", u64::from(u32::MAX));
let data = vec![0u8; 8];
let out = zip::read_entry(std::io::Cursor::new(data.as_slice()), u64::from(u32::MAX), u64::from(u32::MAX), &mut total).unwrap();
assert_eq!(out.len(), 8);
// The buffer must not have reserved the declared size. Capacity may exceed the length, so this
// checks the reservation really was capped rather than a rounded-up length.
assert!(out.capacity() < 1024 * 1024, "a lying declaration reserved {} bytes", out.capacity());
assert!(u64::try_from(out.capacity()).unwrap_or(u64::MAX) <= MAX_PREALLOCATION.saturating_add(64));
}
#[test]
fn an_exhausted_budget_admits_nothing_and_a_zero_length_entry_still_reads() {
let mut total = Budget::new("document", 0);
assert!(matches!(zip::read_entry(std::io::Cursor::new([1u8, 2, 3].as_slice()), 3, 100, &mut total).err(), Some(ReadError::Budget(_))));
let mut empty = Budget::new("document", 0);
assert_eq!(zip::read_entry(std::io::Cursor::new([].as_slice()), 0, 100, &mut empty).ok(), Some(Vec::new()));
}
#[test]
fn a_reader_that_fails_is_reported_and_nothing_is_charged() {
let mut total = Budget::new("document", 1_000);
let error = zip::read_entry(Failing, 100, 1_000, &mut total).err();
assert!(matches!(error, Some(ReadError::Io(_))), "{error:?}");
assert_eq!(total.used(), 0);
}
#[test]
fn read_errors_describe_themselves() {
assert_eq!(ReadError::DeclaredTooLarge { declared: 9, limit: 1 }.to_string(), "the entry declares 9 bytes; the limit is 1");
assert_eq!(ReadError::Overrun { limit: 4 }.to_string(), "the entry produced more than the 4 bytes it is allowed");
assert_eq!(
ReadError::Budget(crate::Exceeded { what: "document", limit: 8, requested: 9 }).to_string(),
"the document limit is 8; another 9 was requested"
);
assert!(std::error::Error::source(&ReadError::Budget(crate::Exceeded { what: "x", limit: 0, requested: 1 })).is_some());
assert!(std::error::Error::source(&ReadError::Io(std::io::Error::other("closed"))).is_some());
assert!(std::error::Error::source(&ReadError::Overrun { limit: 1 }).is_none());
}
#[test]
fn preflight_errors_describe_themselves() {
let cases = [
(Error::MissingEndRecord, "the ZIP end record is missing"),
(Error::AmbiguousEndRecords, "the ZIP has more than one plausible end record"),
(Error::Invalid("why"), "invalid ZIP structure: why"),
(Error::TooManyEntries { actual: 5, limit: 2 }, "the ZIP declares 5 entries; the limit is 2"),
(Error::CentralDirectoryTooLarge { actual: 9, limit: 8 }, "the ZIP central directory is 9 bytes; the limit is 8"),
(Error::NameTooLong { actual: 3, limit: 2 }, "a ZIP entry name is 3 bytes; the limit is 2"),
];
for (error, expected) in cases {
assert_eq!(error.to_string(), expected);
}
}
/// A reader that produces far more bytes than anyone asked for, standing in for a zip bomb.
struct Bomb {
remaining: usize,
chunk: usize,
}
impl std::io::Read for Bomb {
fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
let want = buf.len().min(self.chunk).min(self.remaining);
for byte in buf.get_mut(..want).unwrap_or_default() {
*byte = 0x41;
}
self.remaining = self.remaining.saturating_sub(want);
Ok(want)
}
}
/// A reader that always fails.
struct Failing;
impl std::io::Read for Failing {
fn read(&mut self, _buf: &mut [u8]) -> std::io::Result<usize> {
Err(std::io::Error::other("the entry is corrupt"))
}
}
+58
View File
@@ -0,0 +1,58 @@
//! A ZIP archive being written.
use std::io::{Cursor, Write};
use crate::Error;
/// How an entry is stored.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Compression {
/// Uncompressed: a `mimetype` entry that must be readable in place, or data that is
/// compressed already.
Stored,
/// Deflate at level 6.
Deflated,
/// Deflate at the given level (0 to 9; higher is clamped).
Level(u8),
}
/// A ZIP archive being written into memory. The output is deterministic: the same (1980)
/// timestamp on every entry. Entries are written in the order they are added.
pub struct Writer {
zip: zip::ZipWriter<Cursor<Vec<u8>>>,
}
impl Default for Writer {
fn default() -> Writer {
Writer { zip: zip::ZipWriter::new(Cursor::new(Vec::new())) }
}
}
impl Writer {
pub fn new() -> Writer {
Writer::default()
}
/// Adds an entry. A name the archive already has is an error.
pub fn add(&mut self, name: &str, data: &[u8], compression: Compression) -> Result<(), Error> {
let (method, level) = match compression {
Compression::Stored => (zip::CompressionMethod::Stored, None),
Compression::Deflated => (zip::CompressionMethod::Deflated, Some(6)),
Compression::Level(l) => (zip::CompressionMethod::Deflated, Some(i64::from(l.min(9)))),
};
let options = zip::write::SimpleFileOptions::default()
.compression_method(method)
.compression_level(level)
.last_modified_time(zip::DateTime::default())
// Deflate can grow incompressible data a little, so ZIP64 well before 4 GiB.
.large_file(data.len() as u64 >= u64::from(u32::MAX) / 2);
let fail = |e: &dyn std::fmt::Display| Error::Write(format!("{name}: {e}"));
self.zip.start_file(name, options).map_err(|e| fail(&e))?;
self.zip.write_all(data).map_err(|e| fail(&e))
}
/// The finished archive.
pub fn finish(self) -> Result<Vec<u8>, Error> {
self.zip.finish().map(Cursor::into_inner).map_err(|e| Error::Write(e.to_string()))
}
}
+148
View File
@@ -0,0 +1,148 @@
//! Reading archives within limits, duplicate names, and the writer.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use craft_zip::{Archive, Compression, Error, Limits, Writer, check_entry_count, zip64_entry_count_sample};
const ROOMY: Limits = Limits { max_entries: 1000, max_total_bytes: 1 << 20 };
fn archive(entries: &[(&str, &[u8], Compression)]) -> Vec<u8> {
let mut w = Writer::new();
for (name, data, compression) in entries {
w.add(name, data, *compression).unwrap();
}
w.finish().unwrap()
}
fn find(bytes: &[u8], needle: &[u8], from: usize) -> usize {
from + bytes[from..].windows(needle.len()).position(|w| w == needle).unwrap()
}
#[test]
fn entries_are_written_in_order_and_read_back() {
let bytes = archive(&[
("mimetype", b"application/x-test", Compression::Stored),
("dir/a.xml", &[b'a'; 5000], Compression::Deflated),
("b.bin", b"\x00\x01", Compression::Level(9)),
]);
// A stored first entry is readable in place: local header (30) + name (8).
assert_eq!(&bytes[38..56], b"application/x-test");
assert!(bytes.len() < 1000, "the deflated entry is compressed");
let mut z = Archive::open(&bytes, ROOMY).unwrap();
assert_eq!(z.names().collect::<Vec<_>>(), ["mimetype", "dir/a.xml", "b.bin"]);
assert_eq!((z.len(), z.is_empty(), z.has_duplicate_names()), (3, false, false));
assert!(z.contains("b.bin") && !z.contains("B.bin"));
assert_eq!(z.declared_size("dir/a.xml"), Some(5000));
assert_eq!(z.declared_size("nope"), None);
assert_eq!(z.read("dir/a.xml", 5000).unwrap().unwrap(), vec![b'a'; 5000]);
assert_eq!(z.read("b.bin", 10).unwrap().unwrap(), [0, 1]);
assert_eq!(z.read("nope", 10).unwrap(), None);
assert_eq!(z.budget().used(), 5002);
}
#[test]
fn the_output_is_deterministic() {
let make = || archive(&[("a", b"one", Compression::Deflated), ("b", b"two", Compression::Stored)]);
assert_eq!(make(), make());
}
#[test]
fn adding_a_name_twice_is_an_error() {
let mut w = Writer::new();
w.add("a", b"1", Compression::Stored).unwrap();
assert!(matches!(w.add("a", b"2", Compression::Stored), Err(Error::Write(m)) if m.starts_with("a: ")));
}
#[test]
fn what_is_not_a_zip_is_refused() {
assert!(matches!(Archive::open(b"not a zip", ROOMY).err(), Some(Error::NotZip(_))));
assert!(matches!(Archive::open(b"", ROOMY).err(), Some(Error::NotZip(_))));
let bytes = archive(&[("a", b"abc", Compression::Stored)]);
assert!(Archive::open(&bytes[..bytes.len() - 5], ROOMY).is_err());
}
#[test]
fn an_entry_past_its_limit_is_refused_and_not_charged() {
let bytes = archive(&[("a", &[b'a'; 100], Compression::Deflated)]);
let mut z = Archive::open(&bytes, ROOMY).unwrap();
assert_eq!(z.read("a", 99), Err(Error::EntryTooLarge { name: "a".into(), limit: 99 }));
assert_eq!(z.budget().used(), 0);
assert_eq!(z.read("a", 100).unwrap().map(|d| d.len()), Some(100));
}
#[test]
fn entries_cannot_add_up_past_the_total() {
let data = [b'a'; 100];
let bytes = archive(&[("a", &data, Compression::Deflated), ("b", &data, Compression::Deflated), ("c", &data, Compression::Deflated)]);
let mut z = Archive::open(&bytes, Limits { max_total_bytes: 250, ..ROOMY }).unwrap();
assert!(z.read("a", 1000).unwrap().is_some());
assert!(z.read("b", 1000).unwrap().is_some());
assert_eq!(z.budget().remaining(), 50);
let refused = z.read("c", 1000);
assert_eq!(refused, Err(Error::BudgetSpent { name: "c".into(), limit: 250 }));
assert!(refused.is_err_and(|e| e.is_limit()));
// A caller with its own rule can charge more.
z.budget_mut().charge(50).unwrap();
assert_eq!(z.budget().remaining(), 0);
}
/// The size an entry declares is a claim. One that expands past it is refused whatever the
/// limit, and a small declared size does not get a large entry under a small limit.
#[test]
fn an_entry_that_expands_past_its_declared_size_is_refused() {
let mut bytes = archive(&[("a", &[b'a'; 5000], Compression::Deflated)]);
// The uncompressed size, in the central header (at 24) and the local header (at 22).
let central = find(&bytes, b"PK\x01\x02", 0);
for at in [central + 24, 22] {
assert_eq!(bytes[at..at + 4], 5000u32.to_le_bytes());
bytes[at..at + 4].copy_from_slice(&10u32.to_le_bytes());
}
let mut z = Archive::open(&bytes, ROOMY).unwrap();
assert_eq!(z.declared_size("a"), Some(10));
// `zip` notices first when it can; either way it is refused and nothing is charged.
assert!(matches!(z.read("a", 1 << 20), Err(Error::EntryTooLarge { limit: 10, .. } | Error::Entry { .. })));
assert_eq!(z.budget().used(), 0);
}
#[test]
fn a_broken_entry_is_an_entry_error() {
let mut bytes = archive(&[("a", b"hello world", Compression::Stored)]);
let at = find(&bytes, b"hello", 0);
bytes[at] = b'j';
let mut z = Archive::open(&bytes, ROOMY).unwrap();
assert!(matches!(z.read("a", 100), Err(Error::Entry { name, .. }) if name == "a"));
assert_eq!(z.budget().used(), 0);
}
#[test]
fn two_entries_with_one_name_are_told() {
let mut bytes = archive(&[("a", b"first", Compression::Stored), ("b", b"second", Compression::Stored)]);
assert!(!Archive::open(&bytes, ROOMY).unwrap().has_duplicate_names());
// Rename `b` to `a` in its local header and in the central directory.
let local = find(&bytes, b"PK\x03\x04", 4);
bytes[local + 30] = b'a';
let central = find(&bytes, b"PK\x01\x02", find(&bytes, b"PK\x01\x02", 0) + 4);
assert_eq!(bytes[central + 46], b'b');
bytes[central + 46] = b'a';
let z = Archive::open(&bytes, ROOMY).unwrap();
assert_eq!(z.len(), 1);
assert!(z.has_duplicate_names());
}
#[test]
fn an_archive_declaring_too_many_entries_is_refused_before_it_is_indexed() {
let bomb = zip64_entry_count_sample(20_001);
assert_eq!(check_entry_count(&bomb, 20_000), Err(Error::TooManyEntries { declared: 20_001, limit: 20_000 }));
assert_eq!(Archive::open(&bomb, Limits { max_entries: 20_000, ..ROOMY }).err(), Some(Error::TooManyEntries { declared: 20_001, limit: 20_000 }));
let bytes = archive(&[("a", b"", Compression::Stored), ("b", b"", Compression::Stored), ("c", b"", Compression::Stored)]);
assert_eq!(Archive::open(&bytes, Limits { max_entries: 2, ..ROOMY }).err(), Some(Error::TooManyEntries { declared: 3, limit: 2 }));
assert!(Archive::open(&bytes, Limits { max_entries: 3, ..ROOMY }).is_ok());
}
/// What the strict preflight accepts, the writer writes.
#[test]
fn written_archives_pass_the_strict_preflight() {
use craft_zip::preflight::{Limits, preflight};
let bytes = archive(&[("mimetype", b"x", Compression::Stored), ("a/b.xml", &[b'a'; 300], Compression::Deflated)]);
let directory = preflight(&bytes, Limits { max_entries: 10, max_central_directory_bytes: 1 << 16, max_name_bytes: 256 }).unwrap();
assert_eq!(directory.entries, 2);
}
+76
View File
@@ -0,0 +1,76 @@
//! The crate used the way a dependent app uses it: only the public API, from outside the crate.
//!
//! The unit tests reach into module internals; these cases check that the seam an app actually
//! consumes (a budget, a preflight, a bounded read) is enough to hold the line, and
//! that the examples in the crate README work as written.
use craft_zip::preflight::{self as zip, Limits as ZipLimits};
use craft_zip::{Budget, Exceeded};
/// A reader that never runs out of bytes, standing in for a decompression bomb.
struct Endless;
impl std::io::Read for Endless {
fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
for byte in buf.iter_mut() {
*byte = 0x41;
}
Ok(buf.len())
}
}
#[test]
fn the_readme_budget_example_works() {
let mut total = Budget::new("document", 512 * 1024 * 1024);
assert_eq!(total.charge(1024), Ok(()));
assert_eq!(total.remaining(), 512 * 1024 * 1024 - 1024);
}
#[test]
fn a_shared_budget_bounds_an_import_of_several_entries() {
// An app's whole-import limit, charged by each entry as it is read.
let mut total = Budget::new("document", 1_000);
let entry = vec![0u8; 600];
let first = zip::read_entry(std::io::Cursor::new(entry.as_slice()), 600, 4_096, &mut total).unwrap();
assert_eq!(first.len(), 600);
// Six hundred more bytes do not fit in the four hundred that remain.
let second = zip::read_entry(std::io::Cursor::new(entry.as_slice()), 600, 4_096, &mut total).err();
assert!(second.is_some(), "the total budget must stop the second entry");
// And the budget still reports only the bytes that really arrived.
assert_eq!(total.used(), 600);
assert_eq!(total.remaining(), 400);
}
#[test]
fn an_endless_reader_cannot_get_past_the_smaller_bound() {
let mut total = Budget::new("part", 1_000_000);
let error = zip::read_entry(Endless, u64::MAX, 4_096, &mut total).err();
// The declared size is over the per-entry limit, so the entry is refused before any read.
assert!(matches!(error, Some(zip::ReadError::DeclaredTooLarge { .. })), "{error:?}");
assert_eq!(total.used(), 0);
// With a declaration inside the limit, the read stops there even though the reader is endless.
let mut total = Budget::new("part", 1_000_000);
let error = zip::read_entry(Endless, 4_096, 4_096, &mut total).err();
assert!(matches!(error, Some(zip::ReadError::Overrun { limit: 4_096 })), "{error:?}");
assert_eq!(total.used(), 0, "a bomb charges nothing");
}
#[test]
fn a_budget_refusal_can_be_reported_in_the_apps_own_words() {
let mut total = Budget::new("worksheet", 10);
let refused: Exceeded = total.charge(11).expect_err("11 does not fit in 10");
assert_eq!(refused.what, "worksheet");
assert_eq!(refused.limit, 10);
assert_eq!(refused.requested, 11);
let message = refused.to_string();
assert!(message.contains("worksheet") && message.contains("10") && message.contains("11"), "{message}");
}
#[test]
fn the_readme_zip_preflight_example_works() {
// The bytes of no archive at all: the example's shape is what is checked here.
let limits = ZipLimits { max_entries: 20_000, max_central_directory_bytes: 16 * 1024 * 1024, max_name_bytes: 1_024 };
assert_eq!(zip::preflight(&[], limits), Err(zip::Error::MissingEndRecord));
}