mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-13 07:15:30 +00:00
* Add craft-pcraft-format: the envelope of PhotoCraft's .pcraft bundle The manifest around the document, with the document as a type parameter (Manifest<D>), TileRef, the bundle's entry paths and hash check, FORMAT_VERSION, the manifest decode limits with the nesting and item scan, and the migration runner as a generic step list. Messages and serialized field order are PhotoCraft's, so its saved manifests stay byte-identical when it switches to these types. Part of #59. * ci: do not keep the checkout token in the pcraft-format workflow The checkout step sets persist-credentials: false, as jpeg.yml and media.yml do. No later step in the job uses the token. * pcraft-format: build no entry name from an invalid hash tile_path and blob_path return None unless the hash is 64 lowercase hex digits, so a hash read from a file such as "../x" never becomes an entry name. TileRef::path gives a tile's entry the same way. The docs note that MAX_GROUP_DEPTH is a copy of PhotoCraft's constant, and that check_manifest also refuses a manifest larger than max_manifest_bytes, which PhotoCraft's check before a save does not. * ci: pin actions/checkout by commit and Rust to 1.95 in the pcraft-format workflow * pcraft-format: read_manifest is the one reader of a manifest read_manifest(json, limits, steps, target) checks the JSON against the limits, refuses a format_version newer than target, runs the migration steps, and reads the document as D and the other envelope members. read_manifest_with reads the document with the caller's function instead. Manifest no longer implements Deserialize. Error::TooNew holds the manifest's generator, so a message can name the writer; its message is unchanged. TooNew, Manifest and TileRef are #[non_exhaustive], and Manifest::new and TileRef::new build them. TILES_DIR, BLOBS_DIR, COMPOSITE_DIR and OBJECT_SUFFIX name the bundle's directories and the object suffix. The docs of max_blob_bytes and max_total_bytes say the caller enforces them. * pcraft-format: read_manifest reports errors in serde's struct order read_manifest reads the envelope members and the document in the order the parsed object holds them, and reports a missing member after the members that are present. A manifest with several faults gives the same error as a derived Deserialize of the whole manifest, as PhotoCraft reads it today. * pcraft-format: document that a missing or invalid format_version is Corrupt read_manifest's docs and the README say that a missing or invalid format_version gives Error::Corrupt from migrate, and that a missing generator or document, or any other mistyped member, gives Error::Json. A test covers a string and a fractional format_version. * pcraft-format: read_manifest refuses preview names other than its own A manifest's `thumbnail` and `composite` named any entry the reader then opened, while tile entries go through `is_valid_hash`. `read_manifest` now gives `Error::Corrupt` for a `thumbnail` other than `thumb.png` or a `composite` other than `composite/preview.png`, the only names PhotoCraft writes. The check runs after the members are read, so JSON errors keep their order. The reader's doc now says a missing or invalid `format_version` is `Error::Corrupt`, from `migrate`, not `Error::Json`.
115 lines
4.8 KiB
Rust
115 lines
4.8 KiB
Rust
use crate::Error;
|
|
|
|
/// Deepest nesting of layer groups PhotoCraft saves. The value is a copy of PhotoCraft's
|
|
/// `photocraft_doc::MAX_GROUP_DEPTH`; change both together.
|
|
pub const MAX_GROUP_DEPTH: usize = 100;
|
|
|
|
/// Default [`Limits::max_items`]: the most JSON items (arrays and objects opened, plus elements
|
|
/// and members after a comma) a manifest may hold.
|
|
pub const MAX_MANIFEST_ITEMS: usize = 4 << 20;
|
|
|
|
/// The largest [`Limits::max_depth`] the checks apply. A larger value is treated as this one,
|
|
/// so parsing stays within the stack.
|
|
pub const MAX_DEPTH: usize = 400;
|
|
|
|
/// The deepest JSON nesting of a manifest whose layer groups nest `groups` deep: three levels per
|
|
/// group (the layer object, its content object and the `children` array) and 64 for the document
|
|
/// and the leaf layers' fields.
|
|
pub const fn manifest_depth(groups: usize) -> usize {
|
|
3usize.saturating_mul(groups.saturating_add(1)).saturating_add(64)
|
|
}
|
|
|
|
/// Limits applied while reading a bundle from an untrusted source. The defaults are the ones
|
|
/// PhotoCraft loads with. [`check_manifest`], [`parse_manifest`] and [`crate::read_manifest`]
|
|
/// apply `max_manifest_bytes`, `max_depth` and `max_items`; the caller applies `max_blob_bytes`
|
|
/// and `max_total_bytes`.
|
|
///
|
|
/// The struct is `#[non_exhaustive]`: start from [`Limits::default`] and set the fields you
|
|
/// change.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
#[non_exhaustive]
|
|
pub struct Limits {
|
|
/// Largest `manifest.json`, in bytes (256 MiB).
|
|
pub max_manifest_bytes: usize,
|
|
/// Largest decompressed blob, in bytes (1 GiB). The crate does not read blobs or apply this
|
|
/// limit: the caller that decompresses them enforces it.
|
|
pub max_blob_bytes: usize,
|
|
/// Most decompressed tile and blob bytes in one bundle (16 GiB). The crate does not read
|
|
/// tiles or blobs or apply this limit: the caller that decompresses them enforces it.
|
|
pub max_total_bytes: u64,
|
|
/// Deepest JSON nesting of the manifest (`manifest_depth(MAX_GROUP_DEPTH)`, 367). Values
|
|
/// above [`MAX_DEPTH`] are treated as [`MAX_DEPTH`].
|
|
pub max_depth: usize,
|
|
/// Most JSON items in the manifest ([`MAX_MANIFEST_ITEMS`]).
|
|
pub max_items: usize,
|
|
}
|
|
|
|
impl Default for Limits {
|
|
fn default() -> Self {
|
|
Limits {
|
|
max_manifest_bytes: 256 << 20,
|
|
max_blob_bytes: 1 << 30,
|
|
max_total_bytes: 16 << 30,
|
|
max_depth: manifest_depth(MAX_GROUP_DEPTH),
|
|
max_items: MAX_MANIFEST_ITEMS,
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Checks manifest JSON against [`Limits::max_manifest_bytes`], [`Limits::max_depth`] (at most
|
|
/// [`MAX_DEPTH`]) and
|
|
/// [`Limits::max_items`] in one pass without recursion. Brackets and commas inside strings are
|
|
/// not counted. Returns [`Error::LimitExceeded`] for the first limit passed.
|
|
///
|
|
/// PhotoCraft's own check of a manifest it is about to save tests the depth and the items only.
|
|
/// This check also refuses JSON larger than [`Limits::max_manifest_bytes`], so a writer that runs
|
|
/// it before saving refuses a manifest of more than that size.
|
|
pub fn check_manifest(json: &[u8], limits: &Limits) -> Result<(), Error> {
|
|
if json.len() > limits.max_manifest_bytes {
|
|
return Err(Error::LimitExceeded(format!("`{}` exceeds {} bytes", crate::MANIFEST, limits.max_manifest_bytes)));
|
|
}
|
|
let max_depth = limits.max_depth.min(MAX_DEPTH);
|
|
let (mut depth, mut items, mut in_str, mut escaped) = (0usize, 0usize, false, false);
|
|
for &b in json {
|
|
if in_str {
|
|
match b {
|
|
_ if escaped => escaped = false,
|
|
b'\\' => escaped = true,
|
|
b'"' => in_str = false,
|
|
_ => {}
|
|
}
|
|
continue;
|
|
}
|
|
match b {
|
|
b'"' => in_str = true,
|
|
b'[' | b'{' => {
|
|
depth += 1;
|
|
if depth > max_depth {
|
|
return Err(Error::LimitExceeded(format!("manifest JSON nested deeper than {max_depth} levels")));
|
|
}
|
|
}
|
|
b']' | b'}' => depth = depth.saturating_sub(1),
|
|
_ => {}
|
|
}
|
|
if matches!(b, b'[' | b'{' | b',') {
|
|
items += 1;
|
|
if items > limits.max_items {
|
|
return Err(Error::LimitExceeded(format!("manifest JSON holds more than {} values", limits.max_items)));
|
|
}
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Checks manifest JSON with [`check_manifest`], then parses it into a [`serde_json::Value`]
|
|
/// with serde_json's recursion limit turned off, so a manifest nested deeper than 128 levels
|
|
/// and within [`Limits::max_depth`] parses.
|
|
pub fn parse_manifest(json: &[u8], limits: &Limits) -> Result<serde_json::Value, Error> {
|
|
check_manifest(json, limits)?;
|
|
let mut de = serde_json::Deserializer::from_slice(json);
|
|
de.disable_recursion_limit();
|
|
let value = <serde_json::Value as serde::Deserialize>::deserialize(&mut de)?;
|
|
de.end()?;
|
|
Ok(value)
|
|
}
|