From 8d3f245d2a5cf6431642ed8302244b815d514849 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:03:44 -0500 Subject: [PATCH 01/10] craft-metadata: validation and location fixes ported from open LightCraft pull requests (#1042) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * craft-metadata: EXIF, IPTC-IIM, XMP and GPX from LightCraft lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif (read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse, write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers. They share Metadata, DateTime, Gps and Orientation. Changed in the move: - TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own. - The app namespace is a parameter: parse_xmp_with, write_xmp_with and MergeRules::namespaces take it, and XmpData no longer has lc_settings. - The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and craft-raw read those containers; from_embedded takes the blocks they find. * craft-metadata: a RW2 reads ISO and the rest of its Exif from the JpgFromRaw A RW2's own Exif IFD is a short subset. What it lacks is filled from the Exif of the embedded JpgFromRaw, then from the ISO tags of the Panasonic IFD0 (0x0017, 0x0037), each only in its recorded field type. 65535 in a SHORT ISOSpeedRatings is the overflow marker and states no ISO. The private tags are read only in a RW2 stream, under the caller's parse options. Ported from storytold/lightcraft#3518. * craft-metadata: the Exif rating is written, and 65535 reads as rejected write_exif writes Rating (0x4746), with 0xFFFF for a rejected photo, and from_tiff reads 65535 as -1 instead of clamping it to 5. Ported from storytold/lightcraft#1995. * craft-metadata: one shutter-speed formatter, exact to four digits format_exposure_time shows a duration within 0.1% of 1/n second as 1/n and anything else to four significant digits, so 0.3 s, 1/2.8 s and 0.625 s are no longer rounded to a reciprocal or one decimal. Metadata::exposure_display and the ExposureTime tag row both use it. Ported from storytold/lightcraft#1692. * craft-metadata: JPEG XL containers give their Exif and xml boxes The first Exif and xml box of a JPEG XL container, plain or Brotli-wrapped in brob, honouring the Exif box's TIFF header offset. A brob payload is capped at 32 MiB. A bare codestream has no metadata. Ported from storytold/lightcraft#3756. * craft-metadata: XMP Resource structs flatten their attribute fields A property with rdf:parseType="Resource" may give its fields as attributes as well as child elements. Only the children were flattened into XmpData::properties; the attributes now are too, as for a nested rdf:Description. Ported from storytold/lightcraft#1226 (the XMP part). * craft-metadata: a capture date of just a year or a month stays as stated XMP dates may stop at the year (1983) or the month (1983-04). DateTime now has a DatePrecision; such a date writes back to XMP as stated and only in photoshop:DateCreated, and write_exif leaves it out, since an Exif date cannot say it. Ported from storytold/lightcraft#3502. * craft-metadata: Fujifilm in-camera recipes from the maker note fuji module: the film simulation and the rest of the in-camera recipe (grain, colour chrome, tone, dynamic range, white balance and its shift, sharpness, noise reduction, clarity) from a Fujifilm maker note. recipe_from_exif, recipe_from_bytes (TIFF, JPEG, RAF) and read_recipe_file, which reads only as far as the metadata. from_embedded fills Metadata::fuji_recipe and file_tag_rows lists the recipe under Fujifilm. Ported from storytold/lightcraft#1209. * craft-metadata: the linear-time scan test compares times, not a fixed bound The test asserted that 200 000 attributes scan in under 5 s, which failed on a loaded machine. It now scans 25 000 and 100 000 attributes, takes the best of three runs of each, and asserts that four times the attributes take less than ten times as long. Linear scans give about 4; a quadratic duplicate check gave 18.6 and fails. * craft-metadata: interleave the linear-time scan test's runs The best-of-three ratio still failed once under load. The test now alternates five runs of 50 000 and 200 000 attributes and compares the fastest of each, so a burst of load slows both sizes. Six runs at load average 55 gave ratios of 3.9 to 5.4 against the bound of 10. * craft-metadata: XmpData::toolkit, the x:xmptk of the packet The name of the program that wrote a packet, from x:xmptk on x:xmpmeta (or x:xaptk on the older x:xapmeta), whatever prefix the packet binds to adobe:ns:meta/. write_xmp_with writes XmpWrite::toolkit there, and the test reads it back, escaped characters included. * craft-metadata: mpf, the Multi-Picture index of a JPEG mpf::images lists the images an APP2 MPF index (CIPA DC-007) puts after the primary image, with their file offsets; the index is read with craft-tiff, in either byte order, up to 16 entries. mpf::write inserts an index after the primary's leading APPn segments and appends the other images, with the primary listed as the representative baseline image: the layout LightCraft's gain map JPEG writer uses, for any number of images. mpf::markers lists the marker segments before the first scan, for APPn payloads such as ISO 21496-1 gain map metadata. * craft-metadata: edit, byte-preserving Exif and XMP edits from PhotoCraft PhotoCraft's orientation, resolution and Exif colour-space code (codecs orientation.rs without the pixel transforms, resolution.rs, exif_color.rs), at photocraft f1b5e7d72, as the edit module: - upright_exif and upright_xmp set Orientation to 1 in place, in the entry's own type and width, every well-formed duplicate included. - exif_with_resolution and xmp_with_resolution set the resolution in place and remove entries that cannot be rewritten; the _adding_missing variant appends a new IFD0 when there are none. - export_exif and export_xmp do both and drop the source Exif thumbnail, with the overlap scan bounded. - exif_orientation, exif_resolution, xmp_resolution, photoshop_resolution, exif_color_space and resolution read the same values with the editors' rules (first entry wins, SHORT or LONG only, BigTIFF accepted). The tests that call these functions directly move with them (the codec round trips stay in PhotoCraft): the orientation, resolution, thumbnail and compaction tests, the comparison-counting complexity test and the colour space tests. * craft-metadata: exif_with_pixel_size, the written size in PixelX/YDimension A camera records its sensor's size in the Exif IFD's PixelXDimension and PixelYDimension, which after a crop, resize or rotation names a size the exported file does not have. exif_with_pixel_size rewrites each one that holds one SHORT or LONG, in place and as a LONG, in classic and BigTIFF blocks, with the JPEG Exif prefix kept. Ported from storytold/photocraft#12407 (the format part). * craft-metadata: an ISO 8601 clock needs two-digit hours and minutes Ported from storytold/lightcraft#3523. * craft-metadata: a UTC offset needs a two-digit minute field Ported from storytold/lightcraft#3467. * craft-metadata: tiff:Orientation is read only as an integral code 1 to 8 A fractional or out-of-range value stays a raw property and does not become the photo's orientation. Ported from storytold/lightcraft#3495. * craft-metadata: an XMP ISO is read only when it is a whole number in range A fractional, non-positive or out-of-range ISO is not truncated or saturated into one; the next candidate property is tried instead. Ported from storytold/lightcraft#3633. * craft-metadata: XMP pixel dimensions are read only as exact whole numbers A malformed, fractional, negative or out-of-range tiff:ImageWidth or ImageLength is no longer truncated or saturated, and lets the exif:Pixel dimension take its place. Numbers are compared as exact decimal digits, so wide rationals stay exact. The ISO reader now uses the same exact reader (both PRs added one). Ported from storytold/lightcraft#3690. * craft-metadata: XMP GPS minutes and seconds must be under 60; tell when a read was cut short A coordinate such as 45,60N is no longer normalised to a plausible point; the raw property stays. XmpData::properties_truncated says the flattened property map ran past its key budget, Embedded::chunk_walk_incomplete says a PNG or WebP chunk walk stopped early, and jpeg_segments_with_all_xmp returns every ordinary XMP packet of a JPEG, so a caller that must prove a file carries no location can refuse what it could not inspect. Ported from storytold/lightcraft#3054. * craft-metadata: IPTC Time Created counts only when it is a whole, valid time A date with a six-digit clock and no zone stays date-only; HHMMSS±HHMM carries an offset; clocks and offsets are range-checked. A partial offset, extra digits, a leap second or trailing text no longer becomes a capture time, so it cannot block valid XMP or Exif. Ported from storytold/lightcraft#3448. * craft-metadata: test hostile XMP attribute references The one-pass reference scan (entity_end) already covers the bug that storytold/lightcraft#3007 fixed another way; its test is added here and passes on this implementation: unknown, unterminated and far-away references stay verbatim, a value of a million ampersands or of a million zeros is read once, and multi-byte text around a reference stays whole. Test from storytold/lightcraft#3007. * craft-metadata: parse_gps_position; an XMP coordinate's sign comes from its letter parse_gps_position reads an exif:GPSLatitude and GPSLongitude pair and checks both axes. A negative degree, minute or second part, or a letter of the other axis (10,30E as a latitude), is malformed; the letter may be either case. Ported from storytold/lightcraft#2088 (the format part). * craft-metadata: bound the XMP namespace declarations in scope; merge into an empty rdf:RDF Prefix lookup scans every declaration in scope, so one start tag with thousands of xmlns declarations made parsing and merging quadratic. More than 512 declarations in scope is now TooComplex, in parse_xmp and in merge_xmp's scan. merge_xmp also opens a self-closing around the new description instead of failing. Ported from storytold/lightcraft#3696. * craft-metadata: XmpData::has_location_metadata, every embedded XMP location has_location_metadata answers whether a packet names any Exif GPS property or a non-blank place (IPTC Core location fields, photoshop City/State/Country, IPTC Extension LocationShown and LocationCreated with a place or coordinates), including inside array items and repeated structs that the flattened map and the structured values keep only once (XmpData::names_location, from the parsed tree). The IPTC Extension namespace is now canonicalised and written. extended_xmp_guid reports the announced Extended XMP GUID. A caller that must prove a file has no location checks properties_truncated first. The source also listed LightCraft's own lc:location; a caller with its own namespace checks that itself. Ported from storytold/lightcraft#4121 (the format part). * craft-metadata: is_lens_spec_name, the inverse check of lens_spec_name Whether a lens name has the ' mm f/' shape lens_spec_name writes, so a caller can tell a made-up name from a camera's. The test is new. Ported from storytold/lightcraft#2644 (the format part). * craft-metadata: Sony focus points for 44 bodies SONY_MODELS grows from the alpha 1 and A7S III to the 44 bodies whose focus tags were checked on the public raw.pixls.us samples against ExifTool's reading; bodies whose 0x201b is not the focus mode stay out. This also covers the ILCE-6400 and ILCE-7M3 that storytold/lightcraft#3109 adds. Ported from storytold/lightcraft#1891 (the focus part). * craft-metadata: GPX points say which track they are on; tracks keep their names TrackPoint::track counts the a point came from, Tracklog::names holds each track's (trimmed, at most 200 characters), and only_track gives one track's points, so a map can draw a file's tracks apart and tag from one. At most 10 000 tracks per file (TooComplex past that); a non-finite elevation is dropped. Ported from storytold/lightcraft#921 (the GPX part; #916 carries the same). * craft-metadata: test that from_embedded keeps the EXIF, IPTC, XMP precedence storytold/lightcraft#2211 adds the same function as extract_embedded; this crate already has it as from_embedded. Its tests are added: EXIF camera fields stay, IPTC fills gaps, XMP overrides the user fields, and damaged blocks read as nothing. Tests from storytold/lightcraft#2211. * craft-metadata: the XMP writer keeps a carriage return as XML end-of-line handling turns a literal CR into a line feed when a packet is read, so text with one did not round-trip through other readers. Element text now writes it as a character reference. Ported from storytold/lightcraft#4119 (the writer fix only; that PR also moves LightCraft to quick-xml 0.42, which this crate does not take here, as craft-xml uses 0.41). * craft-metadata: README rows for the ported validation and location checks --------- Co-authored-by: radialmonster Co-authored-by: Sebastian Berschneider Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Michael Anderson <1427851+endthestart@users.noreply.github.com> Co-authored-by: Ilya Dorman Co-authored-by: Clyde C Brown Co-authored-by: MohammadTeimori Co-authored-by: vangel76 Co-authored-by: Max Mittelstädt <62167861+TheGalitube@users.noreply.github.com> Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com> Co-authored-by: ronibalthazar <61232335+ronibalthazar@users.noreply.github.com> Co-authored-by: Andrew Frahn Co-authored-by: jkalkstein <76634771+jkalkstein@users.noreply.github.com> Co-authored-by: d-rouge Co-authored-by: espoon Co-authored-by: Yoann Laissus Co-authored-by: Jason Baldwin <33934381+bitshiftza@users.noreply.github.com> Co-authored-by: ingolfteetz Co-authored-by: smi-leonivkovic --- crates/metadata/README.md | 9 +- crates/metadata/src/container.rs | 352 +++++++++++++++++++- crates/metadata/src/datetime.rs | 27 +- crates/metadata/src/focus.rs | 61 +++- crates/metadata/src/focus/tests.rs | 15 +- crates/metadata/src/gpx.rs | 95 +++++- crates/metadata/src/iptc.rs | 127 +++++++- crates/metadata/src/lib.rs | 76 ++++- crates/metadata/src/xmp.rs | 506 ++++++++++++++++++++++++++--- crates/metadata/src/xmp/tests.rs | 473 ++++++++++++++++++++++++++- crates/metadata/src/xmp_merge.rs | 34 +- 11 files changed, 1683 insertions(+), 92 deletions(-) diff --git a/crates/metadata/README.md b/crates/metadata/README.md index 309f29638..72ea0063d 100644 --- a/crates/metadata/README.md +++ b/crates/metadata/README.md @@ -18,8 +18,9 @@ are one crate. | `parse_iptc(&[u8]) -> Metadata` | IIM record 2: title (2:5), keywords (2:25), by-line (2:80), copyright (2:116), caption (2:120), date and time created (2:55, 2:60). | | `parse_xmp(&str) -> Result` | `Metadata`, every property (`prefix:name`, structs flattened as `a:b/c:d`) and the structured values. `parse_xmp_with` also canonicalises the caller's own namespaces. | | `write_xmp(&Metadata) -> String` | A complete `` packet. `write_xmp_with` sets the toolkit name and adds simple properties in the caller's namespace. | +| `XmpData::has_location_metadata`, `XmpData::properties_truncated` | Whether a packet names GPS or a place anywhere (array items and repeated structs included), and whether the flattened map is complete, for a caller that must prove a file carries no location. `parse_gps_position` reads an XMP coordinate pair. | | `merge_xmp(existing, fresh, MergeRules)` | Updates a packet another program wrote: owned properties are replaced, everything else stays byte for byte. | -| `parse_gpx(&str) -> Result` | GPX track points; `Tracklog::locate` gives the position at a capture time. | +| `parse_gpx(&str) -> Result` | GPX track points with their track and segment; `Tracklog::locate` gives the position at a capture time, `Tracklog::only_track` one track. | | `embedded(&[u8]) -> Embedded` | Sniffs JPEG, PNG, WebP, TIFF-based files, Fujifilm RAF, Canon CR3 and JPEG XL containers and returns the Exif, XMP (and extended XMP), ICC and IPTC blocks. `jpeg_segments`, `png_chunks`, `webp_chunks` are the walkers. | | `cr3::parse_cr3` | The Canon CR3 container: `CMT1`…`CMT4`, the XMP `uuid` box, the thumbnail, the tracks. | | `jxl::metadata` | The `Exif` and `xml ` boxes of a JPEG XL container, plain or Brotli-wrapped in `brob`. | @@ -30,6 +31,10 @@ are one crate. HEIF, PSD and MRW files are read by `craft-heif`, `craft-psd` and `craft-raw`. Put the blocks they return into an `Embedded` and call `from_embedded`. +## Validation + +Only well-formed values become typed metadata; the raw property stays in `XmpData::properties`. XMP orientation must be an integral code 1 to 8; ISO and pixel dimensions exact whole numbers in range (compared as decimal digits); GPS minutes and seconds under 60, the sign from the axis letter. IPTC Time Created counts only as a whole `HHMMSS` or `HHMMSS±HHMM`. A carriage return in written text is ` `. + ## Capture time EXIF candidates are tried in order: DateTimeOriginal, DateTimeDigitized, then IFD0 DateTime. A clock @@ -50,7 +55,7 @@ A foreign namespace that uses a recognised prefix cannot supply recognised prope ## Limits -XMP: 64 levels, 200 000 nodes, 1 MiB of flattened keys and 8 MiB of opaque names per packet +XMP: 64 levels, 200 000 nodes, 512 namespace declarations in scope, 1 MiB of flattened keys and 8 MiB of opaque names per packet (`XmpError::TooComplex`). GPX: 64 levels, 5 000 000 points. MPF: 16 index entries. CR3 and JPEG XL: 4096 boxes; a `brob` box inflates to at most 32 MiB. JPEG ICC is returned only when its APP2 chunks form a complete, consistent, uniquely numbered sequence. TIFF limits are `craft-tiff`'s. No input panics: proptests feed random bytes to every container walker and random text to the XMP parser, and a sample diff --git a/crates/metadata/src/container.rs b/crates/metadata/src/container.rs index 439771b98..bb787c545 100644 --- a/crates/metadata/src/container.rs +++ b/crates/metadata/src/container.rs @@ -25,9 +25,14 @@ pub struct Embedded { pub xmp: Option, /// Extended XMP (JPEG only), reassembled. pub xmp_extended: Option, + /// The GUID every Extended XMP segment carries, set only when all segments share it and + /// their chunks tile exactly the full length they declare (no gap, overlap or excess). + pub xmp_extended_guid: Option, pub icc: Option>, /// IPTC-IIM stream. pub iptc: Option>, + /// True when the PNG/WebP chunk stream could not be fully inspected. + pub chunk_walk_incomplete: bool, } /// Sniff the container and collect its metadata blocks. Unknown containers yield an empty result. @@ -58,11 +63,13 @@ pub fn embedded(bytes: &[u8]) -> Embedded { exif: Some(bytes.to_vec()), xmp: ifd0.bytes(tags::XMP).map(|b| String::from_utf8_lossy(b).into_owned()), xmp_extended: None, + xmp_extended_guid: None, icc: ifd0.bytes(tags::ICC_PROFILE).map(|b| b.to_vec()), iptc: ifd0.value(tags::IPTC_NAA).map(|v| match v { craft_tiff::Value::Byte(b) | craft_tiff::Value::Undefined(b) => b.clone(), other => craft_tiff::writer::encode(t.order, other), }), + chunk_walk_incomplete: false, } } else { Embedded::default() @@ -74,14 +81,27 @@ const XMP_EXT_SIG: &[u8] = b"http://ns.adobe.com/xmp/extension/\0"; /// Walk JPEG markers up to SOS and collect APP1 Exif / XMP, APP2 ICC and APP13 IPTC. pub fn jpeg_segments(b: &[u8]) -> Embedded { + jpeg_segments_inner(b, false).0 +} + +/// Walk JPEG metadata and return every ordinary XMP APP1 packet in segment order, along with the +/// usual [`Embedded`] projection (which retains only the first ordinary packet). +pub fn jpeg_segments_with_all_xmp(b: &[u8]) -> (Embedded, Vec) { + jpeg_segments_inner(b, true) +} + +fn jpeg_segments_inner(b: &[u8], collect_all_xmp: bool) -> (Embedded, Vec) { let mut out = Embedded::default(); + let mut xmp_packets = Vec::new(); if !b.starts_with(&[0xff, 0xd8]) { - return out; + return (out, xmp_packets); } let mut icc_parts: [Option<&[u8]>; 255] = [None; 255]; let mut icc_total = None; let mut icc_invalid = false; let mut ext: Vec<(u32, Vec)> = Vec::new(); + // GUID and declared full length of every Extended XMP segment, to verify they belong together + let mut ext_headers: Vec<(&[u8], u32)> = Vec::new(); let mut i = 2usize; while i + 4 <= b.len() { if b[i] != 0xff { @@ -111,13 +131,22 @@ pub fn jpeg_segments(b: &[u8]) -> Embedded { match marker { 0xe1 if payload.starts_with(b"Exif\0") && out.exif.is_none() => out.exif = Some(payload.get(6..).unwrap_or(&[]).to_vec()), 0xe1 if payload.starts_with(XMP_SIG) && out.xmp.is_none() => { - out.xmp = Some(String::from_utf8_lossy(&payload[XMP_SIG.len()..]).into_owned()); + let packet = String::from_utf8_lossy(&payload[XMP_SIG.len()..]).into_owned(); + if collect_all_xmp { + xmp_packets.push(packet.clone()); + } + out.xmp = Some(packet); + } + 0xe1 if payload.starts_with(XMP_SIG) && collect_all_xmp => { + xmp_packets.push(String::from_utf8_lossy(&payload[XMP_SIG.len()..]).into_owned()); } 0xe1 if payload.starts_with(XMP_EXT_SIG) => { // GUID (32) + full length (4) + offset (4) + data let p = &payload[XMP_EXT_SIG.len()..]; if p.len() >= 40 { + let full = u32::from_be_bytes([p[32], p[33], p[34], p[35]]); let off = u32::from_be_bytes([p[36], p[37], p[38], p[39]]); + ext_headers.push((&p[..32], full)); ext.push((off, p[40..].to_vec())); } } @@ -182,9 +211,27 @@ pub fn jpeg_segments(b: &[u8]) -> Embedded { } if !ext.is_empty() { ext.sort_by_key(|p| p.0); + out.xmp_extended_guid = verified_extension_guid(&ext_headers, &ext); out.xmp_extended = Some(String::from_utf8_lossy(&ext.into_iter().flat_map(|p| p.1).collect::>()).into_owned()); } - out + (out, xmp_packets) +} + +/// The GUID shared by all Extended XMP segments when their offset-sorted chunks are contiguous from +/// zero and add up to the full length every segment declares; `None` otherwise. +fn verified_extension_guid(headers: &[(&[u8], u32)], chunks_by_offset: &[(u32, Vec)]) -> Option { + let (&(guid, full), rest) = headers.split_first()?; + if rest.iter().any(|&(other_guid, other_full)| other_guid != guid || other_full != full) { + return None; + } + let mut next = 0u64; + for (offset, data) in chunks_by_offset { + if u64::from(*offset) != next { + return None; + } + next = next.checked_add(u64::try_from(data.len()).ok()?)?; + } + (next == u64::from(full)).then(|| String::from_utf8_lossy(guid).into_owned()) } /// IPTC-IIM block (resource 0x0404) from a Photoshop image-resource stream (`8BIM` records). @@ -209,10 +256,41 @@ fn photoshop_iptc(mut b: &[u8]) -> Option> { pub fn png_chunks(b: &[u8]) -> Embedded { let mut out = Embedded::default(); let mut i = 8usize; - while i + 12 <= b.len() { - let len = u32::from_be_bytes([b[i], b[i + 1], b[i + 2], b[i + 3]]) as usize; - let typ = &b[i + 4..i + 8]; - let Some(data) = b.get(i + 8..(i + 8).saturating_add(len)) else { break }; + let mut ended = false; + loop { + let Some(header_end) = i.checked_add(8).filter(|end| *end <= b.len()) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(header) = b.get(i..header_end) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(length_bytes) = header.get(..4) else { + out.chunk_walk_incomplete = true; + break; + }; + let Ok(length_bytes) = <[u8; 4]>::try_from(length_bytes) else { + out.chunk_walk_incomplete = true; + break; + }; + let len = u32::from_be_bytes(length_bytes) as usize; + let Some(data_end) = header_end.checked_add(len) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(chunk_end) = data_end.checked_add(4).filter(|end| *end <= b.len()) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(typ) = header.get(4..8) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(data) = b.get(header_end..data_end) else { + out.chunk_walk_incomplete = true; + break; + }; match typ { b"eXIf" => out.exif = Some(crate::strip_exif_header(data).to_vec()), b"iTXt" if data.starts_with(b"XML:com.adobe.xmp\0") => { @@ -227,10 +305,17 @@ pub fn png_chunks(b: &[u8]) -> Embedded { } } } - b"IEND" => break, + b"IEND" => { + ended = len == 0 && chunk_end == b.len(); + out.chunk_walk_incomplete |= !ended; + break; + } _ => {} } - i += 12 + len; + i = chunk_end; + } + if !ended { + out.chunk_walk_incomplete = true; } out } @@ -238,18 +323,69 @@ pub fn png_chunks(b: &[u8]) -> Embedded { /// WebP RIFF chunks `EXIF`, `XMP `, `ICCP`. pub fn webp_chunks(b: &[u8]) -> Embedded { let mut out = Embedded::default(); + let Some(size_bytes) = b.get(4..8) else { + out.chunk_walk_incomplete = true; + return out; + }; + let Ok(size_bytes) = <[u8; 4]>::try_from(size_bytes) else { + out.chunk_walk_incomplete = true; + return out; + }; + let riff_size = u32::from_le_bytes(size_bytes) as usize; + let Some(riff_end) = 8usize.checked_add(riff_size) else { + out.chunk_walk_incomplete = true; + return out; + }; + if riff_size < 4 || riff_end < 12 { + out.chunk_walk_incomplete = true; + return out; + } + if riff_end != b.len() { + out.chunk_walk_incomplete = true; + } + let end = riff_end.min(b.len()); let mut i = 12usize; - while i + 8 <= b.len() { - let id = &b[i..i + 4]; - let len = u32::from_le_bytes([b[i + 4], b[i + 5], b[i + 6], b[i + 7]]) as usize; - let Some(data) = b.get(i + 8..(i + 8).saturating_add(len)) else { break }; + while i < end { + let Some(header_end) = i.checked_add(8).filter(|next| *next <= end) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(header) = b.get(i..header_end) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(length_bytes) = header.get(4..8) else { + out.chunk_walk_incomplete = true; + break; + }; + let Ok(length_bytes) = <[u8; 4]>::try_from(length_bytes) else { + out.chunk_walk_incomplete = true; + break; + }; + let len = u32::from_le_bytes(length_bytes) as usize; + let Some(data_end) = header_end.checked_add(len) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(chunk_end) = data_end.checked_add(len & 1).filter(|next| *next <= end) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(id) = header.get(..4) else { + out.chunk_walk_incomplete = true; + break; + }; + let Some(data) = b.get(header_end..data_end) else { + out.chunk_walk_incomplete = true; + break; + }; match id { b"EXIF" => out.exif = Some(crate::strip_exif_header(data).to_vec()), b"XMP " => out.xmp = Some(String::from_utf8_lossy(data).into_owned()), b"ICCP" => out.icc = Some(data.to_vec()), _ => {} } - i += 8 + len + (len & 1); + i = chunk_end; } out } @@ -258,7 +394,7 @@ pub fn webp_chunks(b: &[u8]) -> Embedded { mod tests { use super::*; use crate::exif::tests::sample_exif; - use craft_tiff::ByteOrder; + use craft_tiff::{ByteOrder, IfdBuilder, TiffWriter, Value}; fn seg(marker: u8, payload: &[u8]) -> Vec { let mut v = vec![0xff, marker]; @@ -329,6 +465,150 @@ mod tests { assert_eq!(m.caption.as_deref(), Some("A caption")); } + fn jpeg_with_orientation(exif_orientation: Option, xmp_orientation: &str) -> Vec { + let mut jpeg = vec![0xff, 0xd8]; + if let Some(orientation) = exif_orientation { + let mut exif = b"Exif\0\0".to_vec(); + exif.extend(crate::exif::write_exif(&crate::Metadata { orientation: Some(orientation), ..Default::default() })); + jpeg.extend(seg(0xe1, &exif)); + } + let packet = format!( + r#""# + ); + let mut xmp = XMP_SIG.to_vec(); + xmp.extend_from_slice(packet.as_bytes()); + jpeg.extend(seg(0xe1, &xmp)); + jpeg.extend_from_slice(&[0xff, 0xd9]); + jpeg + } + + #[test] + fn embedded_xmp_orientation_requires_integral_value_and_respects_exif() { + let malformed = crate::extract(&jpeg_with_orientation(None, "6.9")); + assert_eq!(malformed.orientation, None); + + let malformed_xmp_with_exif = crate::extract(&jpeg_with_orientation(Some(crate::Orientation::Rotate180), "6.5")); + assert_eq!(malformed_xmp_with_exif.orientation.map(|orientation| orientation.to_exif()), Some(3)); + + let valid_xmp = crate::extract(&jpeg_with_orientation(None, "6")); + assert_eq!(valid_xmp.orientation.map(|orientation| orientation.to_exif()), Some(6)); + + let valid_exif = crate::extract(&jpeg_with_orientation(Some(crate::Orientation::Rotate180), "6")); + assert_eq!(valid_exif.orientation.map(|orientation| orientation.to_exif()), Some(3)); + } + + #[test] + fn jpeg_invalid_xmp_iso_stays_absent_and_exif_iso_wins() { + let make_jpeg = |exif: Option>, xmp: &str| { + let mut jpeg = vec![0xff, 0xd8]; + if let Some(exif) = exif { + let mut payload = b"Exif\0\0".to_vec(); + payload.extend(exif); + jpeg.extend(seg(0xe1, &payload)); + } + let mut payload = XMP_SIG.to_vec(); + payload.extend_from_slice(xmp.as_bytes()); + jpeg.extend(seg(0xe1, &payload)); + jpeg.extend_from_slice(&[0xff, 0xda, 0, 2, 0xff, 0xd9]); + jpeg + }; + let invalid = r#""#; + assert_eq!(crate::extract(&make_jpeg(None, invalid)).iso, None); + + let valid = r#""#; + assert_eq!(crate::extract(&make_jpeg(Some(sample_exif(ByteOrder::Big)), valid)).iso, Some(400)); + } + + #[test] + fn jpeg_rejected_xmp_dimensions_keep_exif_dimensions_and_other_metadata() { + let mut jpeg = vec![0xff, 0xd8]; + let mut exif = b"Exif\0\0".to_vec(); + exif.extend(crate::exif::tests::sample_exif(ByteOrder::Big)); + jpeg.extend(seg(0xe1, &exif)); + let packet = r#" + + "#; + let mut xmp = XMP_SIG.to_vec(); + xmp.extend_from_slice(packet.as_bytes()); + jpeg.extend(seg(0xe1, &xmp)); + jpeg.extend_from_slice(&[0xff, 0xda, 0, 2]); + + let metadata = crate::extract(&jpeg); + assert_eq!(metadata.width, Some(6000)); + assert_eq!(metadata.height, Some(4000)); + assert_eq!(metadata.title.as_deref(), Some("Retained")); + } + + #[test] + fn jpeg_rejected_xmp_dimension_falls_back_during_whole_file_extract() { + let mut jpeg = vec![0xff, 0xd8]; + let packet = r#" + + "#; + let mut xmp = XMP_SIG.to_vec(); + xmp.extend_from_slice(packet.as_bytes()); + jpeg.extend(seg(0xe1, &xmp)); + jpeg.extend_from_slice(&[0xff, 0xda, 0, 2]); + + assert_eq!(crate::extract(&jpeg).width, Some(800)); + } + + #[test] + fn jpeg_invalid_iptc_time_created_allows_xmp_and_exif_fallbacks() { + fn iim() -> Vec { + let mut data = Vec::new(); + for (dataset, value) in [(55, &b"20260101"[..]), (60, b"121314+01"), (5, b"retained title")] { + data.extend_from_slice(&[0x1c, 2, dataset]); + data.extend_from_slice(&(value.len() as u16).to_be_bytes()); + data.extend_from_slice(value); + } + data + } + + fn jpeg(xmp: &str, exif: Option<&[u8]>) -> Vec { + let mut jpeg = vec![0xff, 0xd8]; + if let Some(exif) = exif { + let mut payload = b"Exif\0\0".to_vec(); + payload.extend_from_slice(exif); + jpeg.extend_from_slice(&seg(0xe1, &payload)); + } + let mut payload = XMP_SIG.to_vec(); + payload.extend_from_slice(xmp.as_bytes()); + jpeg.extend_from_slice(&seg(0xe1, &payload)); + let mut resource = b"Photoshop 3.0\0".to_vec(); + resource.extend_from_slice(b"8BIM\x04\x04\0\0"); + let iim = iim(); + resource.extend_from_slice(&(iim.len() as u32).to_be_bytes()); + resource.extend_from_slice(&iim); + if iim.len() % 2 == 1 { + resource.push(0); + } + jpeg.extend_from_slice(&seg(0xed, &resource)); + jpeg.extend_from_slice(&[0xff, 0xda, 0, 2]); + jpeg + } + + let xmp = crate::write_xmp(&crate::Metadata { + capture_time: Some(crate::DateTime { year: 2026, month: 1, day: 2, hour: 10, offset_minutes: Some(0), ..Default::default() }), + ..Default::default() + }); + let fallback = jpeg(&xmp, None); + let embedded = embedded(&fallback); + assert!(embedded.iptc.is_some(), "raw IIM remains available"); + assert_eq!(crate::extract(&fallback).capture_time.map(|time| time.to_iso()).as_deref(), Some("2026-01-02T10:00:00+00:00")); + assert_eq!(crate::extract(&fallback).title.as_deref(), Some("retained title")); + + let mut exif_ifd = IfdBuilder::new(); + exif_ifd.set(0x9003, Value::Ascii("2026:01:03 11:12:13".into())); + let mut ifd0 = IfdBuilder::new(); + ifd0.set_child(0x8769, exif_ifd); + let exif = TiffWriter::new(ByteOrder::Little, false).write(&[ifd0]).unwrap(); + let exif_precedence = jpeg(&xmp, Some(&exif)); + assert_eq!(crate::extract(&exif_precedence).capture_time.map(|time| time.to_iso()).as_deref(), Some("2026-01-03T11:12:13")); + } + #[test] fn jpeg_extended_xmp() { let mut j = vec![0xff, 0xd8]; @@ -341,6 +621,28 @@ mod tests { j.extend(seg(0xe1, &p)); } assert_eq!(jpeg_segments(&j).xmp_extended.as_deref(), Some("helloworld")); + assert_eq!(jpeg_segments(&j).xmp_extended_guid, Some("A".repeat(32))); + } + + /// A GUID mix, a short chunk set or a wrong declared length leaves the extension unverified. + #[test] + fn jpeg_extended_xmp_guid_needs_matching_segments_and_length() { + let build = |segments: &[(u8, u32, u32, &[u8])]| { + let mut j = vec![0xff, 0xd8]; + for &(guid, full, off, part) in segments { + let mut p = XMP_EXT_SIG.to_vec(); + p.extend_from_slice(&[guid; 32]); + p.extend_from_slice(&full.to_be_bytes()); + p.extend_from_slice(&off.to_be_bytes()); + p.extend_from_slice(part); + j.extend(seg(0xe1, &p)); + } + jpeg_segments(&j).xmp_extended_guid + }; + assert!(build(&[(b'A', 10, 0, b"hello"), (b'B', 10, 5, b"world")]).is_none(), "two GUIDs"); + assert!(build(&[(b'A', 10, 0, b"hello")]).is_none(), "half the declared length"); + assert!(build(&[(b'A', 9, 0, b"hello"), (b'A', 9, 5, b"world")]).is_none(), "longer than declared"); + assert!(build(&[(b'A', 10, 0, b"hello"), (b'A', 10, 4, b"oworl")]).is_none(), "overlapping chunks"); } #[test] @@ -404,6 +706,7 @@ mod tests { let m = crate::extract(&p); assert_eq!(m.iso, Some(400)); assert_eq!(m.label.as_deref(), Some("Red")); + assert!(!embedded(&p).chunk_walk_incomplete); let mut w = b"RIFF\0\0\0\0WEBP".to_vec(); let mut body = |id: &[u8], data: &[u8]| { @@ -419,11 +722,30 @@ mod tests { let mut ex = b"Exif\0\0".to_vec(); ex.extend_from_slice(&exif); body(b"EXIF", &ex); + let riff_size = w.len().saturating_sub(8) as u32; + w[4..8].copy_from_slice(&riff_size.to_le_bytes()); let e = webp_chunks(&w); + assert!(!e.chunk_walk_incomplete); assert_eq!(e.icc.as_deref(), Some(&b"abc"[..])); assert_eq!(crate::extract(&w).f_number, Some(2.8)); } + #[test] + fn chunk_walk_reports_truncated_png_and_webp() { + let mut png = b"\x89PNG\r\n\x1a\n".to_vec(); + png.extend_from_slice(&32u32.to_be_bytes()); + png.extend_from_slice(b"iTXtshort"); + assert!(png_chunks(&png).chunk_walk_incomplete); + + let mut webp = b"RIFF\0\0\0\0WEBP".to_vec(); + webp.extend_from_slice(b"XMP "); + webp.extend_from_slice(&32u32.to_le_bytes()); + webp.extend_from_slice(b"short"); + let riff_size = webp.len().saturating_sub(8).saturating_add(27) as u32; + webp[4..8].copy_from_slice(&riff_size.to_le_bytes()); + assert!(webp_chunks(&webp).chunk_walk_incomplete); + } + #[test] fn tiff_file_is_its_own_exif() { let t = sample_exif(ByteOrder::Big); diff --git a/crates/metadata/src/datetime.rs b/crates/metadata/src/datetime.rs index 13ab90bca..9089ad437 100644 --- a/crates/metadata/src/datetime.rs +++ b/crates/metadata/src/datetime.rs @@ -107,7 +107,8 @@ impl DateTime { _ => return None, }; let (h, m) = match rest.split_once(':') { - Some((h, m)) => (num(h)?, num(m)?), + Some((h, m)) if m.len() == 2 => (num(h)?, num(m)?), + Some(_) => return None, None if rest.len() == 4 => (num(rest.get(..2)?)?, num(rest.get(2..)?)?), None => (num(rest)?, 0), }; @@ -159,8 +160,13 @@ impl DateTime { None => (t, None), }; let mut cp = clock.split(':'); - dt.hour = num(cp.next()?)?.try_into().ok()?; - dt.minute = num(cp.next()?)?.try_into().ok()?; + let hour = cp.next()?; + let minute = cp.next()?; + if hour.len() != 2 || minute.len() != 2 { + return None; + } + dt.hour = num(hour)?.try_into().ok()?; + dt.minute = num(minute)?.try_into().ok()?; if let Some(sec) = cp.next() { let (whole, fraction) = match sec.split_once('.') { Some((whole, fraction)) => { @@ -233,6 +239,14 @@ impl DateTime { mod tests { use super::*; + #[test] + fn offset_rejects_short_minute_field() { + assert_eq!(DateTime::parse_offset("+08:5"), None); + assert_eq!(DateTime::parse_offset("+1:2"), None); + assert_eq!(DateTime::parse_offset("+08:05"), Some(485)); + assert_eq!(DateTime::parse_offset("-05:30"), Some(-330)); + } + #[test] fn exif_forms() { let d = DateTime::parse_exif("2024:06:01 13:45:59").unwrap(); @@ -367,6 +381,13 @@ mod tests { } } + #[test] + fn iso_rejects_single_digit_clock_fields() { + for value in ["2024-06-01T1:02:03Z", "2024-06-01T01:2:03Z"] { + assert!(DateTime::parse_iso(value).is_none(), "accepted malformed timestamp {value:?}"); + } + } + #[test] fn iso_validates_clock_while_preserving_supported_forms() { assert_eq!(DateTime::parse_iso("2019").unwrap().to_iso(), "2019"); diff --git a/crates/metadata/src/focus.rs b/crates/metadata/src/focus.rs index b956e667e..1abf3ab4e 100644 --- a/crates/metadata/src/focus.rs +++ b/crates/metadata/src/focus.rs @@ -3,7 +3,8 @@ //! //! **Sony** (the MakerNote IFD: header-less in ARWs, after a `SONY DSC` header in JPEGs). Tag identities and value names come from the public ExifTool //! tag-name documentation (Sony table); every layout below was confirmed on real ILCE-1 (α1) and -//! ILCE-7SM3 (A7S III) ARWs, compared with ExifTool's output as a black box. No third-party code was read. +//! ILCE-7SM3 (A7S III) ARWs, compared with ExifTool's output as a black box, then on the public raw.pixls.us +//! samples of every other body in [`SONY_MODELS`]. No third-party code was read. //! //! | Tag | Layout (confirmed) | Use | //! |---|---|---| @@ -79,7 +80,63 @@ impl FocusPoint { /// Sony bodies whose focus tags have been checked against real files. Another body may store the same /// tags differently (or not mean the same by them): it shows no focus point until its files are checked. -pub const SONY_MODELS: &[&str] = &["ILCE-1", "ILCE-7SM3"]; +/// +/// The α1 and A7S III on 333 files of a photographer's library; the others on the public raw.pixls.us samples (one to +/// six ARWs per body, 103 files of the 44 bodies): every point equal to the one ExifTool's FocusLocation and +/// FocusFrameSize give (77 points, one of them rotated), every refusal (26: manual focus, Wide at the exact centre) +/// equal to what ExifTool's focus modes say, and each body's box drawn on its camera JPEG on the subject it +/// focused on (a pigeon, a face, a leaf, a chart patch). Seven of them (DSC-HX95, DSC-HX99, ILCE-6700, ILCE-7CR, +/// ILCE-7RM2, ILCE-7SM2, ILCE-9M3) only have samples without a point, read as ExifTool reads them. Not listed: the +/// bodies without `0x2027` (every NEX, SLT and DSLR-A; ILCE-3000, 3500, 5000, 5100, 6000 and 6001, QX1, 7, 7R, 7S +/// and 7M2; ILCA-68 and 77M2; RX100 to RX100 III, RX10, RX1 and RX1R; UMC-R10C), and the DSC-RX0, RX100 IV, RX100 V, +/// RX10 II, RX10 III and RX1R II, whose `0x201b` reads 0 (Manual) while the enciphered block says AF-S: ExifTool +/// doesn't decode that tag for them. +pub const SONY_MODELS: &[&str] = &[ + "DSC-HX95", + "DSC-HX99", + "DSC-RX0M2", + "DSC-RX100M5A", + "DSC-RX100M6", + "DSC-RX100M7", + "DSC-RX100M7A", + "DSC-RX10M4", + "DSC-RX1RM3", + "ILCA-99M2", + "ILCE-1", + "ILCE-1M2", + "ILCE-6100", + "ILCE-6300", + "ILCE-6400", + "ILCE-6400A", + "ILCE-6500", + "ILCE-6600", + "ILCE-6700", + "ILCE-7C", + "ILCE-7CM2", + "ILCE-7CR", + "ILCE-7M3", + "ILCE-7M4", + "ILCE-7M5", + "ILCE-7RM2", + "ILCE-7RM3", + "ILCE-7RM3A", + "ILCE-7RM4", + "ILCE-7RM4A", + "ILCE-7RM5", + "ILCE-7SM2", + "ILCE-7SM3", + "ILCE-9", + "ILCE-9M2", + "ILCE-9M3", + "ILME-FX2", + "ILME-FX3", + "ILME-FX30", + "ZV-1", + "ZV-1M2", + "ZV-E1", + "ZV-E10", + "ZV-E10M2", +]; /// Whether a camera as the catalog names it (`"SONY ILCE-1"`: the Exif Make, a space, the Model) is /// one whose files can carry a focus point, so a photo from any other camera is never opened to look. diff --git a/crates/metadata/src/focus/tests.rs b/crates/metadata/src/focus/tests.rs index 148f5ed99..e7644adff 100644 --- a/crates/metadata/src/focus/tests.rs +++ b/crates/metadata/src/focus/tests.rs @@ -261,8 +261,11 @@ fn read_exif_finds_it_through_the_maker_note_and_orientation() { // EXIF 8: sensor x becomes 1 - upright y let (cx, cy) = f.center(); assert!((cy - (1.0 - 3226.0 / 8640.0)).abs() < 1e-9 && (cx - 1980.0 / 5760.0).abs() < 1e-9, "{cx} {cy}"); - // an unconfirmed body or another maker: nothing - assert!(crate::read_exif(&sony_file("SONY", "ILCE-7M4", 1, &alpha1_entries())).focus.is_none()); + // a body checked on public samples reads the same layout + let f = crate::read_exif(&sony_file("SONY", "ZV-E10", 1, &alpha1_entries())).focus.expect("ZV-E10 focus point"); + assert!((f.center().0 - 3226.0 / 8640.0).abs() < 1e-9, "{:?}", f.center()); + // a body left out (its 0x201b is not the focus mode) or another maker: nothing + assert!(crate::read_exif(&sony_file("SONY", "DSC-RX100M4", 1, &alpha1_entries())).focus.is_none()); assert!(crate::read_exif(&sony_file("Canon", "ILCE-1", 1, &alpha1_entries())).focus.is_none()); } @@ -339,7 +342,7 @@ fn header_read_matches_the_whole_file_and_reads_little() { assert_eq!(r.focus, whole); assert!(r.bytes_read <= HEADER_PREFIX); // unsupported bodies agree too (no point either way) - let other = sony_file("SONY", "ILCE-7M4", 1, &alpha1_entries()); + let other = sony_file("SONY", "DSC-RX100M4", 1, &alpha1_entries()); assert_eq!(read(&other).focus, None); } @@ -396,8 +399,10 @@ fn the_synthetic_arw_is_laid_out_as_described() { #[test] fn cameras_that_can_have_a_point() { - assert!(camera_supported("SONY ILCE-1") && camera_supported("Sony ILCE-7SM3 ")); - for c in ["SONY ILCE-7M4", "Canon EOS R5", "", "SONY", "ILCE-1", "SON"] { + for c in ["SONY ILCE-1", "Sony ILCE-7SM3 ", "SONY ILCE-1M2", "SONY ILCE-7M4", "SONY ZV-E10", "SONY DSC-RX100M7"] { + assert!(camera_supported(c), "{c}"); + } + for c in ["SONY ILCE-7M2", "SONY DSC-RX100M4", "SONY NEX-7", "Canon EOS R5", "", "SONY", "ILCE-1", "SON", "SONY ILCE-1M"] { assert!(!camera_supported(c), "{c}"); } } diff --git a/crates/metadata/src/gpx.rs b/crates/metadata/src/gpx.rs index 691a2b7e1..c50454f6b 100644 --- a/crates/metadata/src/gpx.rs +++ b/crates/metadata/src/gpx.rs @@ -7,6 +7,10 @@ //! routes (``) carry no timeline and are ignored. The schema defines a track segment as one //! continuous span of logging, so positions are only interpolated *within* a segment, never across //! the gap where reception was lost or the logger was off. +//! +//! Each point also records which `` it came from ([`TrackPoint::track`], with the tracks' +//! ``s in [`Tracklog::names`]), so a map can draw the tracks of a file apart and tag from +//! one of them ([`Tracklog::only_track`]). use quick_xml::events::Event; @@ -15,6 +19,10 @@ use crate::{DateTime, Gps}; /// Nesting / size limits (a track log is flat; these only stop hostile input). const MAX_DEPTH: usize = 64; const MAX_POINTS: usize = 5_000_000; +/// Most tracks (``) in one file. +const MAX_TRACKS: usize = 10_000; +/// Longest track name kept (characters); the rest is cut. +const MAX_NAME_CHARS: usize = 200; /// One timed point of a track. #[derive(Clone, Copy, Debug, PartialEq)] @@ -27,6 +35,8 @@ pub struct TrackPoint { pub elevation: Option, /// Track segment number, counted across the whole file (continuous logging spans). pub segment: u32, + /// The `` it belongs to, counted from 0 in file order. + pub track: u32, } /// How a position was found by [`Tracklog::locate`]. @@ -44,6 +54,8 @@ pub struct Tracklog { pub points: Vec, /// Tracks (``) in the file. pub tracks: usize, + /// Each track's `` (empty when it has none), in file order: `names.len() == tracks`. + pub names: Vec, /// Track points without a usable time (left out of `points`). pub untimed: usize, } @@ -120,7 +132,13 @@ pub fn parse_gpx(s: &str) -> Result { } match name.as_slice() { b"gpx" if path.is_empty() => seen_gpx = true, - b"trk" if parent == Some(b"gpx") => log.tracks += 1, + b"trk" if parent == Some(b"gpx") => { + if log.tracks >= MAX_TRACKS { + return Err(GpxError::TooComplex); + } + log.tracks += 1; + log.names.push(String::new()); + } b"trkseg" if parent == Some(b"trk") && log.points.last().is_some_and(|p| p.segment == segment) => segment += 1, b"trkpt" if parent == Some(b"trkseg") => { let mut p = Pending::default(); @@ -182,6 +200,11 @@ pub fn parse_gpx(s: &str) -> Result { } } b"trkpt" => finish_point(&mut log, pt.take(), segment)?, + b"name" if parent == Some(b"trk") => { + if let Some(n) = log.names.last_mut() { + *n = text.trim().chars().take(MAX_NAME_CHARS).collect(); + } + } _ => {} } text.clear(); @@ -214,7 +237,9 @@ fn finish_point(log: &mut Tracklog, p: Option, segment: u32) -> Result< if log.points.len() >= MAX_POINTS { return Err(GpxError::TooComplex); } - log.points.push(TrackPoint { time, latitude: lat, longitude: lon, elevation: p.ele, segment }); + let track = u32::try_from(log.tracks.saturating_sub(1)).unwrap_or(u32::MAX); + let elevation = p.ele.filter(|e| e.is_finite()); + log.points.push(TrackPoint { time, latitude: lat, longitude: lon, elevation, segment, track }); Ok(()) } @@ -224,6 +249,16 @@ impl Tracklog { Some((self.points.first()?.time, self.points.last()?.time)) } + /// The points of track `index` alone (its segments as they were), or `None` when there is no + /// such track. + pub fn only_track(&self, index: usize) -> Option { + if index >= self.tracks { + return None; + } + let points = self.points.iter().filter(|p| usize::try_from(p.track).ok() == Some(index)).copied().collect(); + Some(Tracklog { points, tracks: 1, names: vec![self.names.get(index).cloned().unwrap_or_default()], untimed: 0 }) + } + /// Where the logger was at `time` (seconds since the epoch, UTC). /// /// Between two points of the same segment that are at most `max_gap` seconds apart, the position @@ -458,8 +493,8 @@ mod tests { #[test] fn interpolation_across_the_antimeridian() { - let a = TrackPoint { time: 0.0, latitude: 0.0, longitude: 179.0, elevation: None, segment: 0 }; - let b = TrackPoint { time: 10.0, latitude: 0.0, longitude: -179.0, elevation: None, segment: 0 }; + let a = TrackPoint { time: 0.0, latitude: 0.0, longitude: 179.0, elevation: None, segment: 0, track: 0 }; + let b = TrackPoint { time: 10.0, latitude: 0.0, longitude: -179.0, elevation: None, segment: 0, track: 0 }; let g = lerp(&a, &b, 0.75); assert!((g.longitude - -179.5).abs() < 1e-9, "{g:?}"); } @@ -487,5 +522,57 @@ mod tests { ); let deep = "".to_string() + &"".repeat(100); assert_eq!(parse_gpx(&deep), Err(GpxError::TooComplex)); + let many = "".to_string() + &"".repeat(MAX_TRACKS + 1) + ""; + assert_eq!(parse_gpx(&many), Err(GpxError::TooComplex)); + } + + #[test] + fn tracks_are_told_apart_with_their_names() { + let s = r#" Morning + + + + + + + "#; + let log = parse_gpx(s).unwrap(); + assert_eq!(log.tracks, 3); + assert_eq!(log.names, vec!["Morning".to_string(), String::new(), String::new()]); + // sorted by time: the third track's point comes first + assert_eq!(log.points.iter().map(|p| p.track).collect::>(), vec![2, 0, 0]); + let first = log.only_track(0).unwrap(); + assert_eq!((first.points.len(), first.tracks, first.names[0].as_str()), (2, 1, "Morning")); + assert!(log.only_track(1).unwrap().points.is_empty(), "an empty track"); + assert!(log.only_track(3).is_none()); + // a different track is a different segment: never interpolated across + assert_ne!(log.points[0].segment, log.points[1].segment); + } + + #[test] + fn hostile_points_are_left_out() { + let s = r#"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx + + + + + NaN + + + + "#; + let log = parse_gpx(s).unwrap(); + assert_eq!(log.points.len(), 3, "{:?}", log.points); + assert_eq!(log.untimed, 1); + assert!(log.points.iter().all(|p| p.latitude.is_finite() && p.longitude.is_finite())); + assert_eq!((log.points[1].latitude, log.points[1].elevation), (90.0, None), "a NaN elevation is no elevation"); + // out of order in the file, in order in the log + assert_eq!((log.points[0].latitude, log.points[2].latitude), (6.0, 5.0)); + assert_eq!(log.names[0].chars().count(), MAX_NAME_CHARS); + // a single point: located at its own time only (within the gap) + let one = parse_gpx(r#""#).unwrap(); + assert_eq!(one.span(), Some((one.points[0].time, one.points[0].time))); + assert!(one.locate(one.points[0].time + 10.0, 60.0).is_some()); + assert!(one.locate(f64::NAN, 60.0).is_none()); } } diff --git a/crates/metadata/src/iptc.rs b/crates/metadata/src/iptc.rs index a398c6fc4..dbde97892 100644 --- a/crates/metadata/src/iptc.rs +++ b/crates/metadata/src/iptc.rs @@ -13,6 +13,67 @@ fn decode(b: &[u8], utf8: bool) -> String { } } +fn decimal_pair(bytes: &[u8]) -> Option { + let [tens, ones] = bytes else { return None }; + if !tens.is_ascii_digit() || !ones.is_ascii_digit() { + return None; + } + Some(u16::from(*tens - b'0') * 10 + u16::from(*ones - b'0')) +} + +fn parse_created(date: Option<&str>, time: Option<&str>) -> Option { + let date = date.filter(|value| value.len() == 8 && value.bytes().all(|byte| byte.is_ascii_digit()))?; + let time = time.filter(|value| !value.is_empty()); + if let Some(time) = time { + let bytes = time.as_bytes(); + let clock = bytes.get(..6)?; + if !clock.iter().all(|byte| byte.is_ascii_digit()) { + return None; + } + let hour = decimal_pair(clock.get(..2)?)?; + let minute = decimal_pair(clock.get(2..4)?)?; + let second = decimal_pair(clock.get(4..6)?)?; + if hour > 23 || minute > 59 || second > 59 { + return None; + } + match bytes.len() { + // IPTC-IIM permits a local wall-clock form, but without an offset + // it is ambiguous; retain the date and omit this time. + 6 => {} + 11 => { + let offset = bytes.get(6..11)?; + if !matches!(offset.first().copied(), Some(b'+' | b'-')) || !offset.get(1..)?.iter().all(|byte| byte.is_ascii_digit()) { + return None; + } + let offset_hour = decimal_pair(offset.get(1..3)?)?; + let offset_minute = decimal_pair(offset.get(3..5)?)?; + if offset_hour > 18 || offset_minute > 59 || (offset_hour == 18 && offset_minute != 0) { + return None; + } + } + _ => return None, + } + } + + let mut iso = format!("{}-{}-{}", date.get(..4)?, date.get(4..6)?, date.get(6..8)?); + if let Some(time) = time.filter(|time| time.len() == 11) { + let bytes = time.as_bytes(); + iso.push('T'); + iso.push_str(time.get(..2)?); + iso.push(':'); + iso.push_str(time.get(2..4)?); + iso.push(':'); + iso.push_str(time.get(4..6)?); + if bytes.len() == 11 { + let offset = time.get(6..)?; + iso.push_str(offset.get(..3)?); + iso.push(':'); + iso.push_str(offset.get(3..)?); + } + } + DateTime::parse_iso(&iso) +} + /// Parse an IIM stream into the fields [`Metadata`] carries (title, keywords, artist, copyright, caption, /// creation date/time). pub fn parse_iptc(b: &[u8]) -> Metadata { @@ -57,16 +118,7 @@ pub fn parse_iptc(b: &[u8]) -> Metadata { } i = at + size; } - if let Some(d) = date.filter(|d| d.len() == 8 && d.bytes().all(|c| c.is_ascii_digit())) { - let mut s = format!("{}-{}-{}", &d[..4], &d[4..6], &d[6..8]); - if let Some(t) = time.filter(|t| t.is_ascii() && t.len() >= 6 && t[..6].bytes().all(|c| c.is_ascii_digit())) { - s += &format!("T{}:{}:{}", &t[..2], &t[2..4], &t[4..6]); - if t.len() >= 11 { - s += &format!("{}:{}", &t[6..9], &t[9..11]); - } - } - m.capture_time = DateTime::parse_iso(&s); - } + m.capture_time = parse_created(date.as_deref(), time.as_deref()); m } @@ -119,6 +171,61 @@ pub(crate) mod tests { assert_eq!(m.capture_time.unwrap().to_iso(), "2021-03-04T10:11:12+01:00"); } + fn capture_time(time: Option<&[u8]>) -> Metadata { + let mut iim = Vec::new(); + ds(&mut iim, 2, 55, b"20260101"); + if let Some(time) = time { + ds(&mut iim, 2, 60, time); + } + ds(&mut iim, 2, 5, b"kept title"); + ds(&mut iim, 2, 25, b"kept keyword"); + parse_iptc(&iim) + } + + #[test] + fn iptc_time_created_rejects_incomplete_or_trailing_text() { + for time in [ + &b"121314+01"[..], + b"121314junk", + b"12A314", + b"12131499", + b"121314+0100junk", + b"121314+", + b"121314+010", + b"121314+01000", + b"12345", + b"240000", + b"250000+0100", + b"126100+0100", + b"235960", + b"235960+0000", + b"121314+1801", + ] { + let metadata = capture_time(Some(time)); + assert_eq!(metadata.capture_time, None, "accepted malformed Time Created {time:?}"); + assert_eq!(metadata.title.as_deref(), Some("kept title"), "unrelated metadata was lost for {time:?}"); + assert_eq!(metadata.keywords, ["kept keyword"], "unrelated metadata was lost for {time:?}"); + } + } + + #[test] + fn iptc_time_created_preserves_supported_forms() { + let cases: &[(Option<&[u8]>, Option<&str>)] = &[ + (None, Some("2026-01-01T00:00:00")), + (Some(b""), Some("2026-01-01T00:00:00")), + (Some(b" \t"), Some("2026-01-01T00:00:00")), + (Some(b"121314"), Some("2026-01-01T00:00:00")), + (Some(b"121314+0100"), Some("2026-01-01T12:13:14+01:00")), + (Some(b"121314-0530"), Some("2026-01-01T12:13:14-05:30")), + (Some(b"121314+1800"), Some("2026-01-01T12:13:14+18:00")), + (Some(b" 121314+0000 \0"), Some("2026-01-01T12:13:14+00:00")), + ]; + for (time, expected) in cases { + let metadata = capture_time(*time); + assert_eq!(metadata.capture_time.map(|value| value.to_iso()).as_deref(), *expected, "Time Created {time:?}"); + } + } + #[test] fn non_utf8_text_drops_trailing_nul() { let mut v = Vec::new(); diff --git a/crates/metadata/src/lib.rs b/crates/metadata/src/lib.rs index 7771934c8..baf9fe18c 100644 --- a/crates/metadata/src/lib.rs +++ b/crates/metadata/src/lib.rs @@ -36,7 +36,7 @@ pub mod tags; mod xmp; mod xmp_merge; -pub use container::{Embedded, embedded, jpeg_segments, png_chunks, webp_chunks}; +pub use container::{Embedded, embedded, jpeg_segments, jpeg_segments_with_all_xmp, png_chunks, webp_chunks}; pub use datetime::{DatePrecision, DateTime}; pub use exif::{from_tiff, read_exif, strip_exif_header, try_read_exif, try_read_exif_with, write_exif}; pub use focus::FocusPoint; @@ -45,7 +45,9 @@ pub use geom::{Orientation, Point, Rect}; pub use gpx::{GpxError, Match, TrackPoint, Tracklog, parse_gpx}; pub use iptc::parse_iptc; pub use tags::{TagRow, file_tag_rows, tag_rows}; -pub use xmp::{CRS_NS, XmpData, XmpError, XmpNamespace, XmpValue, XmpWrite, parse_xmp, parse_xmp_with, write_xmp, write_xmp_with}; +pub use xmp::{ + CRS_NS, XmpData, XmpError, XmpNamespace, XmpValue, XmpWrite, parse_gps_position, parse_xmp, parse_xmp_with, write_xmp, write_xmp_with, +}; pub use xmp_merge::{MergeRules, merge_xmp}; use serde::{Deserialize, Serialize}; @@ -383,6 +385,20 @@ pub fn lens_spec_name(spec: [f64; 4]) -> Option { Some(format!("{focal} mm f/{aperture}")) } +/// Whether `s` has the ` mm f/` shape written by [`lens_spec_name`]. +pub fn is_lens_spec_name(s: &str) -> bool { + let Some((focal, aperture)) = s.trim().split_once(" mm f/") else { return false }; + let is_num_or_range = |part: &str| { + let mut pieces = part.split('-'); + let Some(first) = pieces.next() else { return false }; + let second = pieces.next(); + pieces.next().is_none() + && first.parse::().is_ok_and(|v| v.is_finite() && v > 0.0) + && second.is_none_or(|b| b.parse::().is_ok_and(|v| v.is_finite() && v > 0.0)) + }; + is_num_or_range(focal) && is_num_or_range(aperture) +} + /// Lenient number parsing used by XMP/IPTC: `"28/10"`, `"2.8"`, `"+0.7"`. pub(crate) fn parse_number(s: &str) -> Option { let s = s.trim(); @@ -408,6 +424,62 @@ mod tests { assert_eq!(lens_spec_name([f64::NAN, 50.0, 1.8, 1.8]), None); } + #[test] + fn lens_spec_names_are_recognised() { + for spec in [[50.0, 50.0, 1.8, 1.8], [18.0, 135.0, 3.5, 5.6]] { + assert!(is_lens_spec_name(&lens_spec_name(spec).unwrap())); + } + assert!(is_lens_spec_name(" 24.0-70.0 mm f/2.8 ")); + for not in ["EF 50mm f/1.8", "50 mm f/", "0 mm f/2.8", "50-60-70 mm f/2.8", "nan mm f/2", "50 mm f/1.8-"] { + assert!(!is_lens_spec_name(not), "{not}"); + } + } + + #[test] + fn from_embedded_preserves_exif_iptc_xmp_precedence() { + let exif = Metadata { + make: Some("Example".into()), + model: Some("Camera".into()), + lens_model: Some("35mm F2".into()), + artist: Some("EXIF artist".into()), + orientation: Some(Orientation::from_exif(6)), + exposure_time: Some(1.0 / 125.0), + ..Default::default() + }; + let packet = write_xmp(&Metadata { + artist: Some("XMP artist".into()), + caption: Some("XMP caption".into()), + rating: Some(4), + keywords: vec!["xmp-keyword".into()], + ..Default::default() + }); + let mut iptc = Vec::new(); + for (dataset, value) in [(5u8, "IPTC title"), (120u8, "IPTC caption")] { + iptc.extend_from_slice(&[0x1c, 2, dataset]); + iptc.extend_from_slice(&u16::try_from(value.len()).unwrap().to_be_bytes()); + iptc.extend_from_slice(value.as_bytes()); + } + let blocks = Embedded { exif: Some(write_exif(&exif)), iptc: Some(iptc), xmp: Some(packet.clone()), ..Default::default() }; + let parsed = from_embedded(&blocks); + assert_eq!(parsed.make, exif.make); + assert_eq!(parsed.model, exif.model); + assert_eq!(parsed.lens_model, exif.lens_model); + assert_eq!(parsed.orientation, exif.orientation); + assert_eq!(parsed.exposure_time, exif.exposure_time); + assert_eq!(parsed.title.as_deref(), Some("IPTC title")); + assert_eq!(parsed.caption.as_deref(), Some("XMP caption")); + assert_eq!(parsed.artist.as_deref(), Some("XMP artist")); + assert_eq!(parsed.rating, Some(4)); + assert_eq!(parsed.keywords, vec!["xmp-keyword"]); + assert_eq!(blocks.xmp.as_deref(), Some(packet.as_str())); + } + + #[test] + fn from_embedded_ignores_damaged_blocks() { + let blocks = Embedded { exif: Some(vec![1, 2, 3]), iptc: Some(vec![0x1c, 2]), xmp: Some(">, + /// True when flattened properties were omitted after reaching the cumulative key-size limit. + pub properties_truncated: bool, /// Every top-level property with its full structure (arrays of structs such as local /// corrections, which [`XmpData::properties`] flattens). pub values: BTreeMap, /// The `x:xmptk` attribute of `x:xmpmeta` (older packets: `x:xaptk` of `x:xapmeta`), the name /// of the program that wrote the packet. [`write_xmp_with`] writes [`XmpWrite::toolkit`] there. pub toolkit: Option, + /// True when the parsed tree, before flattening or merging repeated properties, names an Exif + /// GPS property anywhere or holds a place property (see `LOCATION_PROPERTIES`) with non-blank + /// text anywhere, including inside array items that [`XmpData::properties`] flattens to their + /// text and repeated structs that [`XmpData::values`] keeps only once. + pub names_location: bool, +} + +/// Properties that name a place; a blank one is not a location. +const LOCATION_PROPERTIES: [&str; 7] = [ + "Iptc4xmpCore:Location", + "Iptc4xmpCore:CountryCode", + "photoshop:City", + "photoshop:State", + "photoshop:Country", + "Iptc4xmpExt:LocationShown", + "Iptc4xmpExt:LocationCreated", +]; + +impl XmpData { + /// Whether the XMP inventory names GPS or a non-empty place field. LocationShown and + /// LocationCreated are IPTC Extension structures; their flattened array entries can be empty + /// strings even when their struct children contain a city or coordinates, so inspect their + /// retained structured value and flattened child fields as well. Callers that must prove + /// absence must check [`Self::properties_truncated`] first. + pub fn has_location_metadata(&self) -> bool { + const IPTC_LOCATION_STRUCTS: [&str; 2] = ["Iptc4xmpExt:LocationShown", "Iptc4xmpExt:LocationCreated"]; + // Every flattened child of one LocationShown/LocationCreated root shares its answer. + let mut checked_roots: BTreeMap<&str, bool> = BTreeMap::new(); + if self.names_location { + return true; + } + + for (key, values) in &self.properties { + let parts = key.split('/'); + if parts.clone().any(is_gps_name) { + // Even an isolated/blank GPS property is not proof that GPS was removed. + return true; + } + + let Some(location_part) = parts.clone().find(|part| LOCATION_PROPERTIES.contains(part)) else { + continue; + }; + + if values.iter().any(|value| !value.trim().is_empty()) { + return true; + } + if IPTC_LOCATION_STRUCTS.contains(&location_part) { + // The scope is the path up to the location field itself, so text elsewhere in an + // enclosing struct doesn't count as a place. + let scope_len = + key.split('/').take_while(|part| *part != location_part).map(|part| part.len() + 1).sum::() + location_part.len(); + let scope = key.get(..scope_len).unwrap_or(key); + let scope_has_text = *checked_roots.entry(scope).or_insert_with(|| self.location_scope_has_text(scope)); + if scope_has_text { + return true; + } + } + } + + false + } + + /// Whether a LocationShown/LocationCreated field (a `/`-separated path from a top-level + /// property) holds text in its retained structured value or any flattened child. `values` + /// keeps the first text-bearing occurrence across RDF descriptions, while flattened child + /// properties retain every occurrence. + fn location_scope_has_text(&self, scope: &str) -> bool { + let mut parts = scope.split('/'); + let structured = parts.next().and_then(|root| self.values.get(root)).and_then(|value| parts.try_fold(value, |value, part| value.field(part))); + if structured.is_some_and(value_has_text) { + return true; + } + let child_prefix = format!("{scope}/"); + self.properties + .range::((std::ops::Bound::Included(child_prefix.as_str()), std::ops::Bound::Unbounded)) + .take_while(|(candidate, _)| candidate.starts_with(&child_prefix)) + .any(|(_, nested_values)| nested_values.iter().any(|value| !value.trim().is_empty())) + } + + /// The GUID of the JPEG Extended XMP continuation the packet announces + /// (`xmpNote:HasExtendedXMP`), matched by local name so an undeclared or foreign prefix can't + /// hide the announcement. `Some("")` when the announcement is blank. + pub fn extended_xmp_guid(&self) -> Option<&str> { + self.properties + .iter() + .find(|(key, _)| !key.contains('/') && key.rsplit([':', '}']).next() == Some("HasExtendedXMP")) + .map(|(_, values)| values.first().map_or("", |value| value.trim())) + } } impl XmpData { @@ -143,15 +234,75 @@ impl XmpValue { } } +/// Whether a structured value holds any non-blank text. A tree larger than the parser's node +/// budget counts as text-bearing (the conservative answer for a privacy check): parsed trees are +/// capped at MAX_NODES, so only a caller-built `XmpData` can reach it, and the walk stays bounded. +fn value_has_text(value: &XmpValue) -> bool { + let mut pending = Vec::new(); + if pending.try_reserve(1).is_err() { + return true; + } + pending.push(value); + let mut visited = 0usize; + while let Some(value) = pending.pop() { + visited = visited.saturating_add(1); + if visited > MAX_NODES { + return true; + } + let children = match value { + XmpValue::Text(text) if !text.trim().is_empty() => return true, + XmpValue::Text(_) => continue, + XmpValue::Array(items) => items.len(), + XmpValue::Struct(fields) => fields.len(), + }; + if pending.len().saturating_add(children) > MAX_NODES || pending.try_reserve(children).is_err() { + return true; + } + match value { + XmpValue::Array(items) => pending.extend(items), + XmpValue::Struct(fields) => pending.extend(fields.values()), + XmpValue::Text(_) => {} + } + } + false +} + +/// Insert a property into a struct or description map, keeping an earlier text-bearing value when +/// the same name repeats with a blank one: an IPTC location struct repeated with an empty +/// `rdf:li` must not hide the place an export's Remove Location has to refuse. `has_text` +/// remembers each key's answer so every value is walked at most once. +fn insert_keeping_text(map: &mut BTreeMap, has_text: &mut BTreeMap, key: String, value: XmpValue) { + match map.entry(key) { + std::collections::btree_map::Entry::Vacant(slot) => { + slot.insert(value); + } + std::collections::btree_map::Entry::Occupied(mut slot) => { + let existing_has_text = match has_text.get(slot.key()) { + Some(known) => *known, + None => value_has_text(slot.get()), + }; + if existing_has_text { + has_text.insert(slot.key().clone(), true); + return; + } + let new_has_text = value_has_text(&value); + has_text.insert(slot.key().clone(), new_has_text); + slot.insert(value); + } + } +} + +/// A description or struct node as a map of its fields (attributes and child elements). fn node_struct(n: &Node) -> XmpValue { let mut m = BTreeMap::new(); + let mut has_text = BTreeMap::new(); for (k, v) in &n.attrs { if !is_rdf_meta_attr(k) { - m.insert(k.clone(), XmpValue::Text(v.clone())); + insert_keeping_text(&mut m, &mut has_text, k.clone(), XmpValue::Text(v.clone())); } } for c in &n.children { - m.insert(c.name.clone(), node_value(c)); + insert_keeping_text(&mut m, &mut has_text, c.name.clone(), node_value(c)); } XmpValue::Struct(m) } @@ -193,6 +344,18 @@ pub(crate) const MAX_OPAQUE_NAME_BYTES: usize = 8 * 1024 * 1024; /// retains its prefix once per property: node limits alone cannot bound that duplication. const MAX_KEY_BYTES: usize = 1 << 20; +/// Most namespace declarations in scope at once. Prefix lookup scans them, so declarations times the names of a +/// start tag is the work; real packets declare a few dozen. +const MAX_NS_DECLS: usize = 512; + +/// Refuses a scope stack holding more than [`MAX_NS_DECLS`] declarations (one count per start tag over a stack at most [`MAX_DEPTH`] deep). +pub(crate) fn check_scopes(scopes: &[Vec<(String, String)>]) -> Result<(), XmpError> { + if scopes.iter().map(Vec::len).sum::() > MAX_NS_DECLS { + return Err(XmpError::TooComplex); + } + Ok(()) +} + /// Names canonicalised by namespace URI (see [`NAMESPACES`]); conflicting foreign prefixes are opaque. pub(crate) fn canonical_name( qname: &str, @@ -265,6 +428,12 @@ fn xml_safe(s: &str) -> String { s.chars().filter(|&c| is_xml_char(c)).collect() } +/// Escaped text with each carriage return as ` `: XML end-of-line handling turns a literal one +/// into a line feed on read. +fn keep_cr(escaped: std::borrow::Cow<'_, str>) -> String { + escaped.replace('\r', " ") +} + /// Index of the `;` that ends a reference starting at `rest` (its `&`). The scan stops at the first /// byte that cannot appear in a name or a numeric reference, so zero-padded references such as /// `a` are found and each run of name bytes is read once, not once per `&`. @@ -323,6 +492,7 @@ fn parse_tree(s: &str, extra: &[XmpNamespace]) -> Result { } } scopes.push(scope); + check_scopes(&scopes)?; let name = canonical_name(&String::from_utf8_lossy(e.name().as_ref()), &scopes, extra, &mut opaque_budget)?; let attrs = raw_attrs.into_iter().map(|(k, v)| Ok((canonical_name(&k, &scopes, extra, &mut opaque_budget)?, v))).collect:: bool { matches!(k, "rdf:about" | "rdf:ID" | "rdf:nodeID" | "xml:lang" | "rdf:parseType" | "rdf:resource" | "rdf:datatype") || k.starts_with("xmlns") } -fn collect_description(d: &Node, prefix: &str, props: &mut BTreeMap>, budget: &mut usize) { +fn collect_description(d: &Node, prefix: &str, props: &mut BTreeMap>, budget: &mut usize, truncated: &mut bool) { for (k, v) in &d.attrs { + if is_rdf_meta_attr(k) { + continue; + } // the budget is checked before the path is built, not after - if is_rdf_meta_attr(k) || *budget < prefix.len().saturating_add(k.len()) { + if *budget < prefix.len().saturating_add(k.len()) { + *truncated = true; continue; } let key = format!("{prefix}{k}"); - if take_key_bytes(&key, budget) { + if take_key_bytes(&key, budget, truncated) { props.entry(key).or_default().push(v.clone()); } } for c in &d.children { - collect_property(c, prefix, props, budget); + collect_property(c, prefix, props, budget, truncated); } } /// Charge one flattened key's length to the key budget; `false` when the budget is spent (the -/// property is then left out, like other elements a limit has cut). -fn take_key_bytes(key: &str, budget: &mut usize) -> bool { +/// property is then left out, like other elements a limit has cut) and the map is marked partial. +fn take_key_bytes(key: &str, budget: &mut usize, dropped: &mut bool) -> bool { let n = key.len(); if *budget < n { + *dropped = true; return false; } *budget -= n; true } -fn collect_property(p: &Node, prefix: &str, props: &mut BTreeMap>, budget: &mut usize) { +fn collect_property(p: &Node, prefix: &str, props: &mut BTreeMap>, budget: &mut usize, truncated: &mut bool) { // a path that can't fit the budget isn't built (a long ancestor path shared by many children // would otherwise be copied for each of them after the budget is spent) if *budget < prefix.len().saturating_add(p.name.len()) { + *truncated = true; return; } let key = format!("{prefix}{}", p.name); @@ -432,7 +608,7 @@ fn collect_property(p: &Node, prefix: &str, props: &mut BTreeMap = p.attrs.iter().filter(|(k, _)| !is_rdf_meta_attr(k)).collect(); if !field_attrs.is_empty() && p.children.is_empty() && p.text.trim().is_empty() { for (k, v) in field_attrs { let full = format!("{struct_prefix}{k}"); - if take_key_bytes(&full, budget) { + if take_key_bytes(&full, budget, truncated) { props.entry(full).or_default().push(v.clone()); } } return; } let v = p.attr("rdf:resource").map(str::to_string).unwrap_or_else(|| p.text.trim().to_string()); - if take_key_bytes(&key, budget) { + if take_key_bytes(&key, budget, truncated) { props.entry(key).or_default().push(v); } } @@ -479,21 +655,95 @@ fn find_descriptions<'a>(n: &'a Node, out: &mut Vec<&'a Node>) { } } -/// XMP GPS coordinate `DDD,MM,SSk` or `DDD,MM.mmk` (k = N/S/E/W) → signed degrees. -fn parse_gps_coord(s: &str) -> Option { +/// Whether a canonical property name is an Exif GPS property. +fn is_gps_name(name: &str) -> bool { + name.starts_with("exif:GPS") || name.starts_with("exifEX:GPS") +} + +/// Whether the tree names an Exif GPS property anywhere or holds a place property with non-blank +/// text anywhere. Each node is visited at most twice (a place's subtree is scanned once and not +/// walked again), and the parser caps the tree at `MAX_NODES`. +fn tree_names_location(root: &Node) -> bool { + let mut pending = vec![root]; + while let Some(node) = pending.pop() { + if is_gps_name(&node.name) { + return true; + } + for (name, value) in &node.attrs { + if is_gps_name(name) || (LOCATION_PROPERTIES.contains(&name.as_str()) && !value.trim().is_empty()) { + return true; + } + } + if LOCATION_PROPERTIES.contains(&node.name.as_str()) { + if subtree_has_text(node) { + return true; + } + // A blank place may still enclose a GPS property. + if subtree_names_gps(node) { + return true; + } + continue; + } + pending.extend(&node.children); + } + false +} + +/// Whether a node or any descendant holds non-blank text: element text, an `rdf:resource`, or a +/// field attribute (RDF syntax attributes such as `rdf:parseType` and `xml:lang` don't count). +fn subtree_has_text(root: &Node) -> bool { + let mut pending = vec![root]; + while let Some(node) = pending.pop() { + if !node.text.trim().is_empty() { + return true; + } + if node.attrs.iter().any(|(name, value)| (name == "rdf:resource" || !is_rdf_meta_attr(name)) && !value.trim().is_empty()) { + return true; + } + pending.extend(&node.children); + } + false +} + +/// Whether a node or any descendant is an Exif GPS property, as an element or an attribute. +fn subtree_names_gps(root: &Node) -> bool { + let mut pending = vec![root]; + while let Some(node) = pending.pop() { + if is_gps_name(&node.name) || node.attrs.iter().any(|(name, _)| is_gps_name(name)) { + return true; + } + pending.extend(&node.children); + } + false +} + +/// An `exif:GPSLatitude` / `exif:GPSLongitude` pair → (latitude, longitude) in signed degrees; `None` +/// unless both parse and lie on the globe (|latitude| ≤ 90, |longitude| ≤ 180, as the Exif reader checks). +pub fn parse_gps_position(lat: &str, lon: &str) -> Option<(f64, f64)> { + let (lat, lon) = (parse_gps_coord(lat, ['N', 'S'])?, parse_gps_coord(lon, ['E', 'W'])?); + ((-90.0..=90.0).contains(&lat) && (-180.0..=180.0).contains(&lon)).then_some((lat, lon)) +} + +/// XMP GPS coordinate `DDD,MM,SSk` or `DDD,MM.mmk` → signed degrees, k being `[positive, negative]` of +/// its axis (N/S or E/W, either case). The sign comes only from the letter: a negative part +/// (`-10,30N`) is malformed. +fn parse_gps_coord(s: &str, [pos, neg]: [char; 2]) -> Option { let s = s.trim(); let dir = s.chars().last()?; let body = &s[..s.len() - dir.len_utf8()]; let parts: Vec = body.split(',').map(|p| p.trim().parse::().ok()).collect::>()?; + if parts.iter().any(|p| p.is_sign_negative()) { + return None; + } let v = match parts.as_slice() { [d] => *d, - [d, m] => d + m / 60.0, - [d, m, sec] => d + m / 60.0 + sec / 3600.0, + [d, m] if (0.0..60.0).contains(m) => d + m / 60.0, + [d, m, sec] if (0.0..60.0).contains(m) && (0.0..60.0).contains(sec) => d + m / 60.0 + sec / 3600.0, _ => return None, }; let v = match dir.to_ascii_uppercase() { - 'N' | 'E' => v, - 'S' | 'W' => -v, + c if c == pos => v, + c if c == neg => -v, _ => return None, }; v.is_finite().then_some(v) @@ -581,6 +831,183 @@ fn fmt_gps_coord(v: f64, pos: char, neg: char) -> String { format!("{},{:.6}{}", d as i64, m, if v < 0.0 { neg } else { pos }) } +// Pixel dimensions are integers. Keep decimal coefficients as digits so wide rational components +// remain exact and fractions that `f64` rounds to an integer can still be rejected. +struct ExactDecimal { + negative: bool, + coefficient: String, + exponent: i64, +} + +struct ExactRatio { + negative: bool, + numerator: String, + denominator: String, + exponent: i64, +} + +fn exact_decimal(value: &str) -> Option { + let value = value.trim(); + let (negative, value) = if let Some(value) = value.strip_prefix('-') { + (true, value) + } else if let Some(value) = value.strip_prefix('+') { + (false, value) + } else { + (false, value) + }; + let (mantissa, exponent) = if let Some((mantissa, exponent)) = value.split_once('e').or_else(|| value.split_once('E')) { + if exponent.contains('e') || exponent.contains('E') { + return None; + } + (mantissa, exponent.parse::().ok()?) + } else { + (value, 0) + }; + let (whole, fraction) = mantissa.split_once('.').unwrap_or((mantissa, "")); + if fraction.contains('.') { + return None; + } + let digits = whole.bytes().chain(fraction.bytes()); + if !digits.clone().all(|digit| digit.is_ascii_digit()) { + return None; + } + let count = whole.len().checked_add(fraction.len())?; + if count == 0 { + return None; + } + let leading = digits.clone().take_while(|digit| *digit == b'0').count(); + if leading == count { + return Some(ExactDecimal { negative, coefficient: "0".to_owned(), exponent: 0 }); + } + let trailing = digits.clone().rev().take_while(|digit| *digit == b'0').count(); + let significant = count.checked_sub(leading)?.checked_sub(trailing)?; + let mut coefficient = String::with_capacity(significant); + for digit in digits.skip(leading).take(significant) { + coefficient.push(char::from(digit)); + } + let exponent = exponent.checked_sub(i64::try_from(fraction.len()).ok()?)?.checked_add(i64::try_from(trailing).ok()?)?; + Some(ExactDecimal { negative, coefficient, exponent }) +} + +fn exact_ratio(numerator: ExactDecimal, denominator: ExactDecimal) -> Option { + if denominator.coefficient == "0" { + return None; + } + Some(ExactRatio { + negative: numerator.negative ^ denominator.negative, + numerator: numerator.coefficient, + denominator: denominator.coefficient, + exponent: numerator.exponent.checked_sub(denominator.exponent)?, + }) +} + +fn multiply_decimal_by_u32(value: &str, multiplier: u32) -> String { + if multiplier == 0 { + return "0".to_owned(); + } + let mut digits = Vec::with_capacity(value.len().saturating_add(10)); + let mut carry = 0u64; + for digit in value.bytes().rev() { + let product = u64::from(digit - b'0') * u64::from(multiplier) + carry; + digits.push(b'0' + (product % 10) as u8); + carry = product / 10; + } + while carry != 0 { + digits.push(b'0' + (carry % 10) as u8); + carry /= 10; + } + digits.reverse(); + digits.into_iter().map(char::from).collect() +} + +fn decimal_significand(value: &str) -> Option { + let mut prefix = 0u64; + let mut digits = 0u32; + for digit in value.bytes().take(17) { + prefix = prefix.checked_mul(10)?.checked_add(u64::from(digit.checked_sub(b'0')?))?; + digits = digits.checked_add(1)?; + } + let exponent = i32::try_from(digits.checked_sub(1)?).ok()?; + Some(prefix as f64 / 10.0_f64.powi(exponent)) +} + +fn compare_scaled_decimal(left: &str, left_exponent: i64, right: &str, right_exponent: i64) -> Option { + if left == "0" { + return Some(if right == "0" { std::cmp::Ordering::Equal } else { std::cmp::Ordering::Less }); + } + if right == "0" { + return Some(std::cmp::Ordering::Greater); + } + let left_magnitude = i128::try_from(left.len()).ok()?.checked_add(i128::from(left_exponent))?; + let right_magnitude = i128::try_from(right.len()).ok()?.checked_add(i128::from(right_exponent))?; + match left_magnitude.cmp(&right_magnitude) { + std::cmp::Ordering::Equal => {} + ordering => return Some(ordering), + } + let length = left.len().max(right.len()); + for (left_digit, right_digit) in left.bytes().chain(std::iter::repeat(b'0')).zip(right.bytes().chain(std::iter::repeat(b'0'))).take(length) { + match left_digit.cmp(&right_digit) { + std::cmp::Ordering::Equal => {} + ordering => return Some(ordering), + } + } + Some(std::cmp::Ordering::Equal) +} + +fn exact_ratio_to_u32(value: ExactRatio) -> Option { + if value.negative || value.denominator == "0" { + return None; + } + if value.numerator == "0" { + return Some(0); + } + if value.exponent < -128 || value.exponent > 128 { + return None; + } + let numerator_length = i128::try_from(value.numerator.len()).ok()?; + let denominator_length = i128::try_from(value.denominator.len()).ok()?; + let decimal_scale = numerator_length.checked_add(i128::from(value.exponent))?.checked_sub(denominator_length)?; + if !(0..=10).contains(&decimal_scale) { + return None; + } + // The prefix only estimates a candidate. Acceptance compares the full exact coefficients below. + let approximate = + decimal_significand(&value.numerator)? / decimal_significand(&value.denominator)? * 10.0_f64.powi(i32::try_from(decimal_scale).ok()?); + if !approximate.is_finite() { + return None; + } + let integer = approximate.round().clamp(0.0, f64::from(u32::MAX)) as u32; + let product = multiply_decimal_by_u32(&value.denominator, integer); + let ordering = if product == "0" { + compare_scaled_decimal(&value.numerator, value.exponent, "0", 0)? + } else { + let trailing_zeroes = product.bytes().rev().take_while(|digit| *digit == b'0').count(); + let product_exponent = i64::try_from(trailing_zeroes).ok()?; + compare_scaled_decimal(&value.numerator, value.exponent, product.trim_end_matches('0'), product_exponent)? + }; + (ordering == std::cmp::Ordering::Equal).then_some(integer) +} + +fn exact_u32_from_text(value: &str) -> Option { + let value = value.trim(); + let ratio = if let Some((numerator, denominator)) = value.split_once('/') { + exact_ratio(exact_decimal(numerator)?, exact_decimal(denominator)?)? + } else { + let value = exact_decimal(value)?; + ExactRatio { negative: value.negative, numerator: value.coefficient, denominator: "1".to_owned(), exponent: value.exponent } + }; + exact_ratio_to_u32(ratio) +} + +/// A positive whole-number ISO, read exactly (a fraction or an out-of-range value is not one). +fn iso_from_text(value: &str) -> Option { + exact_u32_from_text(value).filter(|&iso| iso > 0) +} + +fn dimension_candidate(first: impl Fn(&str) -> Option, key: &str) -> Option { + first(key).and_then(|value| exact_u32_from_text(&value)) +} + /// Parse an XMP packet (with or without the `` wrapper / `x:xmpmeta` element). pub fn parse_xmp(s: &str) -> Result { parse_xmp_with(s, &[]) @@ -590,16 +1017,22 @@ pub fn parse_xmp(s: &str) -> Result { pub fn parse_xmp_with(s: &str, extra: &[XmpNamespace]) -> Result { let s = s.trim_start_matches('\u{feff}'); let tree = parse_tree(s, extra)?; + let names_location = tree_names_location(&tree); let mut descs = Vec::new(); find_descriptions(&tree, &mut descs); let mut props = BTreeMap::new(); let mut values = BTreeMap::new(); + let mut properties_truncated = false; + let mut value_has_text_by_key = BTreeMap::new(); // One budget for the whole document: several descriptions cannot each spend a fresh one. let mut key_budget = MAX_KEY_BYTES; for d in descs { - collect_description(d, "", &mut props, &mut key_budget); + collect_description(d, "", &mut props, &mut key_budget, &mut properties_truncated); if let XmpValue::Struct(m) = node_struct(d) { - values.extend(m); + // Later descriptions can repeat a property too; the same rule keeps an earlier place. + for (key, value) in m { + insert_keeping_text(&mut values, &mut value_has_text_by_key, key, value); + } } } let first = |k: &str| props.get(k).and_then(|v| v.first()).map(|s| s.trim().to_string()).filter(|s| !s.is_empty()); @@ -620,9 +1053,9 @@ pub fn parse_xmp_with(s: &str, extra: &[XmpNamespace]) -> Result= 0.0).or_else(|| num("exif:PixelXDimension").filter(|v| *v >= 0.0)).map(|v| v as u32), - height: num("tiff:ImageLength").filter(|v| *v >= 0.0).or_else(|| num("exif:PixelYDimension").filter(|v| *v >= 0.0)).map(|v| v as u32), + orientation: num("tiff:Orientation").filter(|v| (1.0..=8.0).contains(v) && v.fract() == 0.0).map(|v| Orientation::from_exif(v as u16)), + width: dimension_candidate(first, "tiff:ImageWidth").or_else(|| dimension_candidate(first, "exif:PixelXDimension")), + height: dimension_candidate(first, "tiff:ImageLength").or_else(|| dimension_candidate(first, "exif:PixelYDimension")), title: first("dc:title"), caption: first("dc:description"), alt_text: first("Iptc4xmpCore:AltTextAccessibility"), @@ -643,7 +1076,8 @@ pub fn parse_xmp_with(s: &str, extra: &[XmpNamespace]) -> Result 0.0).map(|v| v as u32); + let iso = |key: &str| first(key).and_then(|value| iso_from_text(&value)); + m.iso = [iso("exif:ISOSpeedRatings"), iso("exifEX:PhotographicSensitivity")].into_iter().flatten().next(); if let Some(spec) = props.get("exifEX:LensSpecification").filter(|v| v.len() == 4) { let v: Vec = spec.iter().filter_map(|s| parse_number(s)).collect(); if v.len() == 4 { @@ -668,11 +1102,7 @@ pub fn parse_xmp_with(s: &str, extra: &[XmpNamespace]) -> Result Result String { @@ -766,13 +1196,13 @@ pub fn write_xmp_with(meta: &Metadata, with: &XmpWrite) -> String { } x.push_str(">\n"); for (k, v) in &simple { - x.push_str(&format!(" <{k}>{}\n", escape(xml_safe(v)))); + x.push_str(&format!(" <{k}>{}\n", keep_cr(escape(xml_safe(v))))); } for (k, v) in with.properties.iter().filter(|_| own.is_some()) { let p = own.map_or("", |n| n.0); if !k.is_empty() && k.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') { // element content: quotes needn't be escaped, which keeps JSON readable - x.push_str(&format!(" <{p}:{k}>{}\n", partial_escape(xml_safe(v)))); + x.push_str(&format!(" <{p}:{k}>{}\n", keep_cr(partial_escape(xml_safe(v))))); } } let array = |x: &mut String, k: &str, kind: &str, items: &[String], lang: bool| { @@ -782,9 +1212,9 @@ pub fn write_xmp_with(meta: &Metadata, with: &XmpWrite) -> String { x.push_str(&format!(" <{k}>\n \n")); for it in items { if lang { - x.push_str(&format!(" {}\n", escape(xml_safe(it)))); + x.push_str(&format!(" {}\n", keep_cr(escape(xml_safe(it))))); } else { - x.push_str(&format!(" {}\n", escape(xml_safe(it)))); + x.push_str(&format!(" {}\n", keep_cr(escape(xml_safe(it))))); } } x.push_str(&format!(" \n \n")); @@ -799,7 +1229,7 @@ pub fn write_xmp_with(meta: &Metadata, with: &XmpWrite) -> String { array(&mut x, "dc:creator", "Seq", &one(&meta.artist), false); array(&mut x, "dc:subject", "Bag", &meta.keywords, false); array(&mut x, "lr:hierarchicalSubject", "Bag", &meta.hierarchical_keywords, false); - array(&mut x, "exif:ISOSpeedRatings", "Seq", &meta.iso.map(|v| vec![v.to_string()]).unwrap_or_default(), false); + array(&mut x, "exif:ISOSpeedRatings", "Seq", &meta.iso.filter(|value| *value > 0).map(|v| vec![v.to_string()]).unwrap_or_default(), false); if let Some(s) = meta.lens_spec { array(&mut x, "exifEX:LensSpecification", "Seq", &s.iter().map(|&v| frac(v)).collect::>(), false); } diff --git a/crates/metadata/src/xmp/tests.rs b/crates/metadata/src/xmp/tests.rs index b8d5807b7..c3c3181c9 100644 --- a/crates/metadata/src/xmp/tests.rs +++ b/crates/metadata/src/xmp/tests.rs @@ -194,6 +194,109 @@ fn non_positive_iso_is_ignored() { assert_eq!(read(r#"exif:ISOSpeedRatings="200""#), Some(200)); } +fn iso_packet(attrs: &str, element: &str) -> String { + format!( + r#" + {element} + "# + ) +} + +#[test] +fn xmp_iso_rejects_fractional_values() { + let cases = [ + (iso_packet(r#"exif:ISOSpeedRatings="0.5""#, ""), "exif:ISOSpeedRatings", "0.5"), + (iso_packet(r#"exif:ISOSpeedRatings="1.0000000000000001""#, ""), "exif:ISOSpeedRatings", "1.0000000000000001"), + (iso_packet(r#"exif:ISOSpeedRatings="1.0000000000000001/1""#, ""), "exif:ISOSpeedRatings", "1.0000000000000001/1"), + (iso_packet("", "160.9"), "exif:ISOSpeedRatings", "160.9"), + (iso_packet(r#"exifEX:PhotographicSensitivity="160.9""#, ""), "exifEX:PhotographicSensitivity", "160.9"), + (iso_packet("", "0.5"), "exifEX:PhotographicSensitivity", "0.5"), + ( + iso_packet("", "160.9"), + "exifEX:PhotographicSensitivity", + "160.9", + ), + ]; + for (packet, key, raw) in cases { + let d = parse_xmp(&packet).unwrap(); + assert_eq!(d.metadata.iso, None, "{packet}"); + assert_eq!( + d.properties.get(key).map(|values| values.iter().map(String::as_str).collect::>()), + Some(vec![raw]), + "raw property changed: {packet}" + ); + assert_eq!(d.metadata.make, None); + } +} + +#[test] +fn xmp_iso_rejects_out_of_range_values() { + for value in ["4294967296", "0", "-1", "NaN", "inf", "bad"] { + let packet = iso_packet(&format!(r#"exif:ISOSpeedRatings="{value}""#), ""); + let d = parse_xmp(&packet).unwrap(); + assert_eq!(d.metadata.iso, None, "{value}"); + assert_eq!(d.properties.get("exif:ISOSpeedRatings").map(|v| v.first().map(String::as_str)), Some(Some(value))); + } + let packet = iso_packet("", "4294967296"); + assert_eq!(parse_xmp(&packet).unwrap().metadata.iso, None); +} + +#[test] +fn xmp_iso_invalid_primary_allows_fallback() { + for primary in ["0.5", "1.0000000000000001", "160.9", "4294967296", "0", "-1", "bad", "NaN"] { + let packet = iso_packet(&format!(r#"exif:ISOSpeedRatings="{primary}" exifEX:PhotographicSensitivity="200""#), ""); + assert_eq!(parse_xmp(&packet).unwrap().metadata.iso, Some(200), "{primary}"); + } +} + +#[test] +fn xmp_iso_preserves_valid_values() { + for (value, expected) in [ + ("200", 200), + ("200.0", 200), + ("2e2", 200), + ("400/2", 200), + ("1.5/0.5", 3), + ("0.3/0.1", 3), + ("1.0000000000000001/1.0000000000000001", 1), + ("1e39/1e39", 1), + ("1e39/5e38", 2), + ("1e39/2e38", 5), + ("4294967295", u32::MAX), + ] { + let packet = iso_packet(&format!(r#"exif:ISOSpeedRatings="{value}""#), ""); + assert_eq!(parse_xmp(&packet).unwrap().metadata.iso, Some(expected), "{value}"); + } + assert_eq!(parse_xmp(&iso_packet("", "640")).unwrap().metadata.iso, Some(640)); + let packet = iso_packet(r#"exif:ISOSpeedRatings="100" exifEX:PhotographicSensitivity="200""#, ""); + assert_eq!(parse_xmp(&packet).unwrap().metadata.iso, Some(100)); +} + +#[test] +fn xmp_iso_accepts_equal_coefficients_above_u128() { + let value = "340282366920938463463374607431768211456/340282366920938463463374607431768211456"; + assert_eq!(parse_xmp(&iso_packet(&format!(r#"exif:ISOSpeedRatings="{value}""#), "")).unwrap().metadata.iso, Some(1)); +} + +#[test] +fn xmp_iso_rejected_value_is_not_written() { + let packet = iso_packet(r#"exif:ISOSpeedRatings="0.5""#, r#"kept"#); + let parsed = parse_xmp(&packet).unwrap(); + assert_eq!(parsed.metadata.iso, None); + assert_eq!(parsed.metadata.title.as_deref(), Some("kept")); + let written = write_xmp(&parsed.metadata); + assert!(!written.contains("ISOSpeedRatings"), "{written}"); +} + +#[test] +fn xmp_iso_writer_omits_zero() { + let written = write_xmp(&Metadata { iso: Some(0), ..Default::default() }); + assert!(!written.contains("ISOSpeedRatings"), "{written}"); + + let written = write_xmp(&Metadata { iso: Some(200), ..Default::default() }); + assert!(written.contains("200"), "{written}"); +} + #[test] fn negative_xmp_pixel_width_is_ignored() { let read = |attrs: &str| { @@ -210,6 +313,119 @@ fn negative_xmp_pixel_width_is_ignored() { assert_eq!(read(r#"exif:PixelXDimension="-1""#), None); } +fn xmp_dimensions(body: &str) -> XmpData { + let (attributes, elements) = body.split_once('|').unwrap_or((body, "")); + let packet = format!( + r#" + {elements} + "# + ); + parse_xmp(&packet).expect("procedural XMP dimension fixture parses") +} + +#[test] +fn xmp_dimensions_reject_fractional_values() { + for (field, value) in [ + ("tiff:ImageWidth", "640.9"), + ("exif:PixelXDimension", "640.9"), + ("tiff:ImageLength", "480.9"), + ("exif:PixelYDimension", "480.9"), + ("tiff:ImageWidth", "1.0000000000000001"), + ] { + let data = xmp_dimensions(&format!(r#"{field}="{value}""#)); + assert_eq!(data.metadata.width, None, "{field}={value}"); + assert_eq!(data.metadata.height, None, "{field}={value}"); + assert_eq!(data.properties.get(field).map(Vec::as_slice), Some([value.to_owned()].as_slice()), "raw {field}={value}"); + } +} + +#[test] +fn xmp_dimensions_reject_out_of_range_values() { + for (field, value) in [ + ("tiff:ImageWidth", "4294967296"), + ("exif:PixelXDimension", "4294967296"), + ("tiff:ImageLength", "4294967296"), + ("exif:PixelYDimension", "4294967296"), + ("tiff:ImageWidth", "-1"), + ("tiff:ImageWidth", "-0"), + ("tiff:ImageWidth", "0/0"), + ("exif:PixelXDimension", "-1"), + ("tiff:ImageLength", "-1"), + ("exif:PixelYDimension", "-1"), + ("tiff:ImageWidth", "1e100000"), + ("tiff:ImageWidth", "NaN"), + ("tiff:ImageWidth", "Infinity"), + ("tiff:ImageWidth", "not-a-number"), + ] { + let data = xmp_dimensions(&format!(r#"{field}="{value}""#)); + assert_eq!(data.metadata.width, None, "{field}={value}"); + assert_eq!(data.metadata.height, None, "{field}={value}"); + assert_eq!(data.properties.get(field).map(Vec::as_slice), Some([value.to_owned()].as_slice()), "raw {field}={value}"); + } +} + +#[test] +fn xmp_dimensions_invalid_primary_allows_fallback() { + let width = xmp_dimensions(r#"tiff:ImageWidth="0.5" exif:PixelXDimension="640""#); + assert_eq!(width.metadata.width, Some(640)); + let height = xmp_dimensions(r#"tiff:ImageLength="480.9" exif:PixelYDimension="480""#); + assert_eq!(height.metadata.height, Some(480)); +} + +#[test] +fn xmp_dimensions_preserve_valid_values() { + for (field, value, expected) in [ + ("tiff:ImageWidth", "0", 0), + ("tiff:ImageWidth", "0001", 1), + ("tiff:ImageWidth", "0/1e-129", 0), + ("tiff:ImageWidth", "1e-400/1e-400", 1), + ("tiff:ImageWidth", "1e400/1e400", 1), + ("tiff:ImageWidth", "4294967295", u32::MAX), + ("exif:PixelXDimension", "640.0", 640), + ("tiff:ImageLength", "960/2", 480), + ("exif:PixelYDimension", "48e1", 480), + ("tiff:ImageWidth", "1e39/1e39", 1), + ] { + let data = xmp_dimensions(&format!(r#"{field}="{value}""#)); + let actual = if matches!(field, "tiff:ImageWidth" | "exif:PixelXDimension") { data.metadata.width } else { data.metadata.height }; + assert_eq!(actual, Some(expected), "{field}={value}"); + } + assert_eq!(xmp_dimensions(r#"tiff:ImageWidth="640" exif:PixelXDimension="800""#).metadata.width, Some(640)); + assert_eq!(xmp_dimensions(r#"tiff:ImageLength="480" exif:PixelYDimension="800""#).metadata.height, Some(480)); + let element = xmp_dimensions(r#"|640480"#); + assert_eq!(element.metadata.width, Some(640)); + assert_eq!(element.metadata.height, Some(480)); + let sequence = xmp_dimensions(r#"|640"#); + assert_eq!(sequence.metadata.width, Some(640)); +} + +#[test] +fn xmp_dimensions_preserve_large_exact_ratio() { + for coefficient in ["999999999999999999999999999999999999999".to_owned(), "9".repeat(1024)] { + let data = xmp_dimensions(&format!(r#"tiff:ImageWidth="{coefficient}/{coefficient}""#)); + assert_eq!(data.metadata.width, Some(1), "coefficient length {}", coefficient.len()); + } + let denominator = "9".repeat(1024); + let numerator = format!("1{}8", "9".repeat(1023)); + let data = xmp_dimensions(&format!(r#"tiff:ImageWidth="{numerator}/{denominator}""#)); + assert_eq!(data.metadata.width, Some(2)); + + let numerator = format!("{}8", "9".repeat(1023)); + let data = xmp_dimensions(&format!(r#"tiff:ImageWidth="{numerator}/{denominator}""#)); + assert_eq!(data.metadata.width, None); +} + +#[test] +fn xmp_dimensions_rejected_values_are_not_written() { + let data = xmp_dimensions(r#"tiff:ImageWidth="640.9" dc:title="Keep this""#); + assert_eq!(data.metadata.width, None); + assert_eq!(data.metadata.title.as_deref(), Some("Keep this")); + assert_eq!(data.properties.get("tiff:ImageWidth").map(Vec::as_slice), Some(["640.9".to_owned()].as_slice())); + let written = write_xmp(&data.metadata); + assert!(!written.contains("tiff:ImageWidth")); + assert_eq!(parse_xmp(&written).unwrap().metadata.width, None); +} + #[test] fn malformed_inputs() { assert!(parse_xmp("").is_ok() || parse_xmp("").is_err()); @@ -240,11 +456,20 @@ fn lc_extras_and_crs_prefix() { #[test] fn gps_coords() { - assert_eq!(parse_gps_coord("45,30N"), Some(45.5)); - assert_eq!(parse_gps_coord("45,30,36S"), Some(-45.51)); - assert_eq!(parse_gps_coord("45X"), None); - assert_eq!(parse_gps_coord(""), None); - assert_eq!(parse_gps_coord("é"), None); + let lat = ['N', 'S']; + assert_eq!(parse_gps_coord("45,30N", lat), Some(45.5)); + assert_eq!(parse_gps_coord("45,30,36S", lat), Some(-45.51)); + assert_eq!(parse_gps_coord("45X", lat), None); + assert_eq!(parse_gps_coord("", lat), None); + assert_eq!(parse_gps_coord("é", lat), None); + assert_eq!(parse_gps_coord("-10,30N", lat), None, "the sign is the letter's"); + assert_eq!(parse_gps_coord("10,-30N", lat), None); + assert_eq!(parse_gps_coord("10,30E", lat), None, "E is a longitude's"); + assert_eq!(parse_gps_position("10,30E", "20,15N"), None); + assert_eq!(parse_gps_position("10,30N", "20,15W"), Some((10.5, -20.25))); + assert_eq!(parse_gps_position("95,0N", "20,0E"), None, "off the globe"); + assert_eq!(parse_gps_position("10,0N", "180,0.1E"), None); + assert_eq!(parse_gps_position("90,0S", "180,0W"), Some((-90.0, -180.0))); assert_eq!(fmt_gps_coord(-45.51, 'N', 'S'), "45,30.600000S"); assert_eq!(fmt_gps_coord(10.9999999999, 'N', 'S'), "11,0.000000N"); assert_eq!(fmt_gps_coord(-0.9999999999, 'E', 'W'), "1,0.000000W"); @@ -268,6 +493,49 @@ fn xmp_gps_out_of_range_rejected() { assert_eq!(gps("90,0S", "180,0W"), Some(Gps { latitude: -90.0, longitude: -180.0, altitude: None })); } +#[test] +fn xmp_gps_component_bounds() { + let parse = |lat: &str, lon: &str| { + let x = format!( + r#""# + ); + parse_xmp(&x).unwrap() + }; + + for (lat, lon) in [ + ("45,60N", "7,15E"), + ("45,99N", "7,15E"), + ("45,0,60N", "7,15E"), + ("45,0,99N", "7,15E"), + ("45,15N", "7,60E"), + ("45,15N", "7,99E"), + ("45,15N", "7,0,60E"), + ("45,15N", "7,0,99E"), + ] { + let xmp = parse(lat, lon); + assert_eq!(xmp.metadata.gps, None, "{lat}, {lon}"); + assert_eq!(xmp.properties.get("exif:GPSLatitude").and_then(|v| v.first()).map(String::as_str), Some(lat)); + assert_eq!(xmp.properties.get("exif:GPSLongitude").and_then(|v| v.first()).map(String::as_str), Some(lon)); + } + + for (lat, lon, expected) in [ + ("45,59.5N", "7,59.5E", (45.0 + 59.5 / 60.0, 7.0 + 59.5 / 60.0)), + ("45,59,59S", "7,59,59W", (-(45.0 + 59.0 / 60.0 + 59.0 / 3600.0), -(7.0 + 59.0 / 60.0 + 59.0 / 3600.0))), + ("0,0N", "0,0E", (0.0, 0.0)), + ("90,0N", "180,0W", (90.0, -180.0)), + (" 45,30n ", " 7,15e ", (45.5, 7.25)), + ] { + let xmp = parse(lat, lon); + let gps = xmp.metadata.gps.expect("valid XMP GPS components"); + assert!((gps.latitude - expected.0).abs() < 1e-9, "{lat}: {}", gps.latitude); + assert!((gps.longitude - expected.1).abs() < 1e-9, "{lon}: {}", gps.longitude); + } + + assert_eq!(parse("95,0N", "7,0E").metadata.gps, None, "combined latitude range remains checked"); + assert_eq!(parse("45,0N", "200,0E").metadata.gps, None, "combined longitude range remains checked"); +} + /// MWG Region Guidelines v2.0, element form (`rdf:parseType="Resource"`, self-closing `Area`) — how /// Lightroom typically writes it. One named face, one untyped/unnamed region (still usable), one /// region with a description. `AppliedToDimensions` is present but irrelevant: normalized areas @@ -506,7 +774,10 @@ fn writer_drops_characters_xml_1_0_forbids() { let illegal: Vec = x.chars().filter(|&c| (c < '\u{20}' && !matches!(c, '\t' | '\n' | '\r')) || c == '\u{FFFE}' || c == '\u{FFFF}').collect(); assert!(illegal.is_empty(), "illegal XML chars in output: {illegal:?}"); - assert!(x.contains(">abcde\tf\ng\rh<"), "allowed characters must survive: {x}"); + // a carriage return is written as a reference: a literal one would read back as a line feed (XML end-of-line handling) + assert!(x.contains(">abcde\tf\ng h<"), "allowed characters must survive: {x}"); + let back = parse_xmp(&x).unwrap(); + assert_eq!(back.metadata.caption.as_deref(), Some("abcde\tf\ng\rh"), "and read back unchanged"); } #[test] @@ -544,6 +815,7 @@ fn flattened_keys_stay_within_the_cumulative_budget() { assert!(total <= MAX_KEY_BYTES, "flattened-key text {total} is over the {MAX_KEY_BYTES} budget"); assert!(d.properties.len() < 3000, "keys stop being added once the budget is spent"); assert!(!d.properties.is_empty(), "properties parsed before the budget are kept"); + assert!(d.properties_truncated, "omitted flattened keys are reported to callers"); } #[test] @@ -579,6 +851,15 @@ fn malformed_capture_time_without_fallback_is_not_projected() { assert_eq!(parsed.properties["exif:DateTimeOriginal"], vec!["2026-01-01T".to_string()]); } +#[test] +fn a_start_tag_with_thousands_of_namespace_declarations_is_refused() { + let decls: String = (0..2000).map(|i| format!(" xmlns:p{i}=\"urn:x{i}\"")).collect(); + let xmp = format!( + "" + ); + assert!(matches!(parse_xmp(&xmp), Err(XmpError::TooComplex))); +} + #[test] fn many_ampersands_before_one_semicolon_are_read_once() { // 200 000 `&` then a `;`: each `&` once searched the whole rest of the attribute for the `;` @@ -617,7 +898,7 @@ fn the_callers_namespace_is_written_and_read_only_when_passed() { let x = write_lc(&Metadata::default(), &[("flag", "pick")]); assert!(x.contains("x:xmptk=\"Editor\"")); assert!(x.contains( - "xmlns:Iptc4xmpCore=\"http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/\"\n xmlns:lc=\"http://ns.example.com/lc/1.0/\"\n xmlns:mwg-rs=" + "xmlns:Iptc4xmpCore=\"http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/\"\n xmlns:Iptc4xmpExt=\"http://iptc.org/std/Iptc4xmpExt/2008-02-29/\"\n xmlns:lc=\"http://ns.example.com/lc/1.0/\"\n xmlns:mwg-rs=" )); assert!(!x.contains("xmlns:crs")); let renamed = x.replace("xmlns:lc=", "xmlns:zz=").replace("lc:flag", "zz:flag"); @@ -671,3 +952,181 @@ fn the_toolkit_is_read_and_written_back() { let bare = r#""#; assert_eq!(crate::parse_xmp(bare).unwrap().toolkit, None); } + +fn orientation_packet(value: &str) -> String { + format!( + r#""# + ) +} + +#[test] +fn xmp_orientation_rejects_fractional_values_and_keeps_raw_properties() { + for value in ["6.5", "6.9", "1.5", "8.9", "13/2"] { + let parsed = parse_xmp(&orientation_packet(value)).unwrap(); + assert_eq!(parsed.metadata.orientation, None, "{value}"); + assert_eq!(parsed.properties["tiff:Orientation"], vec![value.to_string()], "{value}"); + assert!(!write_xmp(&parsed.metadata).contains("tiff:Orientation"), "rejected {value} must not become a written orientation"); + } +} + +#[test] +fn xmp_orientation_integral_forms_keep_their_meaning() { + for code in 1..=8 { + let parsed = parse_xmp(&orientation_packet(&code.to_string())).unwrap(); + assert_eq!(parsed.metadata.orientation.map(|orientation| orientation.to_exif()), Some(code)); + } + for value in ["6.0", "6/1", " 6 ", " 6.0 "] { + let parsed = parse_xmp(&orientation_packet(value)).unwrap(); + assert_eq!(parsed.metadata.orientation.map(|orientation| orientation.to_exif()), Some(6), "{value}"); + } +} + +#[test] +fn xmp_orientation_rejects_existing_invalid_values() { + for value in ["0", "9", "-1", "nan", "inf"] { + let parsed = parse_xmp(&orientation_packet(value)).unwrap(); + assert_eq!(parsed.metadata.orientation, None, "{value}"); + assert_eq!(parsed.properties["tiff:Orientation"], vec![value.to_string()], "{value}"); + } +} + +/// References resolve, anything else (unknown, unterminated, or a `;` too far away to end one) stays as +/// written, and a value that is nothing but `&` is handled in one pass. +#[test] +fn attribute_references_resolve_and_hostile_ones_stay_verbatim() { + assert_eq!(unescape_attr("a&bAB<>"'"), "a&bAB<>\"'"); + assert_eq!(unescape_attr("&bogus; & &#xZZ; tail&"), "&bogus; & &#xZZ; tail&"); + let far = format!("&{};", "x".repeat(40)); + assert_eq!(unescape_attr(&far), far); + let many = "&".repeat(1 << 20); + assert_eq!(unescape_attr(&many), many); + // multi-byte text around a reference is never cut inside a character + assert_eq!(unescape_attr("é&日&é;"), "é&日&é;"); + // leading zeros are valid in numeric references, however many there are + assert_eq!(unescape_attr("AB"), "AB"); + let zeros = format!("&#{};", "0".repeat(1 << 20)); + assert_eq!(unescape_attr(&zeros), zeros); +} + +/// A small packet keeps every property and is not marked truncated. +#[test] +fn a_packet_within_the_budget_keeps_a_complete_inventory() { + let d = parse_xmp(r#""#).unwrap(); + assert!(!d.properties_truncated); + assert_eq!(d.properties.len(), 1); +} + +/// A later description that repeats `LocationShown` with an empty item must not erase the city +/// an earlier description named: the packet still holds a place the export has to refuse. +#[test] +fn a_later_empty_location_keeps_an_earlier_city() { + let packet = concat!( + r#""#, + r#""#, + r#"Rome"#, + r#""#, + r#""# + ); + let parsed = parse_xmp(packet).expect("valid XMP"); + assert!(parsed.has_location_metadata(), "the earlier city is still in the packet"); + let city = parsed + .values + .get("Iptc4xmpExt:LocationShown") + .and_then(|value| value.items().first()) + .and_then(|item| item.field("Iptc4xmpExt:City")) + .and_then(XmpValue::text); + assert_eq!(city, Some("Rome")); +} + +/// GPS nested in a struct array item flattens to the item's (empty) text; the tree inventory still +/// names it. +#[test] +fn gps_inside_a_bag_item_counts_as_location() { + let packet = concat!( + r#""#, + r#""#, + r#"37,46.494N"# + ); + let parsed = parse_xmp(packet).expect("valid XMP"); + assert!(parsed.properties.keys().all(|key| !key.contains("GPS")), "flattening drops the item's fields"); + assert!(parsed.names_location); + assert!(parsed.has_location_metadata()); +} + +/// A location struct nested in another struct is judged by its own fields, not by unrelated text +/// next to it. +#[test] +fn a_nested_location_is_judged_by_its_own_fields() { + let packet = |city: &str| { + format!( + r#"Bridge{city}"# + ) + }; + assert!(!parse_xmp(&packet("")).expect("valid XMP").has_location_metadata(), "a blank place beside a title"); + assert!(parse_xmp(&packet("Rome")).expect("valid XMP").has_location_metadata()); +} + +/// A place inside a struct array item has no flattened key of its own; the tree inventory finds +/// it, and a blank one there still doesn't count. +#[test] +fn a_location_inside_an_array_item_counts_unless_blank() { + let packet = |city: &str| { + format!( + r#"{city}"# + ) + }; + let with_city = parse_xmp(&packet("Rome")).expect("valid XMP"); + assert!(with_city.properties.keys().all(|key| !key.contains("Location")), "flattening drops the item's fields"); + assert!(with_city.has_location_metadata()); + assert!(!parse_xmp(&packet(" ")).expect("valid XMP").has_location_metadata(), "a blank place is no location"); +} + +/// A struct repeated in one description keeps its first copy in `values` when that copy has text +/// elsewhere; a place only the later copy names must still be found. +#[test] +fn a_place_in_a_later_repeated_struct_counts() { + let packet = concat!( + r#""#, + r#"Bridge"#, + r#"Rome"#, + r#""# + ); + assert!(parse_xmp(packet).expect("valid XMP").has_location_metadata()); +} + +/// The same `rdf:Description` can repeat `LocationShown`; a later blank item must not replace the +/// city of the earlier one (its `rdf:li` flattens to empty text, so only the structured value has it). +#[test] +fn a_repeated_empty_location_in_one_description_keeps_the_city() { + let packet = concat!( + r#""#, + r#""#, + r#"Rome"#, + r#""#, + r#""# + ); + let parsed = parse_xmp(packet).expect("valid XMP"); + assert!(parsed.has_location_metadata(), "the earlier city is still in the packet"); + let city = parsed + .values + .get("Iptc4xmpExt:LocationShown") + .and_then(|value| value.items().first()) + .and_then(|item| item.field("Iptc4xmpExt:City")) + .and_then(XmpValue::text); + assert_eq!(city, Some("Rome")); +} + +/// One walker answers both the parser and the Remove Location check: nested text counts, blanks +/// don't, and a tree past the node budget counts as text-bearing instead of being walked in full. +#[test] +fn value_has_text_finds_nested_text_and_stops_at_the_node_budget() { + let blank = XmpValue::Array(vec![XmpValue::Text(" ".into()), XmpValue::Struct(BTreeMap::new())]); + assert!(!value_has_text(&blank)); + let nested = XmpValue::Array(vec![XmpValue::Struct(BTreeMap::from([("Iptc4xmpExt:City".to_string(), XmpValue::Text("Rome".into()))]))]); + assert!(value_has_text(&nested)); + let oversized = XmpValue::Array(vec![XmpValue::Text(String::new()); MAX_NODES + 1]); + assert!(value_has_text(&oversized), "a tree past the node budget is treated as text-bearing"); + let data = XmpData { values: BTreeMap::from([("Iptc4xmpExt:LocationShown".to_string(), nested)]), ..XmpData::default() }; + let data = XmpData { properties: BTreeMap::from([("Iptc4xmpExt:LocationShown".to_string(), vec![String::new()])]), ..data }; + assert!(data.has_location_metadata(), "the structured value's city is found"); +} diff --git a/crates/metadata/src/xmp_merge.rs b/crates/metadata/src/xmp_merge.rs index 3056225b0..33714269a 100644 --- a/crates/metadata/src/xmp_merge.rs +++ b/crates/metadata/src/xmp_merge.rs @@ -23,7 +23,7 @@ use std::collections::{BTreeSet, HashSet}; use quick_xml::events::Event; -use crate::xmp::{MAX_OPAQUE_NAME_BYTES, XmpError, XmpNamespace, canonical_name, parse_xmp_with, unescape_attr}; +use crate::xmp::{MAX_OPAQUE_NAME_BYTES, XmpError, XmpNamespace, canonical_name, check_scopes, parse_xmp_with, unescape_attr}; /// Which properties [`merge_xmp`] takes from the fresh packet. Names are canonical /// `prefix:name` (prefixes as [`crate::parse_xmp`] reports them); `prefix:*` is a whole namespace. @@ -77,6 +77,8 @@ struct Scan { descs: Vec, /// Where the first `` starts. rdf_close: Option, + /// The span of a self-closing `` (used only when there is no ``). + rdf_empty: Option<(usize, usize)>, } enum Kind { @@ -134,6 +136,7 @@ fn scan(s: &str, extra: &[XmpNamespace]) -> Result { raw_attrs.push((k, v)); } scopes.push(scope); + check_scopes(&scopes)?; let raw_name = String::from_utf8_lossy(e.name().as_ref()).into_owned(); let name = canonical_name(&raw_name, &scopes, extra, &mut opaque_budget)?; let kind = match stack.last() { @@ -150,6 +153,9 @@ fn scan(s: &str, extra: &[XmpNamespace]) -> Result { _ => Kind::Other, }; if empty { + if matches!(kind, Kind::Rdf) { + out.rdf_empty = out.rdf_empty.or(Some((before, after))); + } scopes.pop(); if let Kind::Prop { desc, start, name } = kind && let Some(d) = out.descs.get_mut(desc) @@ -215,7 +221,9 @@ pub fn merge_xmp(existing: &str, fresh: &str, rules: MergeRules) -> Result = fresh_desc.props.iter().map(|p| p.name.as_str()).collect(); @@ -263,8 +271,16 @@ pub fn merge_xmp(existing: &str, fresh: &str, rules: MergeRules) -> Result"); if written > 0 { - let at = old.get(..rdf_close).map_or(rdf_close, |h| h.trim_end().len()); - edits.push((at, at, block)); + if let Some(rdf_close) = ex.rdf_close { + let at = old.get(..rdf_close).map_or(rdf_close, |h| h.trim_end().len()); + edits.push((at, at, block)); + } else if let Some((s, e)) = ex.rdf_empty { + // a self-closing `` is opened up around the new description + let tag = old.get(s..e).ok_or_else(|| XmpError::Xml("bad span".into()))?; + let open = tag.trim_end().strip_suffix("/>").ok_or_else(|| XmpError::Xml("bad rdf:RDF tag".into()))?; + let name = tag.get(1..).and_then(|t| t.split(|c: char| c.is_whitespace() || c == '/' || c == '>').next()).unwrap_or("rdf:RDF"); + edits.push((s, e, format!("{open}>{block}\n"))); + } } edits.sort_by_key(|e| (e.0, e.1)); // overlapping edits can't happen (properties sit inside their description, which is either @@ -404,6 +420,16 @@ mod tests { assert_eq!(merge_xmp(&again, &fresh(1, &[]), RULES).unwrap(), again, "idempotent"); } + #[test] + fn a_self_closing_rdf_element_is_opened_not_replaced() { + let ex = ""; + let out = merge_xmp(ex, &fresh(3, &[]), RULES).unwrap(); + assert!(out.starts_with(""), "{out}"); + assert_eq!(parse_xmp(&out).unwrap().properties.get("xmp:Rating"), Some(&vec!["3".to_string()])); + assert!(merge_xmp("", &fresh(3, &[]), RULES).is_err(), "no RDF element at all"); + } + #[test] fn owned_if_present_and_prefix_independence() { // the existing packet binds the namespaces to other prefixes, and has a capture time From 392b8720cd25ba00bb1cc9d3a8b3c895cfa04259 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:04:07 -0500 Subject: [PATCH 02/10] craft-psd: less copying and compression work when encoding planes (#1043) - PackBits scans each row once; RLE planes encode their rows on worker threads on native targets (rayon), keeping the exact packet boundaries and the fallback when RLE does not pay. - ZIP output grows in bounded blocks and is compacted to its encoded size, instead of reserving the raw size; native threads reuse a finished encoder's working buffers. - 16/32-bit prediction works on owned buffers in one pass. - Filter-effects planes and pattern blocks encode from borrowed rows and owned buffers without the intermediate copies. The encoded bytes are unchanged. Ported from storytold/photocraft#7797 (its crates/psd part), at a1ee8355a4651d379286cd13f92dabe4344e295f. Co-authored-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> --- Cargo.lock | 1 + crates/psd/Cargo.toml | 3 + crates/psd/src/compression.rs | 295 ++++++++++++++++++++++------ crates/psd/src/compression/tests.rs | 292 +++++++++++++++++++++++++++ crates/psd/src/filter_effects.rs | 67 ++++++- crates/psd/src/patterns.rs | 61 +++--- crates/psd/src/patterns/tests.rs | 121 ++++++++++++ 7 files changed, 743 insertions(+), 97 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 3a66892d8..b6b4afa92 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1711,6 +1711,7 @@ dependencies = [ "encoding_rs", "flate2", "proptest", + "rayon", "serde", "thiserror 2.0.21", ] diff --git a/crates/psd/Cargo.toml b/crates/psd/Cargo.toml index d9d02abae..c9aa6dbe0 100644 --- a/crates/psd/Cargo.toml +++ b/crates/psd/Cargo.toml @@ -21,6 +21,9 @@ flate2 = { version = "1.1.10", default-features = false, features = ["zlib-rs", encoding_rs = { workspace = true } serde = { workspace = true, optional = true } +[target.'cfg(not(target_arch = "wasm32"))'.dependencies] +rayon = { workspace = true } + [dev-dependencies] proptest = { workspace = true } craft-psd = { path = ".", features = ["testgen"] } diff --git a/crates/psd/src/compression.rs b/crates/psd/src/compression.rs index 7d05f5471..d684525a8 100644 --- a/crates/psd/src/compression.rs +++ b/crates/psd/src/compression.rs @@ -68,34 +68,90 @@ pub mod packbits { /// Encodes one row, appending to `out`. Never emits the `-128` no-op. pub fn encode(src: &[u8], out: &mut Vec) { + // Constant 16-bit samples with different bytes contain only literal packets. + if src.len() >= 128 + && let Some(&[a, b]) = src.first_chunk::<2>() + && a != b + { + let (blocks, tail) = src.as_chunks::<8>(); + if blocks.iter().all(|v| *v == [a, b, a, b, a, b, a, b]) && tail.iter().zip([a, b].into_iter().cycle()).all(|(&v, p)| v == p) { + for chunk in src.chunks(128) { + out.push((chunk.len() - 1) as u8); + out.extend_from_slice(chunk); + } + return; + } + } + packets(src, |header, bytes| { + out.push(header); + out.extend_from_slice(bytes); + }); + } + + /// The encoded row length, without allocating or writing its payload. + pub fn encoded_len(src: &[u8]) -> usize { + let mut len = 0; + packets(src, |_, bytes| len += 1 + bytes.len()); + len + } + + /// Visit PackBits run and literal packets in their deterministic encoding order. + fn packets(src: &[u8], mut emit: impl FnMut(u8, &[u8])) { let n = src.len(); let mut i = 0; while i < n { // Length of the run starting at i. let mut run = 1; - while i + run < n && run < 128 && src[i + run] == src[i] { + while i + run < n && run < 8 && src[i + run] == src[i] { run += 1; } + if run == 8 { + while run + 8 <= 128 && src.get(i + run..).and_then(|s| s.first_chunk::<8>()) == Some(&[src[i]; 8]) { + run += 8; + } + while i + run < n && run < 128 && src[i + run] == src[i] { + run += 1; + } + } // A literal never stops before a 2-byte run, so any run of >= 2 // seen here starts a packet boundary: a run packet is never larger. if run >= 2 { - out.push((1i16 - run as i16) as i8 as u8); - out.push(src[i]); + emit((1i16 - run as i16) as i8 as u8, &src[i..i + 1]); i += run; continue; } // Literal: extend until a run of >= 3 begins or 128 bytes. let start = i; let mut len = 0; - while i < n && len < 128 { + // Short packets keep the scalar scan; lookahead pays only on longer literals. + while i < n && len < 16 { if i + 2 < n && src[i] == src[i + 1] && src[i] == src[i + 2] { break; } i += 1; len += 1; } - out.push((len - 1) as u8); - out.extend_from_slice(&src[start..start + len]); + if len == 16 { + while len + 8 <= 128 { + let Some(&[a, b, c, d, e, f, g, h, j, k]) = src.get(i..).and_then(|s| s.first_chunk::<10>()) else { break }; + let word = u64::from_le_bytes([a, b, c, d, e, f, g, h]); + let different = (word ^ u64::from_le_bytes([b, c, d, e, f, g, h, j])) | (word ^ u64::from_le_bytes([c, d, e, f, g, h, j, k])); + // A zero byte marks a triple; scalar scanning keeps the first boundary. + if different.wrapping_sub(0x0101010101010101) & !different & 0x8080808080808080 != 0 { + break; + } + i += 8; + len += 8; + } + while i < n && len < 128 { + if i + 2 < n && src[i] == src[i + 1] && src[i] == src[i + 2] { + break; + } + i += 1; + len += 1; + } + } + emit((len - 1) as u8, &src[start..start + len]); } } @@ -369,82 +425,178 @@ fn decode_rle(data: &[u8], layout: &PlaneLayout, total: usize) -> Result Ok(out) } +/// Encode independent row chunks and join their count tables and payloads in order. fn encode_rle(decoded: &[u8], layout: &PlaneLayout) -> Result> { let rows = layout.rows()?; - let rb = layout.row_bytes(); - let cs = layout.count_size(); if rows == 0 { return Ok(Vec::new()); } - // Rows are independent, so they pack in parallel chunks of rows. Two passes: the first only - // measures each packed row, so the output is allocated once at its exact size and the second - // packs every row straight into its place. Collecting packed chunks and then copying them into - // the output held the encoded image twice, with growth slack, at the save's peak (photocraft#6607). + let rb = layout.row_bytes(); + let cs = layout.count_size(); let threads = if cfg!(target_arch = "wasm32") { 1 } else { std::thread::available_parallelism().map_or(1, |v| v.get()).clamp(1, 32) }; - let chunk_rows = rows.div_ceil(threads).max(1); - let row = |r: usize| &decoded[r * rb..(r + 1) * rb]; - let mut lens = vec![0usize; rows]; - for_each_chunk(threads, lens.chunks_mut(chunk_rows).enumerate().collect(), |(c, lens): (usize, &mut [usize])| { - let mut scratch = Vec::with_capacity(rb + rb.div_ceil(128)); - for (i, len) in lens.iter_mut().enumerate() { - scratch.clear(); - packbits::encode(row(c * chunk_rows + i), &mut scratch); - *len = scratch.len(); - } - })?; - let mut out = vec![0u8; rows * cs + lens.iter().sum::()]; - let (counts, mut packed) = out.split_at_mut(rows * cs); - for (count, &n) in counts.chunks_exact_mut(cs).zip(&lens) { - if cs == 2 { - let n = u16::try_from(n).map_err(|_| PsdError::LimitExceeded("RLE row exceeds 65535 bytes (use PSB)"))?; - count.copy_from_slice(&n.to_be_bytes()); - } else { - let n = u32::try_from(n).map_err(|_| PsdError::LimitExceeded("RLE row too large"))?; - count.copy_from_slice(&n.to_be_bytes()); - } + let step = rows.div_ceil(threads).max(1); + let ranges: Vec<_> = (0..rows).step_by(step).map(|start| start..start.saturating_add(step).min(rows)).collect(); + // Scan packets once. Each worker owns its count table and payload; joining copies each + // payload once, trading temporary encoded bytes for the former second packet scan. + let mut chunks: Vec<_> = map_chunks(threads, ranges.clone(), |range| -> Result> { + let table = range.len().checked_mul(cs).ok_or(PsdError::LimitExceeded("RLE table overflow"))?; + let hint = + if cfg!(target_arch = "wasm32") { table } else { table.saturating_add(range.len().saturating_mul(rb.saturating_add(rb.div_ceil(128)))) }; + encode_rle_chunk(decoded, layout, range, hint) + })? + .into_iter() + .collect::>()?; + if chunks.len() == 1 { + return Ok(chunks.remove(0).into_boxed_slice().into_vec()); } - let mut parts = Vec::with_capacity(threads); - for (c, lens) in lens.chunks(chunk_rows).enumerate() { - let (part, rest) = packed.split_at_mut(lens.iter().sum()); - parts.push((c, lens, part)); - packed = rest; + let total = chunks.iter().try_fold(0usize, |n, c| n.checked_add(c.len()).ok_or(PsdError::LimitExceeded("RLE size overflow")))?; + let table = rows.checked_mul(cs).ok_or(PsdError::LimitExceeded("RLE table overflow"))?; + let mut out = Vec::new(); + out.try_reserve_exact(total).map_err(|_| PsdError::OutOfMemory("not enough memory for encoded RLE"))?; + out.resize(table, 0); + for (range, chunk) in ranges.into_iter().zip(chunks) { + let counts = range.len() * cs; + out[range.start * cs..range.end * cs].copy_from_slice(&chunk[..counts]); + out.extend_from_slice(&chunk[counts..]); } - for_each_chunk(threads, parts, |(c, lens, mut part): (usize, &[usize], &mut [u8])| { - let mut scratch = Vec::with_capacity(rb + rb.div_ceil(128)); - for (i, &n) in lens.iter().enumerate() { - scratch.clear(); - packbits::encode(row(c * chunk_rows + i), &mut scratch); - let (dst, rest) = std::mem::take(&mut part).split_at_mut(n); - dst.copy_from_slice(&scratch); - part = rest; - } - })?; Ok(out) } -/// Runs `f` on every item, each on its own thread unless `threads` is 1 (wasm32). -fn for_each_chunk(threads: usize, items: Vec, f: impl Fn(T) + Sync) -> Result<()> { - if threads == 1 { - items.into_iter().for_each(f); - return Ok(()); - } - std::thread::scope(|scope| { - let handles: Vec<_> = items.into_iter().map(|item| scope.spawn(|| f(item))).collect(); - // Join every worker instead of aborting the scope with a thread panic. - let mut ok = true; - for h in handles { - ok &= h.join().is_ok(); +/// Encode one worker's rows, measuring packets only when the spare capacity cannot bound them. +fn encode_rle_chunk(decoded: &[u8], layout: &PlaneLayout, range: std::ops::Range, hint: usize) -> Result> { + const OOM: PsdError = PsdError::OutOfMemory("not enough memory for encoded RLE"); + let rb = layout.row_bytes(); + let cs = layout.count_size(); + let table = range.len().checked_mul(cs).ok_or(PsdError::LimitExceeded("RLE table overflow"))?; + let mut out = Vec::new(); + out.try_reserve_exact(hint.max(table)).or_else(|_| out.try_reserve_exact(table)).map_err(|_| OOM)?; + out.resize(table, 0); + let row_bound = rb.saturating_add(rb.div_ceil(128)); + for (i, row) in range.enumerate() { + let src = &decoded[row * rb..(row + 1) * rb]; + if out.capacity() - out.len() < row_bound { + let needed = packbits::encoded_len(src); + out.try_reserve(needed).or_else(|_| out.try_reserve_exact(needed)).map_err(|_| OOM)?; } - if ok { Ok(()) } else { Err(PsdError::LimitExceeded("an RLE worker panicked")) } - }) + let start = out.len(); + packbits::encode(src, &mut out); + let n = out.len() - start; + let count = &mut out[i * cs..(i + 1) * cs]; + if cs == 2 { + count.copy_from_slice(&u16::try_from(n).map_err(|_| PsdError::LimitExceeded("RLE row exceeds 65535 bytes (use PSB)"))?.to_be_bytes()); + } else { + count.copy_from_slice(&u32::try_from(n).map_err(|_| PsdError::LimitExceeded("RLE row too large"))?.to_be_bytes()); + } + } + Ok(out) +} + +/// Ordered work on the caller's pool, including nested channel and row encoding. +fn map_chunks(threads: usize, items: Vec, f: impl Fn(T) -> R + Sync) -> Result> { + if threads == 1 { + return Ok(items.into_iter().map(f).collect()); + } + #[cfg(not(target_arch = "wasm32"))] + { + use rayon::prelude::*; + std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| items.into_par_iter().map(&f).collect())) + .map_err(|_| PsdError::LimitExceeded("an RLE worker panicked")) + } + #[cfg(target_arch = "wasm32")] + Ok(items.into_iter().map(f).collect()) } /// zlib-compresses `data` at the default level. pub fn zip_compress(data: &[u8]) -> Vec { + #[cfg(not(target_arch = "wasm32"))] + let mut e = zip_encoder(); + #[cfg(target_arch = "wasm32")] let mut e = flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::default()); - // Writing into a Vec cannot fail. + // The in-memory output always accepts the complete write. let _ = e.write_all(data); - e.finish().unwrap_or_default() + #[cfg(not(target_arch = "wasm32"))] + { + if e.try_finish().is_err() { + return Vec::new(); + } + let output = std::mem::take(e.get_mut()); + // Return only the finished codec; the slot stays empty during compression. + let _ = ZIP_ENCODER.try_with(|slot| slot.set(Some(e))); + output.into_vec() + } + #[cfg(target_arch = "wasm32")] + { + e.finish().unwrap_or_default() + } +} + +#[cfg(not(target_arch = "wasm32"))] +thread_local! { + /// One finished codec per thread; encoded output moves out before it is retained. + static ZIP_ENCODER: std::cell::Cell>> = const { std::cell::Cell::new(None) }; +} + +/// Reset a thread's finished encoder without reallocating its fixed-size working buffers. +/// Nested calls and thread teardown can use a fresh encoder without accessing a busy slot. +#[cfg(not(target_arch = "wasm32"))] +fn zip_encoder() -> flate2::write::ZlibEncoder { + if let Some(mut encoder) = ZIP_ENCODER.try_with(std::cell::Cell::take).ok().flatten() + && encoder.reset(ZipOutput::default()).is_ok() + { + return encoder; + } + flate2::write::ZlibEncoder::new(ZipOutput::default(), flate2::Compression::default()) +} + +/// Bounded native output blocks avoid reallocating large ZIP buffers as they grow. +#[cfg(not(target_arch = "wasm32"))] +#[derive(Default)] +struct ZipOutput { + blocks: Vec>, + current: Vec, +} + +#[cfg(not(target_arch = "wasm32"))] +impl std::io::Write for ZipOutput { + /// Accept all input bytes into bounded output blocks without reserving raw image size. + fn write(&mut self, mut bytes: &[u8]) -> std::io::Result { + let len = bytes.len(); + while !bytes.is_empty() { + if self.current.len() == self.current.capacity() { + if !self.current.is_empty() { + self.blocks.push(std::mem::take(&mut self.current)); + } + self.current = Vec::with_capacity(if self.blocks.is_empty() { 8192 } else { 256 << 10 }); + } + let part = &mut self.current; + let (head, rest) = bytes.split_at((part.capacity() - part.len()).min(bytes.len())); + part.extend_from_slice(head); + bytes = rest; + } + Ok(len) + } + + /// Complete a no-op flush because every write is already retained in memory. + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } +} + +#[cfg(not(target_arch = "wasm32"))] +impl ZipOutput { + /// Join encoded ZIP blocks into one vector sized to the compressed payload. + fn into_vec(mut self) -> Vec { + self.blocks.push(self.current); + if self.blocks.len() == 1 { + return self.blocks.pop().unwrap_or_default().into_boxed_slice().into_vec(); + } + let len = self.blocks.iter().fold(0usize, |n, c| n.saturating_add(c.len())); + let mut out = Vec::with_capacity(len); + for part in self.blocks { + out.extend_from_slice(&part); + } + out + } } /// Decompresses a zlib stream, requiring at least `expected` output bytes and @@ -489,6 +641,20 @@ pub fn predict(buf: &mut [u8], layout: &PlaneLayout) -> Result<()> { if rb == 0 { return Ok(()); } + #[cfg(not(target_arch = "wasm32"))] + if buf.len() >= 1 << 20 && matches!(layout.depth, 1 | 8 | 16 | 32) { + use rayon::prelude::*; + let threads = rayon::current_num_threads().min(32); + let rows = buf.len() / rb; + let step = rows.div_ceil(threads).max(1) * rb; + return buf[..rows * rb].par_chunks_mut(step).try_for_each(|part| predict_rows(part, layout)); + } + predict_rows(buf, layout) +} + +/// Apply per-row prediction, reusing scratch for float byte-plane shuffling. +fn predict_rows(buf: &mut [u8], layout: &PlaneLayout) -> Result<()> { + let rb = layout.row_bytes(); let w = layout.width; let mut scratch = Vec::new(); for row in buf.chunks_exact_mut(rb) { @@ -509,7 +675,6 @@ pub fn predict(buf: &mut [u8], layout: &PlaneLayout) -> Result<()> { } } 32 => { - scratch.clear(); scratch.resize(rb, 0); // Split into four byte planes (MSB first) and delta the whole // plane-major row. Plane k's first delta uses the last byte of diff --git a/crates/psd/src/compression/tests.rs b/crates/psd/src/compression/tests.rs index 9529af65b..9f8c61664 100644 --- a/crates/psd/src/compression/tests.rs +++ b/crates/psd/src/compression/tests.rs @@ -137,13 +137,84 @@ fn layout(planes: usize, width: usize, height: usize, depth: u16, version: Versi PlaneLayout { planes, width, height, depth, version } } +/// Verify PackBits round trips empty, literal and repeated-byte inputs. fn pb_roundtrip(src: &[u8]) -> Vec { let enc = packbits::encode_vec(src); + assert_eq!(packbits::encoded_len(src), enc.len()); let dec = packbits::decode(&enc, src.len()).unwrap(); assert_eq!(dec, src, "encoded: {enc:?}"); enc } +/// Compare block scanning with scalar packet boundaries over varied runs and literals. +#[test] +fn packbits_block_scan_preserves_scalar_packets() { + /// Encode PackBits with the previous scalar packet scan as a byte oracle. + fn scalar(src: &[u8]) -> Vec { + let mut out = Vec::new(); + let mut i = 0; + while i < src.len() { + let mut run = 1; + while i + run < src.len() && run < 128 && src[i + run] == src[i] { + run += 1; + } + if run >= 2 { + out.extend_from_slice(&[(1i16 - run as i16) as u8, src[i]]); + i += run; + } else { + let start = i; + while i < src.len() && i - start < 128 { + if i + 2 < src.len() && src[i] == src[i + 1] && src[i] == src[i + 2] { + break; + } + i += 1; + } + out.push((i - start - 1) as u8); + out.extend_from_slice(&src[start..i]); + } + } + out + } + let check = |src: &[u8]| { + let expected = scalar(src); + let mut actual = vec![42]; + packbits::encode(src, &mut actual); + assert_eq!(&actual[1..], expected, "source {src:?}"); + assert_eq!(packbits::encoded_len(src), expected.len()); + assert!(expected.len() <= src.len() + src.len().div_ceil(128)); + }; + for n in 0..=10 { + for mut bits in 0..3usize.pow(n) { + let src: Vec<_> = (0..n) + .map(|_| { + let b = [0, 1, 255][bits % 3]; + bits /= 3; + b + }) + .collect(); + check(&src); + } + } + for start in 0..260 { + for run in [2, 3, 7, 8, 9, 127, 128, 129, 256] { + let mut src: Vec<_> = (0..520).map(|i| (i % 251) as u8).collect(); + src[start..start + run].fill(255); + check(&src); + } + } + for pair in [[0, 1], [0, 255], [128, 127], [255, 255]] { + for n in [127, 128, 129, 255, 256, 257, 12000, 12001] { + let mut src: Vec<_> = pair.into_iter().cycle().take(n).collect(); + check(&src); + for i in [0, 7, 8, 127.min(n - 1), 128.min(n - 1), n - 1] { + src[i] ^= 1; + check(&src); + src[i] ^= 1; + } + } + } +} + #[test] fn zip_absurd_declared_size_is_an_error() { let z = zip_compress(&[7u8; 64]); @@ -452,6 +523,24 @@ fn prediction_32bit_floats_roundtrip() { assert_eq!(decode_planes(Compression::ZipPrediction, &enc, &l).unwrap(), v); } +/// Compare fused float prediction with explicit byte-plane shuffling and delta coding. +#[test] +fn fused_prediction_matches_shuffled_byte_deltas() { + for w in [1, 2, 3, 31, 256, 257] { + let l = layout(2, w, 3, 32, Version::Psb); + let mut actual: Vec<_> = (0..l.decoded_len().unwrap()).map(|i| (i * 31 + i / 7) as u8).collect(); + let expected: Vec<_> = actual + .chunks_exact(w * 4) + .flat_map(|row| { + let shuffled: Vec<_> = (0..4).flat_map(|k| row.iter().skip(k).step_by(4).copied()).collect(); + (0..shuffled.len()).map(move |i| shuffled[i].wrapping_sub(if i == 0 { 0 } else { shuffled[i - 1] })) + }) + .collect(); + predict(&mut actual, &l).unwrap(); + assert_eq!(actual, expected, "width {w}"); + } +} + #[test] fn zip_bomb_is_bounded() { let big = zip_compress(&vec![0u8; 1 << 20]); @@ -463,3 +552,206 @@ fn zip_bomb_is_bounded() { fn zip_garbage_errors() { assert!(zip_decompress(&[1, 2, 3, 4], 4).is_err()); } + +/// Compare bounded ZIP output with the original growing writer over varied inputs. +#[test] +fn zip_output_matches_the_growing_writer() { + let mut seed = 0x12345678u32; + let noise = (0..1_048_577) + .map(|_| { + seed ^= seed << 13; + seed ^= seed >> 17; + seed ^= seed << 5; + seed as u8 + }) + .collect(); + for bytes in [Vec::new(), vec![0; 131072], (0..131072).map(|i| (i * 31 + i / 173) as u8).collect(), noise].iter().cycle().take(12) { + let mut old = flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::default()); + old.write_all(bytes).unwrap(); + let packed = zip_compress(bytes); + assert_eq!(packed, old.finish().unwrap()); + #[cfg(not(target_arch = "wasm32"))] + assert_eq!(packed.capacity(), packed.len()); + } +} + +/// Workers retain only private codec state; a checked-out slot supports nested compression. +#[test] +#[cfg(not(target_arch = "wasm32"))] +fn zip_encoder_reuse_is_private_and_retains_no_output() { + std::thread::scope(|scope| { + let workers: Vec<_> = (0..2) + .map(|byte| { + scope.spawn(move || { + let mut seed = 0x12345678u32; + for len in [1_048_577usize, 32769, 0, 259, 258, 257, 1, 32767, 32768, 2, 262145] { + let bytes: Vec<_> = (0..len) + .map(|_| { + seed ^= seed << 13; + seed ^= seed >> 17; + seed ^= seed << 5; + if byte == 0 { 0 } else { seed as u8 } + }) + .collect(); + let packed = zip_compress(&bytes); + if byte == 1 && len == 1_048_577 { + assert!(packed.len() > 256 << 10, "the retention oracle must exercise multiple output blocks"); + } + let encoder = ZIP_ENCODER.with(std::cell::Cell::take).unwrap(); + assert_eq!(encoder.total_in(), len as u64); + assert_eq!(encoder.total_out(), packed.len() as u64); + assert_eq!(encoder.get_ref().blocks.capacity(), 0); + assert_eq!(encoder.get_ref().current.capacity(), 0); + assert_eq!(zip_decompress(&zip_compress(b"nested"), 6).unwrap(), b"nested"); + ZIP_ENCODER.with(|slot| slot.set(Some(encoder))); + let mut fresh = flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::default()); + fresh.write_all(&bytes).unwrap(); + assert_eq!(zip_compress(&bytes), fresh.finish().unwrap()); + } + }) + }) + .collect(); + for worker in workers { + worker.join().unwrap(); + } + }); +} + +/// Compression stays byte-identical during teardown, including an unavailable native cache. +#[test] +#[cfg(not(target_arch = "wasm32"))] +fn zip_compression_survives_thread_local_teardown() { + use std::sync::{ + Arc, + atomic::{AtomicBool, Ordering}, + }; + struct OnDrop(Arc<(AtomicBool, AtomicBool)>); + impl Drop for OnDrop { + /// Record whether teardown reached the destroyed cache and preserved fresh-stream bytes. + fn drop(&mut self) { + let unavailable = ZIP_ENCODER.try_with(|_| ()).is_err(); + let mut fresh = flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::default()); + fresh.write_all(b"teardown").unwrap(); + let same = zip_compress(b"teardown") == fresh.finish().unwrap(); + self.0.0.store(same, Ordering::SeqCst); + self.0.1.store(unavailable, Ordering::SeqCst); + } + } + thread_local! { static AFTER_CODEC: std::cell::Cell> = const { std::cell::Cell::new(None) }; } + let checked = Arc::new((AtomicBool::new(false), AtomicBool::new(false))); + let flag = checked.clone(); + std::thread::spawn(move || { + AFTER_CODEC.with(|slot| slot.set(Some(OnDrop(flag)))); + zip_compress(b"first"); + }) + .join() + .unwrap(); + assert!(checked.0.load(Ordering::SeqCst), "teardown must preserve the fresh encoder's bytes"); + // OS-key TLS may destroy keys in another order; native TLS runs these destructors in reverse. + #[cfg(any(target_vendor = "apple", target_os = "linux", target_env = "msvc"))] + assert!(checked.1.load(Ordering::SeqCst), "teardown must exercise the uncached fallback"); +} + +/// Verify multiple writes retain their bytes and respect ZIP output block bounds. +#[test] +#[cfg(not(target_arch = "wasm32"))] +fn zip_output_blocks_stay_bounded_across_writes() { + let bytes = vec![42; (1 << 20) + 1]; + let mut output = ZipOutput::default(); + output.write_all(&[]).unwrap(); + for part in bytes.chunks(32767) { + output.write_all(part).unwrap(); + } + assert!(output.blocks.iter().chain([&output.current]).all(|b| b.capacity() <= 256 << 10)); + let packed = output.into_vec(); + assert_eq!(packed, bytes); + assert_eq!(packed.capacity(), packed.len()); +} + +/// Verify compressible ZIP output retains capacity proportional to encoded size. +#[test] +fn compressible_zip_does_not_retain_a_raw_sized_allocation() { + let raw = vec![0; 1 << 20]; + let packed = zip_compress(&raw); + assert!(packed.len() < 1 << 14); + assert!(packed.capacity() < 1 << 16); + assert_eq!(zip_decompress(&packed, raw.len()).unwrap(), raw); +} + +/// Verify failed RLE allocation hints fall back to incremental growth with identical bytes. +#[test] +fn rle_failed_hint_and_incremental_growth_preserve_bytes() { + for version in [Version::Psd, Version::Psb] { + for width in [0, 1, 127, 128, 129, 1000] { + let l = layout(1, width, 9, 8, version); + for fill in [false, true] { + let bytes: Vec<_> = (0..l.decoded_len().unwrap()).map(|i| if fill { 0 } else { (i * 13 + i / 17) as u8 }).collect(); + let expected = encode_planes(Compression::Rle, &bytes, &l).unwrap(); + for hint in [usize::MAX, 0, l.rows().unwrap() * l.count_size()] { + assert_eq!(encode_rle_chunk(&bytes, &l, 0..l.rows().unwrap(), hint).unwrap(), expected); + } + } + } + } +} + +/// Verify measured packet lengths match encoding over mixed runs and literals. +#[test] +fn counted_packbits_matches_mixed_runs_and_literals() { + for n in 0..520 { + let bytes: Vec<_> = (0..n).map(|i| if i % 23 < 9 { (i / 23) as u8 } else { (i * 31) as u8 }).collect(); + assert_eq!(packbits::encoded_len(&bytes), packbits::encode_vec(&bytes).len()); + } +} + +/// Compare parallel prediction with independent serial rows at each supported depth. +#[test] +fn prediction_matches_independent_serial_rows() { + for depth in [1, 8, 16, 32] { + let layout = layout(4, if depth == 1 { 8193 } else { 257 }, 1031, depth, Version::Psb); + let rb = layout.row_bytes(); + let bytes: Vec<_> = (0..layout.decoded_len().unwrap() + 3).map(|i| (i * 13 + i / 17) as u8).collect(); + let mut expected = bytes.clone(); + for row in expected.chunks_exact_mut(rb) { + let src = row.to_vec(); + if depth == 16 { + for (i, p) in row.as_chunks_mut::<2>().0.iter_mut().enumerate().skip(1) { + let a = u16::from_be_bytes([src[i * 2], src[i * 2 + 1]]); + let b = u16::from_be_bytes([src[i * 2 - 2], src[i * 2 - 1]]); + *p = a.wrapping_sub(b).to_be_bytes(); + } + } else { + let shuffled = if depth == 32 { (0..4).flat_map(|c| src.iter().skip(c).step_by(4).copied()).collect::>() } else { src }; + for (i, p) in row.iter_mut().enumerate() { + *p = shuffled[i].wrapping_sub(if i == 0 { 0 } else { shuffled[i - 1] }); + } + } + } + #[cfg(not(target_arch = "wasm32"))] + for threads in [1, 2, 4] { + use rayon::prelude::*; + let pool = rayon::ThreadPoolBuilder::new().num_threads(threads).build().unwrap(); + pool.install(|| { + (0..3).into_par_iter().for_each(|_| { + let mut got = bytes.clone(); + predict(&mut got, &layout).unwrap(); + assert_eq!(got, expected, "depth {depth}, pool {threads}"); + }) + }); + } + let mut got = bytes; + predict(&mut got, &layout).unwrap(); + assert_eq!(got, expected); + } +} + +/// Verify RLE chunks use the caller's pool and propagate worker panics as errors. +#[cfg(not(target_arch = "wasm32"))] +#[test] +fn rle_chunks_use_the_callers_pool_and_report_panics() { + let pool = rayon::ThreadPoolBuilder::new().num_threads(2).build().unwrap(); + pool.install(|| { + map_chunks(8, (0..64).collect(), |_| assert!(matches!(rayon::current_thread_index(), Some(0 | 1)))).unwrap(); + assert!(map_chunks(2, vec![0, 1], |_| panic!("worker")).is_err()); + }); +} diff --git a/crates/psd/src/filter_effects.rs b/crates/psd/src/filter_effects.rs index b3d545d72..abda0c503 100644 --- a/crates/psd/src/filter_effects.rs +++ b/crates/psd/src/filter_effects.rs @@ -189,6 +189,52 @@ impl FilterEffects { let mut out = Vec::new(); out.put_u32(self.version); for it in &self.items { + let section = out.begin_len(true); + write_pascal(&mut out, it.id.as_bytes(), 1); + out.put_u32(it.version); + let at = out.begin_len(true); + write_rect(&mut out, &it.rect); + out.put_u32(it.depth); + out.put_u32(it.max_channels); + for i in 0..it.max_channels as usize + 2 { + match it.slots.get(i).and_then(Option::as_ref) { + Some(p) => { + out.put_u32(1); + write_plane(&mut out, p); + } + None => out.put_u32(0), + } + } + // A 64-bit length never overflows. + let _ = out.end_len(at, true); + match &it.mask { + Some((r, p)) => { + out.put_u8(1); + write_rect(&mut out, r); + write_plane(&mut out, p); + } + None => out.put_u8(0), + } + let _ = out.end_len(section, true); + out.resize(out.len().next_multiple_of(4), 0); + } + if self.items.is_empty() { + // One empty section, so the block still parses. + out.put_u64(0); + } + out + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Serialize filter-effect sections through staging buffers as a byte oracle. + fn staged_bytes(fx: &FilterEffects) -> Vec { + let mut out = Vec::new(); + out.put_u32(fx.version); + for it in &fx.items { let mut body = Vec::new(); write_pascal(&mut body, it.id.as_bytes(), 1); body.put_u32(it.version); @@ -219,17 +265,12 @@ impl FilterEffects { out.put(&body); out.resize(out.len().next_multiple_of(4), 0); } - if self.items.is_empty() { + if fx.items.is_empty() { // One empty section, so the block still parses. out.put_u64(0); } out } -} - -#[cfg(test)] -mod tests { - use super::*; fn item(id: &str, mask: bool) -> FilterEffectsItem { let rect = Rect { top: 1, left: 2, bottom: 5, right: 6 }; @@ -240,6 +281,20 @@ mod tests { FilterEffectsItem { id: id.into(), version: 1, rect, depth: 8, max_channels: 24, slots, mask: mask.then(|| (rect, plane(128))) } } + /// Compare direct filter-effect serialization with staged bytes and section lengths. + #[test] + fn direct_sections_match_staged_bytes() { + for version in [1, 2, 3] { + let mut large = item(&"é".repeat(140), true); + large.rect.top = -7; + large.slots[0].as_mut().unwrap().data = vec![73; 8193]; + for items in [Vec::new(), vec![large, item("odd", false), item("four", true)]] { + let fx = FilterEffects { version, items }; + assert_eq!(fx.to_bytes(), staged_bytes(&fx)); + } + } + } + /// The unpadded body of `it`'s section. fn section(it: &FilterEffectsItem) -> Vec { let b = FilterEffects { version: 3, items: vec![it.clone()] }.to_bytes(); diff --git a/crates/psd/src/patterns.rs b/crates/psd/src/patterns.rs index f7fd80a51..23ca8cf3d 100644 --- a/crates/psd/src/patterns.rs +++ b/crates/psd/src/patterns.rs @@ -25,6 +25,8 @@ use crate::compression::{Compression, MAX_DECODED_BYTES, PlaneLayout, decode_planes, encode_planes}; use crate::error::{PsdError, Result}; use crate::header::Version; +use crate::io::WriteExt; +use std::borrow::Cow; /// Largest pattern edge accepted (Photoshop's own limit is far lower). const MAX_EDGE: u32 = 30_000; @@ -417,7 +419,11 @@ fn zeroed_pattern_plane(len: usize) -> Result> { Ok(out) } +/// Append one validated pattern, backpatching lengths in its final output buffer. fn write_pattern(p: &PsdPattern, out: &mut Vec) -> Result<()> { + if !matches!(p.depth, 1 | 8 | 16 | 32) { + return Err(PsdError::Unsupported(format!("pattern depth {}", p.depth))); + } if p.width > MAX_EDGE || p.height > MAX_EDGE || p.width > i16::MAX as u32 || p.height > i16::MAX as u32 { return Err(PsdError::LimitExceeded("pattern size")); } @@ -444,46 +450,50 @@ fn write_pattern(p: &PsdPattern, out: &mut Vec) -> Result<()> { out.extend_from_slice(&pal); } let rect = [0i32, 0, p.height as i32, p.width as i32]; - let mut body = Vec::new(); + out.extend_from_slice(&3u32.to_be_bytes()); + let at = out.begin_len(false); for v in rect { - body.extend_from_slice(&v.to_be_bytes()); + out.extend_from_slice(&v.to_be_bytes()); } const SLOTS: u32 = 24; - body.extend_from_slice(&SLOTS.to_be_bytes()); + out.extend_from_slice(&SLOTS.to_be_bytes()); let layout = PlaneLayout { planes: 1, width: p.width as usize, height: p.height as usize, depth: p.depth, version: Version::Psd }; - let array = |plane: &[u8], body: &mut Vec| -> Result<()> { + let array = |plane: &[u8], out: &mut Vec| -> Result<()> { let (comp, data) = if p.depth == 8 { - (1u8, encode_planes(Compression::Rle, plane, &layout)?) + (1u8, Cow::Owned(encode_planes(Compression::Rle, plane, &layout)?)) } else { - (0u8, encode_planes(Compression::Raw, plane, &layout)?) + let total = layout.decoded_len()?; + if plane.len() != total { + return Err(PsdError::invalid(format!("decoded buffer has {} bytes, layout requires {}", plane.len(), total))); + } + (0u8, Cow::Borrowed(plane)) }; - body.extend_from_slice(&1u32.to_be_bytes()); - body.extend_from_slice(&((23 + data.len()) as u32).to_be_bytes()); - body.extend_from_slice(&u32::from(p.depth).to_be_bytes()); + out.extend_from_slice(&1u32.to_be_bytes()); + let len = u32::try_from(data.len()).ok().and_then(|n| n.checked_add(23)).ok_or(PsdError::LimitExceeded("pattern array length"))?; + out.extend_from_slice(&len.to_be_bytes()); + out.extend_from_slice(&u32::from(p.depth).to_be_bytes()); for v in rect { - body.extend_from_slice(&v.to_be_bytes()); + out.extend_from_slice(&v.to_be_bytes()); } - body.extend_from_slice(&p.depth.to_be_bytes()); - body.push(comp); - body.extend_from_slice(&data); + out.extend_from_slice(&p.depth.to_be_bytes()); + out.push(comp); + out.extend_from_slice(&data); Ok(()) }; let nc = p.channels.len().min(SLOTS as usize); for c in &p.channels[..nc] { - array(c, &mut body)?; + array(c, out)?; } for _ in nc..SLOTS as usize { - body.extend_from_slice(&0u32.to_be_bytes()); + out.extend_from_slice(&0u32.to_be_bytes()); } // User mask slot: unused; sheet mask slot: transparency, where Photoshop keeps it (photocraft#8054). - body.extend_from_slice(&0u32.to_be_bytes()); + out.extend_from_slice(&0u32.to_be_bytes()); match &p.alpha { - Some(a) => array(a, &mut body)?, - None => body.extend_from_slice(&0u32.to_be_bytes()), + Some(a) => array(a, out)?, + None => out.extend_from_slice(&0u32.to_be_bytes()), } - out.extend_from_slice(&3u32.to_be_bytes()); - out.extend_from_slice(&(body.len() as u32).to_be_bytes()); - out.extend_from_slice(&body); + out.end_len(at, false)?; Ok(()) } @@ -578,11 +588,10 @@ fn pattern_has_user_mask(r: &mut Rd) -> Result { pub fn write_pattern_block(patterns: &[PsdPattern]) -> Result> { let mut out = Vec::new(); for p in patterns { - let mut one = Vec::new(); - write_pattern(p, &mut one)?; - out.extend_from_slice(&(one.len() as u32).to_be_bytes()); - let pad = (4 - one.len() % 4) % 4; - out.extend_from_slice(&one); + let at = out.begin_len(false); + write_pattern(p, &mut out)?; + out.end_len(at, false)?; + let pad = (4 - (out.len() - at - 4) % 4) % 4; out.extend(std::iter::repeat_n(0u8, pad)); } Ok(out) diff --git a/crates/psd/src/patterns/tests.rs b/crates/psd/src/patterns/tests.rs index d1e27692f..ae591bc49 100644 --- a/crates/psd/src/patterns/tests.rs +++ b/crates/psd/src/patterns/tests.rs @@ -17,6 +17,56 @@ fn pat(depth: u16, alpha: bool) -> PsdPattern { } } +/// Compare direct pattern serialization with staged bytes at supported depths. +#[test] +fn direct_writes_match_staged_bytes() { + for depth in [1, 8, 16, 32] { + for alpha in [false, true] { + let mut p = pat(depth, alpha); + if depth == 1 { + p.channels = vec![vec![0xA8; 3]; 3]; + p.alpha = alpha.then(|| vec![0xF8; 3]); + } + p.id = "é".repeat(140); + let mut indexed = p.clone(); + indexed.mode = 2; + indexed.channels.truncate(1); + indexed.palette = Some(vec![73; 17]); + let patterns = [p, indexed]; + let mut block = Vec::new(); + let mut file = b"8BPT\0\x01\0\0\0\x02".to_vec(); + for p in &patterns { + let mut one = Vec::new(); + staged_pattern(p, &mut one).unwrap(); + block.extend_from_slice(&(one.len() as u32).to_be_bytes()); + block.extend_from_slice(&one); + block.resize(block.len().next_multiple_of(4), 0); + file.extend_from_slice(&one); + } + assert_eq!(write_pattern_block(&patterns).unwrap(), block); + assert_eq!(write_pat_file(&patterns).unwrap(), file); + } + } +} + +/// Verify borrowed raw pattern planes retain layout and length validation. +#[test] +fn raw_pattern_writes_still_validate_layout_and_plane_length() { + for depth in [1, 16, 32, 7] { + let p = pat(depth, false); + assert!(write_pattern_block(&[PsdPattern { channels: vec![vec![0; 2]], ..p.clone() }]).is_err()); + assert!(write_pat_file(&[PsdPattern { channels: Vec::new(), alpha: Some(vec![0; 2]), ..p }]).is_err()); + } +} + +/// Verify matching plane lengths cannot bypass unsupported pattern-depth checks. +#[test] +fn writers_reject_unsupported_depth_with_matching_plane_lengths() { + let p = PsdPattern { depth: 7, channels: vec![vec![0; 15]; 3], alpha: Some(vec![0; 15]), ..pat(8, false) }; + assert!(write_pattern_block(std::slice::from_ref(&p)).is_err()); + assert!(write_pat_file(&[p]).is_err()); +} + /// photocraft#9318 review: a name the reader refuses (more than 65,536 UTF-16 units with its NUL) is /// refused on write, so a saved library can't become a file that loads nothing. #[test] @@ -350,3 +400,74 @@ fn zero_size_pattern_rectangles_are_rejected() { set_i32(&mut block, vma + 8, 0); assert!(parse_pattern_block(&block).is_err()); } + +/// Serialize a pattern with the previous staged writer as a byte oracle. +fn staged_pattern(p: &PsdPattern, out: &mut Vec) -> Result<()> { + if p.width > MAX_EDGE || p.height > MAX_EDGE || p.width > i16::MAX as u32 || p.height > i16::MAX as u32 { + return Err(PsdError::LimitExceeded("pattern size")); + } + out.extend_from_slice(&1u32.to_be_bytes()); + out.extend_from_slice(&p.mode.to_be_bytes()); + out.extend_from_slice(&(p.height as u16).to_be_bytes()); + out.extend_from_slice(&(p.width as u16).to_be_bytes()); + let units: Vec = p.name.encode_utf16().chain(std::iter::once(0)).collect(); + // The reader refuses longer names (see `read_pattern`). + if units.len() > 65_536 { + return Err(PsdError::LimitExceeded("pattern name length")); + } + out.extend_from_slice(&(units.len() as u32).to_be_bytes()); + for u in units { + out.extend_from_slice(&u.to_be_bytes()); + } + let id = p.id.as_bytes(); + let idl = id.len().min(255); + out.push(idl as u8); + out.extend_from_slice(&id[..idl]); + if p.mode == 2 { + let mut pal = p.palette.clone().unwrap_or_default(); + pal.resize(768, 0); + out.extend_from_slice(&pal); + } + let rect = [0i32, 0, p.height as i32, p.width as i32]; + let mut body = Vec::new(); + for v in rect { + body.extend_from_slice(&v.to_be_bytes()); + } + const SLOTS: u32 = 24; + body.extend_from_slice(&SLOTS.to_be_bytes()); + let layout = PlaneLayout { planes: 1, width: p.width as usize, height: p.height as usize, depth: p.depth, version: Version::Psd }; + let array = |plane: &[u8], body: &mut Vec| -> Result<()> { + let (comp, data) = if p.depth == 8 { + (1u8, encode_planes(Compression::Rle, plane, &layout)?) + } else { + (0u8, encode_planes(Compression::Raw, plane, &layout)?) + }; + body.extend_from_slice(&1u32.to_be_bytes()); + body.extend_from_slice(&((23 + data.len()) as u32).to_be_bytes()); + body.extend_from_slice(&u32::from(p.depth).to_be_bytes()); + for v in rect { + body.extend_from_slice(&v.to_be_bytes()); + } + body.extend_from_slice(&p.depth.to_be_bytes()); + body.push(comp); + body.extend_from_slice(&data); + Ok(()) + }; + let nc = p.channels.len().min(SLOTS as usize); + for c in &p.channels[..nc] { + array(c, &mut body)?; + } + for _ in nc..SLOTS as usize { + body.extend_from_slice(&0u32.to_be_bytes()); + } + // User mask slot: unused; sheet mask slot: transparency, where Photoshop keeps it (photocraft#8054). + body.extend_from_slice(&0u32.to_be_bytes()); + match &p.alpha { + Some(a) => array(a, &mut body)?, + None => body.extend_from_slice(&0u32.to_be_bytes()), + } + out.extend_from_slice(&3u32.to_be_bytes()); + out.extend_from_slice(&(body.len() as u32).to_be_bytes()); + out.extend_from_slice(&body); + Ok(()) +} From 5538d8a0d7e51b617193f4e75bc0da58ce07860b Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:04:26 -0500 Subject: [PATCH 03/10] Add craft-pptx: the PresentationML model (from DeckCraft) (#1044) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add craft-drawingml: units, colours, preset shapes, custom geometry and text warps The parts of DrawingML that need no document model, moved from DeckCraft (60f91f78): crates/color, crates/geom (preset, warp, Xfrm and the EMU conversions) and crates/pptx/src/custgeom.rs with CustomPath from crates/model. The code is unchanged except that custgeom reads craft-xml trees and writes to a craft-xml Writer instead of DeckCraft's own XML helper, and matches elements by local name. The 59 tests of the moved code come with it; six more cover the custom geometry limits. * drawingml: evaluate a custom geometry's guide lists once per coordinate space read_cust_geom rebuilt Guides and evaluated avLst and gdLst again for every path: up to 1,000 paths times up to 10,000 guides, each name copied and hashed. Guides are now cached per path coordinate space (w, h). At most 16 spaces and 4 MiB of guide names are kept per geometry; paths in further spaces are left out. The first space is always kept, and a geometry without guides is not capped. Ported from storytold/deckcraft#1789. A list counts as having guides only when it has gd children, as that PR's review asked. * drawingml: the model types for fills, lines, text, tables, charts and diagrams Moved from DeckCraft (60f91f78) crates/model/src: style.rs, text.rs, table.rs, chart.rs, numfmt.rs and trendline.rs, plus Diagram, DiagramNode and the MediaId and SlideId newtypes from lib.rs. The files are unchanged except that deckcraft_color:: is crate::color:: and Fill and TableBorder allow clippy::large_enum_variant. Their 60 tests come with them. Readers and writers follow. * drawingml: the theme model (colour, font and format schemes) Moved from DeckCraft crates/model/src/theme.rs at 60f91f78: FontSet, ScriptFont, FontScheme, FormatScheme, Theme and default_format. The app's theme gallery, its variants and its own default theme stay in the app. Theme::default() is a plain theme built in place, so nothing here depends on a gallery. * drawingml: readers and writers for fills, lines, text, charts, themes, table styles and diagrams Moved from DeckCraft crates/pptx/src at 60f91f78: read/dml.rs, write/dml.rs, read/chart.rs, read/chartex.rs, write/chart.rs, read/theme.rs, write/theme.rs, tables.rs and dgm.rs, onto craft-xml. Package lookups (Imp, Part, Exp, Out) are the caller's part::Source and part::Sink. The chart frame budget and kept-part capture stay with the presentation reader. * drawingml: what WordCraft and GridCraft read that DeckCraft's copy did not Every ST_PresetColorVal name has its colour, in any case and with the dk/lt/med abbreviations; a sysClr without lastClr takes the usual value of its name; a Strict percentage (50%) reads anywhere a percentage does; a chart series keeps its c:f formulas and the writer uses them. * craft-drawingml: extra named theme colours and explicit tint rows Ported from the format part of storytold/deckcraft#1746 by @jagjordi: ColorScheme::{extra, tints, computed_tints}, ExtraColor, GalleryVariant, variants_of, and the a:clrScheme extension that keeps them in a file. * drawingml xmlx: copy only used, bounded namespace declarations onto kept fragments and mc branches From storytold/deckcraft#1740 (988a4920), which landed after the extraction commit 60f91f78. * Add craft-pptx: the PresentationML model (from DeckCraft) Presentation, masters, layouts, slides, shapes, placeholders, media, comments, sections, show settings, properties, embedded fonts, kept parts and drawing guides from DeckCraft's crates/model/src/lib.rs, and transitions and animations from anim.rs. The DrawingML types come from craft-drawingml. The placeholder matching from DeckCraft's resolve.rs moves as placeholder::parents, without the rest of resolve. Presentation::default() is now an empty 16:9 deck with no masters; the starting deck stays with the application. --------- Co-authored-by: Jordi Altayó --- Cargo.lock | 10 + Cargo.toml | 1 + README.md | 3 + crates/pptx/Cargo.toml | 17 + crates/pptx/README.md | 21 + crates/pptx/src/anim.rs | 668 ++++++++++++++++++++ crates/pptx/src/lib.rs | 1079 ++++++++++++++++++++++++++++++++ crates/pptx/src/placeholder.rs | 106 ++++ crates/pptx/src/tests.rs | 131 ++++ 9 files changed, 2036 insertions(+) create mode 100644 crates/pptx/Cargo.toml create mode 100644 crates/pptx/README.md create mode 100644 crates/pptx/src/anim.rs create mode 100644 crates/pptx/src/lib.rs create mode 100644 crates/pptx/src/placeholder.rs create mode 100644 crates/pptx/src/tests.rs diff --git a/Cargo.lock b/Cargo.lock index b6b4afa92..3924f3435 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1657,6 +1657,16 @@ dependencies = [ "thiserror 2.0.21", ] +[[package]] +name = "craft-pptx" +version = "0.1.0" +dependencies = [ + "craft-drawingml", + "kurbo 0.13.1", + "serde", + "serde_json", +] + [[package]] name = "craft-prc" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index 30072815d..c9c524fe5 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -87,6 +87,7 @@ craft-metafile = { path = "crates/metafile" } craft-offcrypto = { path = "crates/offcrypto" } craft-opc = { path = "crates/opc" } craft-ora = { path = "crates/ora" } +craft-pptx = { path = "crates/pptx" } craft-penpot = { path = "crates/penpot" } craft-pcraft-format = { path = "crates/pcraft-format" } craft-plotter = { path = "crates/plotter" } diff --git a/README.md b/README.md index 6c6f169f5..5df977463 100644 --- a/README.md +++ b/README.md @@ -262,6 +262,9 @@ craft- = { git = "https://github.com/storytold/craft-libs", rev = ", + /// Embedded sound media. + pub sound: Option, + /// Loop Until Next Sound (`p:stSnd loop="1"`). + #[serde(skip_serializing_if = "std::ops::Not::not")] + pub sound_loop: bool, + /// [Stop Previous Sound] (`p:endSnd`): the slide stops a sound still playing. + #[serde(skip_serializing_if = "std::ops::Not::not")] + pub stop_sound: bool, + /// Unmodelled transition XML kept for round-trip. + #[serde(skip_serializing_if = "Option::is_none")] + pub raw: Option, +} + +impl Default for Transition { + fn default() -> Self { + Transition { + kind: "none".into(), + option: String::new(), + duration_ms: 1000, + advance_on_click: true, + advance_after_ms: None, + sound: None, + sound_loop: false, + stop_sound: false, + raw: None, + } + } +} + +/// (id, label, category, default duration ms, effect options) +pub static TRANSITIONS: &[(&str, &str, &str, u32, &[&str])] = &[ + ("none", "None", "Subtle", 0, &[]), + ("morph", "Morph", "Subtle", 2000, &["objects", "words", "characters"]), + ("fade", "Fade", "Subtle", 700, &["smoothly", "throughBlack"]), + ("push", "Push", "Subtle", 1000, &["u", "d", "l", "r"]), + ("wipe", "Wipe", "Subtle", 1000, &["r", "l", "u", "d", "ru", "lu", "rd", "ld"]), + ("split", "Split", "Subtle", 1500, &["vertOut", "vertIn", "horzOut", "horzIn"]), + ("reveal", "Reveal", "Subtle", 2000, &["smoothlyLeft", "smoothlyRight", "blackLeft", "blackRight"]), + ("cut", "Cut", "Subtle", 0, &["", "throughBlack"]), + ("randomBar", "Random Bars", "Subtle", 1000, &["vert", "horz"]), + ("shape", "Shape", "Subtle", 2000, &["circle", "diamond", "plus", "in", "out"]), + ("uncover", "Uncover", "Subtle", 1000, &["l", "r", "u", "d", "lu", "ru", "ld", "rd"]), + ("cover", "Cover", "Subtle", 1000, &["l", "r", "u", "d", "lu", "ru", "ld", "rd"]), + ("flash", "Flash", "Subtle", 1000, &[]), + ("fallOver", "Fall Over", "Exciting", 2000, &["l", "r"]), + ("drape", "Drape", "Exciting", 2000, &["l", "r"]), + ("curtains", "Curtains", "Exciting", 2000, &[]), + ("wind", "Wind", "Exciting", 2000, &["l", "r"]), + ("prestige", "Prestige", "Exciting", 2000, &[]), + ("fracture", "Fracture", "Exciting", 2000, &[]), + ("crush", "Crush", "Exciting", 2000, &[]), + ("peelOff", "Peel Off", "Exciting", 1500, &["l", "r"]), + ("pageCurlDouble", "Page Curl", "Exciting", 1500, &["l", "r", "singleL", "singleR"]), + ("airplane", "Airplane", "Exciting", 2000, &["l", "r"]), + ("origami", "Origami", "Exciting", 2000, &["l", "r"]), + ("dissolve", "Dissolve", "Exciting", 1200, &[]), + ("checker", "Checkerboard", "Exciting", 1500, &["horz", "vert"]), + ("blinds", "Blinds", "Exciting", 1600, &["vert", "horz"]), + ("clock", "Clock", "Exciting", 2000, &["clockwise", "counterClockwise", "wedge"]), + ("ripple", "Ripple", "Exciting", 1400, &["center", "lu", "ru", "ld", "rd"]), + ("honeycomb", "Honeycomb", "Exciting", 3000, &[]), + ("glitter", "Glitter", "Exciting", 2500, &["l", "r", "u", "d", "hexL", "hexR", "hexU", "hexD"]), + ("vortex", "Vortex", "Exciting", 3000, &["l", "r", "u", "d"]), + ("shred", "Shred", "Exciting", 2000, &["stripsIn", "stripsOut", "particlesIn", "particlesOut"]), + ("switch", "Switch", "Exciting", 1250, &["l", "r"]), + ("flip", "Flip", "Exciting", 1250, &["l", "r"]), + ("gallery", "Gallery", "Exciting", 1500, &["l", "r"]), + ("cube", "Cube", "Exciting", 1250, &["l", "r", "u", "d"]), + ("doors", "Doors", "Exciting", 1400, &["vert", "horz"]), + ("box", "Box", "Exciting", 1250, &["l", "r", "u", "d"]), + ("comb", "Comb", "Exciting", 1000, &["horz", "vert"]), + ("zoom", "Zoom", "Exciting", 1000, &["in", "out", "rotate"]), + ("random", "Random", "Exciting", 2000, &[]), + ("pan", "Pan", "Dynamic Content", 1000, &["u", "d", "l", "r"]), + ("ferris", "Ferris Wheel", "Dynamic Content", 2000, &["l", "r"]), + ("conveyor", "Conveyor", "Dynamic Content", 2000, &["l", "r"]), + ("rotate", "Rotate", "Dynamic Content", 2000, &["l", "r", "u", "d"]), + ("window", "Window", "Dynamic Content", 1250, &["vert", "horz"]), + ("orbit", "Orbit", "Dynamic Content", 2000, &["l", "r", "u", "d"]), + ("flythrough", "Fly Through", "Dynamic Content", 1000, &["in", "out", "inBounce", "outBounce"]), +]; + +/// The Effect Options label PowerPoint shows for a transition option. Options are stored as the +/// file format's values: for moving transitions (Push, Wipe, Cover, Pan, Cube…) the direction +/// things move, so "l" is the effect coming "From Right"; Glitter and the preset effects (Fall +/// Over, Wind…) name the side itself; Ripple names where the wave heads. +pub fn transition_option_label(kind: &str, option: &str) -> String { + let side = |c: char| match c { + 'l' => "Left", + 'r' => "Right", + 'u' => "Top", + _ => "Bottom", + }; + let opposite = |c: char| match c { + 'l' => 'r', + 'r' => 'l', + 'u' => 'd', + _ => 'u', + }; + let corner = |o: &str, flip: bool| -> Option { + let mut cs = o.chars(); + let (h, v) = (cs.next()?, cs.next()?); + let (h, v) = if flip { (opposite(h), opposite(v)) } else { (h, v) }; + Some(format!("From {}-{}", side(v), side(h))) + }; + let moving = matches!( + kind, + "push" + | "wipe" + | "cover" + | "uncover" + | "pan" + | "vortex" + | "cube" + | "box" + | "rotate" + | "orbit" + | "gallery" + | "conveyor" + | "ferris" + | "switch" + | "flip" + ); + match (kind, option) { + (_, "horz") => "Horizontal".into(), + (_, "vert") => "Vertical".into(), + ("split", o) => format!("{} {}", if o.starts_with("horz") { "Horizontal" } else { "Vertical" }, if o.ends_with("In") { "In" } else { "Out" }), + ("fade" | "cut", "throughBlack") => "Through Black".into(), + ("fade", _) => "Smoothly".into(), + ("cut", _) => "Cut".into(), + ("clock", "counterClockwise") => "Counterclockwise".into(), + ("clock", "wedge") => "Wedge".into(), + ("clock", _) => "Clockwise".into(), + ("zoom", "rotate") => "Zoom and Rotate".into(), + ("ripple", "center") => "Center".into(), + ("ripple", o) => corner(o, true).unwrap_or_else(|| "Center".into()), + ("glitter", o) => { + let (pat, d) = match o.strip_prefix("hex") { + Some(d) => ("Hexagons", d.to_ascii_lowercase()), + None => ("Diamonds", o.to_string()), + }; + format!("{pat} From {}", side(d.chars().next().unwrap_or('l'))) + } + ("pageCurlDouble", o) => match o { + "singleL" => "Single Left".into(), + "singleR" => "Single Right".into(), + "r" => "Double Right".into(), + _ => "Double Left".into(), + }, + ("shred", o) => { + format!("{} {}", if o.starts_with("particles") { "Particles" } else { "Strips" }, if o.ends_with("Out") { "Out" } else { "In" }) + } + ("reveal", o) => format!( + "{} {}", + if o.starts_with("black") { "Through Black" } else { "Smoothly" }, + if o.ends_with("Right") { "From Right" } else { "From Left" } + ), + ("flythrough", o) => { + format!("{}{}", if o.starts_with("out") { "Out" } else { "In" }, if o.ends_with("Bounce") { " with Bounce" } else { "" }) + } + ("morph", o) => match o { + "words" => "Words".into(), + "characters" => "Characters".into(), + _ => "Objects".into(), + }, + (_, o) if o.len() == 2 && moving => corner(o, true).unwrap_or_default(), + (_, o) if o.len() == 1 && moving => format!("From {}", side(opposite(o.chars().next().unwrap_or('l')))), + (_, "l") => "Left".into(), + (_, "r") => "Right".into(), + (_, "u") => "Top".into(), + (_, "d") => "Bottom".into(), + (_, "in") => "In".into(), + (_, "out") => "Out".into(), + (_, o) => o.to_string(), + } +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum AnimClass { + #[default] + Entrance, + Emphasis, + Exit, + Path, + Media, +} + +impl AnimClass { + pub fn xml(self) -> &'static str { + match self { + AnimClass::Entrance => "entr", + AnimClass::Emphasis => "emph", + AnimClass::Exit => "exit", + AnimClass::Path => "path", + AnimClass::Media => "mediacall", + } + } + pub fn from_xml(s: &str) -> Self { + match s { + "emph" => AnimClass::Emphasis, + "exit" => AnimClass::Exit, + "path" => AnimClass::Path, + "mediacall" => AnimClass::Media, + _ => AnimClass::Entrance, + } + } +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum AnimStart { + #[default] + OnClick, + WithPrevious, + AfterPrevious, +} + +/// Text build: as one object, all at once by paragraph, or by paragraph level. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum TextBuild { + #[default] + AsOne, + ByParagraph, + ByWord, + ByLetter, +} + +/// Most paragraphs a by-paragraph build animates separately; later ones follow the last of them. +pub const MAX_BUILD_PARAGRAPHS: usize = 1_000; + +/// What a paragraph does in a by-paragraph build. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum BuildRole { + /// Animated on its own: a filled paragraph at the shallowest filled level. + Top, + /// Follows the top paragraph with this index (deeper, empty or blank paragraphs after it). + Member(usize), + /// Before the first top paragraph: not animated. + Unowned, +} + +/// Does the paragraph show any text? A by-paragraph build doesn't animate blank ones on their own. +pub fn build_filled(p: &text::Paragraph) -> bool { + p.runs.iter().any(|r| !r.text.trim().is_empty()) +} + +/// The role of every paragraph of `text` in a by-paragraph build, by paragraph index: what the +/// timeline plays, and what the PPTX reader and writer map to one file effect per paragraph. +pub fn paragraph_build_roles(text: &text::TextBody) -> Vec { + let top = text.paragraphs.iter().filter(|p| build_filled(p)).map(|p| p.level).min().unwrap_or(0); + let mut tops = 0; + let mut owner: Option = None; + text.paragraphs + .iter() + .enumerate() + .map(|(i, p)| { + if p.level <= top && build_filled(p) && tops < MAX_BUILD_PARAGRAPHS { + tops += 1; + owner = Some(i); + BuildRole::Top + } else { + owner.map_or(BuildRole::Unowned, BuildRole::Member) + } + }) + .collect() +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Animation { + pub shape: ShapeId, + pub class: AnimClass, + /// Effect id from [`ANIMATIONS`] (e.g. `fade`, `fly`, `zoom`, `spin`, `pulse`, `lines`…). + pub effect: String, + /// Effect option (direction etc.). + pub option: String, + pub start: AnimStart, + pub duration_ms: u32, + pub delay_ms: u32, + /// Repeat count (0/1 = once); `u32::MAX` = until next click. + pub repeat: u32, + pub rewind: bool, + pub text_build: TextBuild, + /// Paragraph index when the effect animates one paragraph of a text body. + pub paragraph: Option, + /// Motion path in slide-relative units (SVG-like `M 0 0 L 0.25 0 E`). + pub path: Option, + /// Triggered by a click on another shape. + pub trigger: Option, + /// Emphasis amount: spin degrees, grow/shrink scale (1.5 = 150%), transparency… + pub amount: Option, + /// Emphasis colour (fill/line/font colour change). + pub color: Option, + /// PowerPoint preset id/subtype read from or written to files. + pub preset_id: Option, + pub preset_subtype: Option, + /// After animation: dim to colour, hide, hide on next click. + pub after: Option, + pub sound: Option, + pub smooth_start: f64, + pub smooth_end: f64, + pub bounce_end: f64, + pub auto_reverse: bool, +} + +impl Default for Animation { + fn default() -> Self { + Animation { + shape: ShapeId(0), + class: AnimClass::Entrance, + effect: "fade".into(), + option: String::new(), + start: AnimStart::OnClick, + duration_ms: 500, + delay_ms: 0, + repeat: 1, + rewind: false, + text_build: TextBuild::AsOne, + paragraph: None, + path: None, + trigger: None, + amount: None, + color: None, + preset_id: None, + preset_subtype: None, + after: None, + sound: None, + smooth_start: 0.0, + smooth_end: 0.0, + bounce_end: 0.0, + auto_reverse: false, + } + } +} + +/// One [`ANIMATIONS`] row: (id, label, class, PowerPoint preset id, default duration ms, options). +pub type AnimationInfo = (&'static str, &'static str, AnimClass, u32, u32, &'static [&'static str]); + +/// (id, label, class, PowerPoint preset id, default duration ms, options) +pub static ANIMATIONS: &[AnimationInfo] = &[ + ("appear", "Appear", AnimClass::Entrance, 1, 0, &[]), + ("fade", "Fade", AnimClass::Entrance, 10, 500, &[]), + ("fly", "Fly In", AnimClass::Entrance, 2, 500, &["b", "lb", "l", "lt", "t", "rt", "r", "rb"]), + ("float", "Float In", AnimClass::Entrance, 42, 1000, &["u", "d"]), + ("split", "Split", AnimClass::Entrance, 16, 500, &["horzIn", "horzOut", "vertIn", "vertOut"]), + ("wipe", "Wipe", AnimClass::Entrance, 22, 500, &["b", "l", "t", "r"]), + ("shape", "Shape", AnimClass::Entrance, 6, 2000, &["circleIn", "circleOut", "boxIn", "boxOut", "diamondIn", "diamondOut", "plusIn", "plusOut"]), + ("wheel", "Wheel", AnimClass::Entrance, 21, 2000, &["1", "2", "3", "4", "8"]), + ("randomBars", "Random Bars", AnimClass::Entrance, 14, 500, &["horz", "vert"]), + ("grow", "Grow & Turn", AnimClass::Entrance, 31, 1000, &[]), + ("zoom", "Zoom", AnimClass::Entrance, 53, 500, &["in", "out"]), + ("swivel", "Swivel", AnimClass::Entrance, 45, 2000, &["horz", "vert"]), + ("bounce", "Bounce", AnimClass::Entrance, 26, 2000, &[]), + ("blinds", "Blinds", AnimClass::Entrance, 3, 500, &["horz", "vert"]), + ("box", "Box", AnimClass::Entrance, 4, 2000, &["in", "out"]), + ("checkerboard", "Checkerboard", AnimClass::Entrance, 5, 500, &["across", "down"]), + ("circle", "Circle", AnimClass::Entrance, 6, 2000, &["in", "out"]), + ("diamond", "Diamond", AnimClass::Entrance, 8, 2000, &["in", "out"]), + ("dissolve", "Dissolve In", AnimClass::Entrance, 9, 500, &[]), + ("peek", "Peek In", AnimClass::Entrance, 12, 500, &["b", "l", "t", "r"]), + ("strips", "Strips", AnimClass::Entrance, 18, 500, &["lu", "ru", "ld", "rd"]), + ("expand", "Expand", AnimClass::Entrance, 55, 1000, &[]), + ("rise", "Rise Up", AnimClass::Entrance, 37, 1000, &[]), + ("spinner", "Spinner", AnimClass::Entrance, 49, 500, &[]), + ("pulse", "Pulse", AnimClass::Emphasis, 26, 500, &[]), + ("colorPulse", "Color Pulse", AnimClass::Emphasis, 27, 250, &[]), + ("teeter", "Teeter", AnimClass::Emphasis, 32, 1000, &[]), + ("spin", "Spin", AnimClass::Emphasis, 8, 2000, &["clockwise", "counterClockwise"]), + ("growShrink", "Grow/Shrink", AnimClass::Emphasis, 6, 2000, &["both", "horz", "vert"]), + ("desaturate", "Desaturate", AnimClass::Emphasis, 25, 500, &[]), + ("darken", "Darken", AnimClass::Emphasis, 24, 500, &[]), + ("lighten", "Lighten", AnimClass::Emphasis, 30, 500, &[]), + ("transparency", "Transparency", AnimClass::Emphasis, 9, 500, &[]), + ("objectColor", "Object Color", AnimClass::Emphasis, 19, 500, &[]), + ("complementaryColor", "Complementary Color", AnimClass::Emphasis, 21, 500, &[]), + ("lineColor", "Line Color", AnimClass::Emphasis, 7, 2000, &[]), + ("fillColor", "Fill Color", AnimClass::Emphasis, 1, 2000, &[]), + ("fontColor", "Font Color", AnimClass::Emphasis, 3, 2000, &[]), + ("underline", "Underline", AnimClass::Emphasis, 18, 500, &[]), + ("boldFlash", "Bold Flash", AnimClass::Emphasis, 10, 2000, &[]), + ("wave", "Wave", AnimClass::Emphasis, 34, 500, &[]), + ("disappear", "Disappear", AnimClass::Exit, 1, 0, &[]), + ("fadeOut", "Fade", AnimClass::Exit, 10, 500, &[]), + ("flyOut", "Fly Out", AnimClass::Exit, 2, 500, &["b", "lb", "l", "lt", "t", "rt", "r", "rb"]), + ("floatOut", "Float Out", AnimClass::Exit, 42, 1000, &["u", "d"]), + ("splitOut", "Split", AnimClass::Exit, 16, 500, &["horzIn", "horzOut", "vertIn", "vertOut"]), + ("wipeOut", "Wipe", AnimClass::Exit, 22, 500, &["b", "l", "t", "r"]), + ("shapeOut", "Shape", AnimClass::Exit, 6, 2000, &["circleIn", "circleOut"]), + ("wheelOut", "Wheel", AnimClass::Exit, 21, 2000, &["1", "2", "3", "4", "8"]), + ("randomBarsOut", "Random Bars", AnimClass::Exit, 14, 500, &["horz", "vert"]), + ("shrinkTurn", "Shrink & Turn", AnimClass::Exit, 31, 1000, &[]), + ("zoomOut", "Zoom", AnimClass::Exit, 53, 500, &["in", "out"]), + ("swivelOut", "Swivel", AnimClass::Exit, 45, 2000, &["horz", "vert"]), + ("bounceOut", "Bounce", AnimClass::Exit, 26, 2000, &[]), + ("dissolveOut", "Dissolve Out", AnimClass::Exit, 9, 500, &[]), + ("blindsOut", "Blinds", AnimClass::Exit, 3, 500, &["horz", "vert"]), + ("boxOut", "Box", AnimClass::Exit, 4, 2000, &["in", "out"]), + ("checkerboardOut", "Checkerboard", AnimClass::Exit, 5, 500, &["across", "down"]), + ("diamondOut", "Diamond", AnimClass::Exit, 8, 2000, &["in", "out"]), + ("peekOut", "Peek Out", AnimClass::Exit, 12, 500, &["b", "l", "t", "r"]), + ("stripsOut", "Strips", AnimClass::Exit, 18, 500, &["lu", "ru", "ld", "rd"]), + ("sinkDown", "Sink Down", AnimClass::Exit, 37, 1000, &[]), + ("spinnerOut", "Spinner", AnimClass::Exit, 49, 500, &[]), + ("contract", "Contract", AnimClass::Exit, 55, 1000, &[]), + ("lines", "Lines", AnimClass::Path, 64, 2000, &["down", "up", "right", "left"]), + ("arcs", "Arcs", AnimClass::Path, 37, 2000, &["down", "up", "right", "left"]), + ("turns", "Turns", AnimClass::Path, 43, 2000, &["down", "up", "right", "left"]), + ( + "shapes", + "Shapes", + AnimClass::Path, + 1, + 2000, + &["circle", "diamond", "hexagon", "triangle", "square", "trapezoid", "octagon", "parallelogram", "pentagon"], + ), + ("loops", "Loops", AnimClass::Path, 61, 2000, &[]), + ("diagonalDownRight", "Diagonal Down Right", AnimClass::Path, 49, 2000, &[]), + ("diagonalUpRight", "Diagonal Up Right", AnimClass::Path, 56, 2000, &[]), + ("zigzag", "Zigzag", AnimClass::Path, 38, 2000, &[]), + ("sineWave", "Sine Wave", AnimClass::Path, 40, 2000, &[]), + ("pathWave", "Wave", AnimClass::Path, 47, 2000, &[]), + ("decayingWave", "Decaying Wave", AnimClass::Path, 60, 2000, &[]), + ("heartbeat", "Heartbeat", AnimClass::Path, 45, 2000, &[]), + ("sCurve1", "S Curve 1", AnimClass::Path, 59, 2000, &[]), + ("sCurve2", "S Curve 2", AnimClass::Path, 39, 2000, &[]), + ("stairsDown", "Stairs Down", AnimClass::Path, 62, 2000, &[]), + ("bounceRight", "Bounce Right", AnimClass::Path, 54, 2000, &[]), + ("bounceLeft", "Bounce Left", AnimClass::Path, 41, 2000, &[]), + ("spiralRight", "Spiral Right", AnimClass::Path, 46, 2000, &[]), + ("spiralLeft", "Spiral Left", AnimClass::Path, 55, 2000, &[]), + ("spring", "Spring", AnimClass::Path, 53, 2000, &[]), + ("funnel", "Funnel", AnimClass::Path, 52, 2000, &[]), + ("swoosh", "Swoosh", AnimClass::Path, 30, 2000, &[]), + ("star4", "4 Point Star", AnimClass::Path, 16, 2000, &[]), + ("star5", "5 Point Star", AnimClass::Path, 5, 2000, &[]), + ("star6", "6 Point Star", AnimClass::Path, 11, 2000, &[]), + ("star8", "8 Point Star", AnimClass::Path, 17, 2000, &[]), + ("heart", "Heart", AnimClass::Path, 9, 2000, &[]), + ("crescentMoon", "Crescent Moon", AnimClass::Path, 6, 2000, &[]), + ("teardrop", "Teardrop", AnimClass::Path, 18, 2000, &[]), + ("football", "Football", AnimClass::Path, 12, 2000, &[]), + ("plusPath", "Plus", AnimClass::Path, 32, 2000, &[]), + ("rightTriangle", "Right Triangle", AnimClass::Path, 2, 2000, &[]), + ("curvedSquare", "Curved Square", AnimClass::Path, 20, 2000, &[]), + ("figure8Horizontal", "Horizontal Figure 8", AnimClass::Path, 26, 2000, &[]), + ("figure8Vertical", "Vertical Figure 8", AnimClass::Path, 22, 2000, &[]), + ("loopDeLoop", "Loop de Loop", AnimClass::Path, 24, 2000, &[]), + ("bean", "Bean", AnimClass::Path, 31, 2000, &[]), + ("peanut", "Peanut", AnimClass::Path, 27, 2000, &[]), + ("neutron", "Neutron", AnimClass::Path, 29, 2000, &[]), + ("customPath", "Custom Path", AnimClass::Path, 0, 2000, &[]), + ("play", "Play", AnimClass::Media, 1, 0, &[]), + ("pause", "Pause", AnimClass::Media, 2, 0, &[]), + ("stop", "Stop", AnimClass::Media, 3, 0, &[]), +]; + +pub fn animation_info(effect: &str, class: AnimClass) -> Option<&'static AnimationInfo> { + ANIMATIONS.iter().find(|a| a.0 == effect && a.2 == class).or_else(|| ANIMATIONS.iter().find(|a| a.0 == effect)) +} + +/// The path of a motion-path preset beyond Lines, Arcs, Turns, Shapes and Loops, in slide +/// fractions from the shape's position (our path syntax: `M`, `L`, `Z`, ending in `E`). +/// `None` for an effect that isn't one of these presets. +pub fn preset_path(effect: &str) -> Option { + use std::f64::consts::{PI, TAU}; + use std::fmt::Write; + // Sample `f` over 0..=1 into a polyline starting at (0, 0). + fn poly(n: usize, closed: bool, f: impl Fn(f64) -> (f64, f64)) -> String { + let (x0, y0) = f(0.0); + let mut s = String::from("M 0 0"); + for k in 1..=n { + let (x, y) = f(k as f64 / n as f64); + let _ = write!(s, " L {:.4} {:.4}", x - x0, y - y0); + } + if closed { + s.push_str(" Z"); + } + s.push_str(" E"); + s + } + fn pts(p: &[(f64, f64)], closed: bool) -> String { + let mut s = String::from("M 0 0"); + for (x, y) in p { + let _ = write!(s, " L {x:.4} {y:.4}"); + } + if closed { + s.push_str(" Z"); + } + s.push_str(" E"); + s + } + // A closed curve in polar form r(θ), starting at its top. + fn polar(scale: f64, r: impl Fn(f64) -> f64) -> String { + poly(96, true, |t| { + let a = -PI / 2.0 + TAU * t; + let rr = r(a + PI / 2.0) * scale; + (rr * a.cos(), rr * a.sin()) + }) + } + fn star(n: usize, outer: f64, inner: f64) -> String { + let mut p = Vec::new(); + let start = (0.0, -outer); + for k in 1..=2 * n { + let a = -PI / 2.0 + PI * k as f64 / n as f64; + let r = if k % 2 == 1 { inner } else { outer }; + p.push((r * a.cos() - start.0, r * a.sin() - start.1)); + } + pts(&p, true) + } + let spiral = |dir: f64| { + poly(120, false, move |t| { + let r = 0.12 * (1.0 - t); + let a = dir * 3.0 * TAU * t; + (dir * (0.12 - r * a.cos()), -r * a.sin()) + }) + }; + let bounce = |dir: f64| poly(90, false, move |t| (dir * 0.3 * t, 0.15 * t - 0.1 * (1.0 - t) * (3.0 * PI * t).sin().abs())); + Some(match effect { + "diagonalDownRight" => pts(&[(0.2, 0.2)], false), + "diagonalUpRight" => pts(&[(0.2, -0.2)], false), + "zigzag" => pts(&[(0.05, 0.04), (0.1, 0.0), (0.15, 0.04), (0.2, 0.0), (0.25, 0.04), (0.3, 0.0)], false), + "sineWave" => poly(64, false, |t| (0.3 * t, 0.04 * (2.0 * TAU * t).sin())), + "pathWave" => poly(48, false, |t| (0.3 * t, 0.06 * (TAU * t).sin())), + "decayingWave" => poly(96, false, |t| (0.3 * t, 0.07 * (-3.0 * t).exp() * (3.0 * TAU * t).sin())), + "heartbeat" => pts(&[(0.08, 0.0), (0.1, -0.08), (0.13, 0.06), (0.15, 0.0), (0.25, 0.0)], false), + "sCurve1" => poly(48, false, |t| (0.2 * t, 0.1 - 0.1 * (PI * t).cos())), + "sCurve2" => poly(48, false, |t| (0.1 - 0.1 * (PI * t).cos(), 0.2 * t)), + "stairsDown" => pts(&[(0.06, 0.0), (0.06, 0.06), (0.12, 0.06), (0.12, 0.12), (0.18, 0.12), (0.18, 0.18)], false), + "bounceRight" => bounce(1.0), + "bounceLeft" => bounce(-1.0), + "spiralRight" => spiral(1.0), + "spiralLeft" => spiral(-1.0), + "spring" => poly(160, false, |t| (0.3 * t + 0.03 * (5.0 * TAU * t).cos(), 0.03 * (5.0 * TAU * t).sin())), + "funnel" => poly(160, false, |t| { + let r = 0.12 * (1.0 - t); + (r * (3.0 * TAU * t).cos(), 0.25 * t + 0.03 * (3.0 * TAU * t).sin() * (1.0 - t)) + }), + "swoosh" => poly(48, false, |t| (0.3 * t, -0.12 * (PI * t).sin() + 0.1 * t * t)), + "star4" => star(4, 0.1, 0.04), + "star5" => star(5, 0.1, 0.04), + "star6" => star(6, 0.1, 0.05), + "star8" => star(8, 0.1, 0.06), + "heart" => poly(96, true, |t| { + let a = TAU * t; + (0.006 * 16.0 * a.sin().powi(3), -0.006 * (13.0 * a.cos() - 5.0 * (2.0 * a).cos() - 2.0 * (3.0 * a).cos() - (4.0 * a).cos())) + }), + "crescentMoon" => { + // The outer arc of a circle, back along a smaller, offset one. + let mut p: Vec<(f64, f64)> = (1..=32) + .map(|k| { + let a = -PI / 2.0 - PI * k as f64 / 32.0; + (0.1 * a.cos(), 0.1 + 0.1 * a.sin()) + }) + .collect(); + p.extend((1..=32).map(|k| { + let a = PI / 2.0 - PI * k as f64 / 32.0; + (0.06 * a.cos() - 0.03, 0.1 + 0.1 * a.sin()) + })); + pts(&p, true) + } + "teardrop" => polar(0.1, |a| 0.6 + 0.4 * (a / 2.0).cos().abs().powf(3.0)), + "football" => poly(64, true, |t| { + let a = TAU * t; + (0.12 * a.sin(), 0.06 * (1.0 - a.cos()) * (1.0 - 0.3 * (2.0 * a).cos().abs())) + }), + "plusPath" => pts( + &[ + (0.04, 0.0), + (0.04, 0.04), + (0.08, 0.04), + (0.08, 0.08), + (0.04, 0.08), + (0.04, 0.12), + (0.0, 0.12), + (0.0, 0.08), + (-0.04, 0.08), + (-0.04, 0.04), + (0.0, 0.04), + ], + true, + ), + "rightTriangle" => pts(&[(0.0, 0.15), (0.15, 0.15)], true), + "curvedSquare" => poly(96, true, |t| { + let a = -PI / 2.0 + TAU * t; + let sg = |v: f64| v.signum() * v.abs().sqrt(); + (0.1 * sg(a.cos()), 0.1 + 0.1 * sg(a.sin())) + }), + "figure8Horizontal" => poly(96, true, |t| { + let a = TAU * t; + (0.15 * a.sin(), 0.06 * (2.0 * a).sin()) + }), + "figure8Vertical" => poly(96, true, |t| { + let a = TAU * t; + (0.06 * (2.0 * a).sin(), 0.15 * (1.0 - a.cos())) + }), + "loopDeLoop" => poly(120, false, |t| (0.3 * t - 0.05 * (2.0 * TAU * t).sin(), -0.05 * (1.0 - (2.0 * TAU * t).cos()))), + "bean" => polar(0.1, |a| 0.75 + 0.25 * a.cos() + 0.15 * (2.0 * a).cos()), + "peanut" => polar(0.1, |a| 0.7 + 0.3 * (2.0 * a).cos().abs()), + "neutron" => polar(0.1, |a| 0.35 + 0.65 * (3.0 * a).cos().abs()), + _ => return None, + }) +} + +#[cfg(test)] +mod option_label_tests { + use super::*; + + #[test] + fn transition_option_labels() { + assert_eq!(transition_option_label("push", "l"), "From Right"); + assert_eq!(transition_option_label("push", "u"), "From Bottom"); + assert_eq!(transition_option_label("wipe", "r"), "From Left"); + assert_eq!(transition_option_label("cover", "ru"), "From Bottom-Left"); + assert_eq!(transition_option_label("ripple", "ru"), "From Bottom-Left"); + assert_eq!(transition_option_label("glitter", "hexU"), "Hexagons From Top"); + assert_eq!(transition_option_label("glitter", "l"), "Diamonds From Left"); + assert_eq!(transition_option_label("wind", "l"), "Left"); + assert_eq!(transition_option_label("pageCurlDouble", "singleR"), "Single Right"); + assert_eq!(transition_option_label("shred", "particlesOut"), "Particles Out"); + assert_eq!(transition_option_label("zoom", "rotate"), "Zoom and Rotate"); + assert_eq!(transition_option_label("split", "vertIn"), "Vertical In"); + for (id, _, _, _, opts) in TRANSITIONS { + for o in opts.iter() { + assert!(!transition_option_label(id, o).is_empty(), "{id}/{o}"); + } + } + } +} + +#[cfg(test)] +mod build_role_tests { + use super::*; + use text::TextBody; + + #[test] + fn blank_and_deeper_paragraphs_follow_the_top_one_before_them() { + let mut t = TextBody::from_text("Before\nLead\n \nSub\nNext"); + for (p, lvl) in t.paragraphs.iter_mut().zip([1, 0, 0, 1, 0]) { + p.level = lvl; + } + use BuildRole::*; + assert_eq!(paragraph_build_roles(&t), vec![Unowned, Top, Member(1), Member(1), Top]); + } +} diff --git a/crates/pptx/src/lib.rs b/crates/pptx/src/lib.rs new file mode 100644 index 000000000..ac935300a --- /dev/null +++ b/crates/pptx/src/lib.rs @@ -0,0 +1,1079 @@ +//! PresentationML (ECMA-376 part 1, chapter 19): the model of a `.pptx` presentation. +//! +//! A [`Presentation`] has slide masters (each with a theme and layouts) and slides. Each slide +//! uses a layout; placeholders on a slide inherit position, formatting and text styles from the +//! matching placeholder on its layout and master ([`placeholder::parents`]). Slides and masters +//! sit behind `Arc`s, so a snapshot of the whole deck costs one clone of the slide list. +//! +//! DrawingML (fills, lines, text bodies, tables, charts, themes) comes from [`craft_drawingml`]; +//! [`anim`] holds transitions and animations. Media bytes live in [`Presentation::media`] and +//! everything else names them by [`MediaId`]. The types derive serde with camelCase names, so a +//! presentation can be stored as JSON. +#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] + +pub mod anim; +pub mod placeholder; + +use std::sync::Arc; + +pub use craft_drawingml::color::{ColorScheme, ColorTransform, Rgba, SchemeSlot}; +pub use craft_drawingml::custgeom::CustomPath; +pub use craft_drawingml::diagram::{Diagram, DiagramNode}; +pub use craft_drawingml::style::{self, ColorBase, ColorRef, Effects, Fill, Line, ShapeStyle}; +pub use craft_drawingml::table::{self, Cell, Table}; +pub use craft_drawingml::text::{self, Paragraph, Run, RunProps, TextBody}; +pub use craft_drawingml::theme::{self, Theme}; +pub use craft_drawingml::{MediaId, SlideId, Xfrm, chart, chart::Chart}; +pub use kurbo::{Rect, Size}; +use serde::{Deserialize, Serialize}; + +pub use anim::{AnimClass, AnimStart, Animation, Transition}; + +macro_rules! id_type { + ($(#[$doc:meta])* $name:ident) => { + $(#[$doc])* + #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)] + #[serde(transparent)] + pub struct $name(pub u32); + impl std::fmt::Display for $name { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}", self.0) + } + } + }; +} +id_type!( + /// A shape, unique across the presentation. + ShapeId +); +id_type!( + /// A slide layout. + LayoutId +); +id_type!( + /// A slide master. + MasterId +); + +/// Placeholder types. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum PhType { + Title, + CtrTitle, + SubTitle, + #[default] + Body, + Obj, + Chart, + Table, + ClipArt, + Diagram, + Media, + Picture, + Date, + Footer, + SlideNum, + Header, + SlideImage, +} + +impl PhType { + pub fn xml(self) -> &'static str { + match self { + PhType::Title => "title", + PhType::CtrTitle => "ctrTitle", + PhType::SubTitle => "subTitle", + PhType::Body => "body", + PhType::Obj => "obj", + PhType::Chart => "chart", + PhType::Table => "tbl", + PhType::ClipArt => "clipArt", + PhType::Diagram => "dgm", + PhType::Media => "media", + PhType::Picture => "pic", + PhType::Date => "dt", + PhType::Footer => "ftr", + PhType::SlideNum => "sldNum", + PhType::Header => "hdr", + PhType::SlideImage => "sldImg", + } + } + pub fn from_xml(s: &str) -> Self { + match s { + "title" => PhType::Title, + "ctrTitle" => PhType::CtrTitle, + "subTitle" => PhType::SubTitle, + "obj" => PhType::Obj, + "chart" => PhType::Chart, + "tbl" => PhType::Table, + "clipArt" => PhType::ClipArt, + "dgm" => PhType::Diagram, + "media" => PhType::Media, + "pic" => PhType::Picture, + "dt" => PhType::Date, + "ftr" => PhType::Footer, + "sldNum" => PhType::SlideNum, + "hdr" => PhType::Header, + "sldImg" => PhType::SlideImage, + _ => PhType::Body, + } + } + pub fn is_title(self) -> bool { + matches!(self, PhType::Title | PhType::CtrTitle) + } + /// The prompt shown in an empty placeholder in Normal view. + pub fn prompt(self) -> &'static str { + match self { + PhType::Title | PhType::CtrTitle => "Click to add title", + PhType::SubTitle => "Click to add subtitle", + PhType::Body | PhType::Obj => "Click to add text", + PhType::Chart => "Click icon to add chart", + PhType::Table => "Click icon to add table", + PhType::Diagram => "Click icon to add SmartArt graphic", + PhType::Media => "Click icon to add media", + PhType::Picture | PhType::ClipArt => "Click icon to add picture", + PhType::Date => "Date", + PhType::Footer => "Footer", + PhType::SlideNum => "‹#›", + PhType::Header => "Header", + PhType::SlideImage => "", + } + } + /// Footer-row placeholders are not shown on slides unless the deck turns them on. + pub fn is_footer_kind(self) -> bool { + matches!(self, PhType::Date | PhType::Footer | PhType::SlideNum | PhType::Header) + } +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Placeholder { + pub kind: PhType, + pub idx: u32, + /// `vert` orientation flag. + pub vertical: bool, + /// `quarter`, `half`, `full` size hint. + pub size: Option, + pub has_custom_prompt: bool, +} + +/// Shape geometry: a preset name with adjust values, or a custom path. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", tag = "kind")] +pub enum Geom { + Preset { + name: String, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + adj: Vec, + }, + /// Custom geometry: sub-paths in their own coordinate space (w, h) scaled to the shape. + Custom { paths: Vec }, +} + +impl Default for Geom { + fn default() -> Self { + Geom::Preset { name: "rect".into(), adj: vec![] } + } +} + +impl Geom { + pub fn preset(name: &str) -> Self { + Geom::Preset { name: name.into(), adj: vec![] } + } + pub fn preset_name(&self) -> Option<&str> { + match self { + Geom::Preset { name, .. } => Some(name), + Geom::Custom { .. } => None, + } + } +} + +/// Embedded audio or video on a slide. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct MediaClip { + pub media: MediaId, + pub video: bool, + /// Poster frame image. + pub poster: Option, + /// Crop of the picture (left, top, right, bottom as fractions), PowerPoint's `srcRect` on the video. + pub crop: [f64; 4], + pub trim_start_ms: u32, + pub trim_end_ms: u32, + pub fade_in_ms: u32, + pub fade_out_ms: u32, + /// 0–1. + pub volume: f64, + pub autoplay: bool, + pub loop_play: bool, + pub rewind: bool, + pub play_across_slides: bool, + pub hide_while_not_playing: bool, + pub full_screen: bool, + /// Bookmarks (name, ms). + pub bookmarks: Vec<(String, u32)>, + /// Probed length in ms (0 = unknown). + pub duration_ms: u32, + /// Probed picture size of a video in pixels (0 = unknown). + pub width: u32, + pub height: u32, +} + +/// One ink stroke (Draw tab): points in slide coordinates with pressure (`0..=1`, +/// where `1` is [`InkStroke::width`]). +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct InkStroke { + pub points: Vec<(f64, f64, f32)>, + pub color: Rgba, + pub width: f64, + pub highlighter: bool, +} + +/// True when every sample is the placeholder `0.5` written before a device reported pressure. +/// Those strokes were drawn at [`InkStroke::width`] and still are. +pub fn ink_pressure_is_legacy(points: &[(f64, f64, f32)]) -> bool { + !points.is_empty() && points.iter().all(|p| (p.2 - 0.5).abs() <= 1.0e-3) +} + +/// Width of one ink sample. `pressure` of `1` is the pen width. A legacy flat stroke +/// (`legacy`) draws at the pen width. A missing reading draws at the pen width too. +/// A light touch stays visible (at least 8% of the pen). +pub fn ink_sample_width(pen: f64, pressure: f32, legacy: bool) -> f64 { + let pen = if pen.is_finite() { pen.abs() } else { 0.0 }; + if legacy { + return pen; + } + let p = if pressure.is_finite() { pressure.clamp(0.08, 1.0) } else { 1.0 }; + pen * f64::from(p) +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", tag = "type")] +pub enum ShapeKind { + /// An autoshape or text box. + Shape, + Picture { + fill: style::PictureFill, + }, + Group { + children: Vec, + /// Child coordinate space. + child: Xfrm, + }, + Connector { + start: Option<(ShapeId, u32)>, + end: Option<(ShapeId, u32)>, + }, + Table(Table), + Chart(Box), + Media(MediaClip), + Ink { + strokes: Vec, + }, + /// A 3-D model (`p:contentPart` with a `model3d` relationship, usually a `.glb`). + /// `poster` is the fallback picture an `mc:AlternateContent` choice carried for older readers. + Model3d { + media: MediaId, + #[serde(default)] + poster: Option, + }, + /// Something we read but don't model (SmartArt without fallback, OLE…): its XML is kept and its + /// fallback picture, if any, is shown. + Opaque { + xml: String, + preview: Option, + label: String, + /// The parts its XML points to (SmartArt data, an OLE embedding…), so a save writes them + /// back: (relationship id used in `xml`, relationship type, the part as a media item). + #[serde(default, skip_serializing_if = "Vec::is_empty")] + parts: Vec<(String, String, MediaId)>, + }, +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Shape { + pub id: ShapeId, + pub name: String, + /// `None`: placeholder inheriting its box from the layout/master. + pub xfrm: Option, + pub kind: ShapeKind, + pub geom: Geom, + pub ph: Option, + /// `None`: inherited (placeholder chain or style); `Some(Fill::None)`: no fill. + pub fill: Option, + pub line: Option, + pub effects: Option, + pub style: Option, + pub text: Option, + pub hidden: bool, + pub locked: bool, + /// Text box (`txBox="1"`): no fill, autosize, wrap off by default. + pub text_box: bool, + /// Alt text. + pub descr: String, + pub title: String, + pub decorative: bool, + pub click: Option, + pub hover: Option, + /// Add-in tags (`p:custDataLst` → `ppt/tags/tagN.xml`): name/value pairs kept for round trip. + #[serde(skip_serializing_if = "Vec::is_empty")] + pub tags: Vec<(String, String)>, + /// Unmodelled XML kept for round-trip (extension lists etc.). + #[serde(skip_serializing_if = "Option::is_none")] + pub raw_ext: Option, + /// On a SmartArt group: the diagram's text outline, from which its shapes are regenerated. + #[serde(skip_serializing_if = "Option::is_none")] + pub diagram: Option, +} + +impl Default for Shape { + fn default() -> Self { + Shape { + id: ShapeId(0), + name: String::new(), + xfrm: None, + kind: ShapeKind::Shape, + geom: Geom::default(), + ph: None, + fill: None, + line: None, + effects: None, + style: None, + text: None, + hidden: false, + locked: false, + text_box: false, + descr: String::new(), + title: String::new(), + decorative: false, + click: None, + hover: None, + tags: vec![], + raw_ext: None, + diagram: None, + } + } +} + +impl Shape { + /// A stand-in shape for laying out one table cell's `body`: a table, so the cell's text takes + /// the master's other-text style as PowerPoint's does. + pub fn table_cell(body: text::TextBody) -> Shape { + Shape { text: Some(body), kind: ShapeKind::Table(Table::default()), ..Default::default() } + } + pub fn is_placeholder(&self) -> bool { + self.ph.is_some() + } + pub fn ph_type(&self) -> Option { + self.ph.as_ref().map(|p| p.kind) + } + pub fn is_group(&self) -> bool { + matches!(self.kind, ShapeKind::Group { .. }) + } + pub fn children(&self) -> &[Shape] { + match &self.kind { + ShapeKind::Group { children, .. } => children, + _ => &[], + } + } + pub fn children_mut(&mut self) -> Option<&mut Vec> { + match &mut self.kind { + ShapeKind::Group { children, .. } => Some(children), + _ => None, + } + } + /// Is this a line-like shape (no area)? + pub fn is_line(&self) -> bool { + matches!(self.kind, ShapeKind::Connector { .. }) || self.geom.preset_name().is_some_and(craft_drawingml::preset::is_line_like) + } + pub fn kind_name(&self) -> &'static str { + match &self.kind { + ShapeKind::Shape if self.ph.is_some() => "placeholder", + ShapeKind::Shape if self.text_box => "textBox", + ShapeKind::Shape => "shape", + ShapeKind::Picture { .. } => "picture", + ShapeKind::Group { .. } => "group", + ShapeKind::Connector { .. } => "connector", + ShapeKind::Table(_) => "table", + ShapeKind::Chart(_) => "chart", + ShapeKind::Media(m) if m.video => "video", + ShapeKind::Media(_) => "audio", + ShapeKind::Ink { .. } => "ink", + ShapeKind::Model3d { .. } => "model", + ShapeKind::Opaque { .. } => "object", + } + } +} + +/// Walk shapes depth-first (groups included). +pub fn walk<'a>(shapes: &'a [Shape], f: &mut dyn FnMut(&'a Shape, usize)) { + fn rec<'a>(shapes: &'a [Shape], depth: usize, f: &mut dyn FnMut(&'a Shape, usize)) { + if depth > 64 { + return; + } + for s in shapes { + f(s, depth); + rec(s.children(), depth + 1, f); + } + } + rec(shapes, 0, f); +} + +pub fn find_shape(shapes: &[Shape], id: ShapeId) -> Option<&Shape> { + fn rec(shapes: &[Shape], id: ShapeId, depth: usize) -> Option<&Shape> { + if depth > 64 { + return None; + } + for s in shapes { + if s.id == id { + return Some(s); + } + if let Some(f) = rec(s.children(), id, depth + 1) { + return Some(f); + } + } + None + } + rec(shapes, id, 0) +} + +pub fn find_shape_mut(shapes: &mut [Shape], id: ShapeId) -> Option<&mut Shape> { + fn rec(shapes: &mut [Shape], id: ShapeId, depth: usize) -> Option<&mut Shape> { + if depth > 64 { + return None; + } + for s in shapes.iter_mut() { + if s.id == id { + return Some(s); + } + if let ShapeKind::Group { children, .. } = &mut s.kind + && let Some(f) = rec(children, id, depth + 1) + { + return Some(f); + } + } + None + } + rec(shapes, id, 0) +} + +/// The list holding `id` among its siblings: the tree's top level, or a group's children. +pub fn sibling_list_mut(shapes: &mut Vec, id: ShapeId) -> Option<&mut Vec> { + fn rec(shapes: &mut Vec, id: ShapeId, depth: usize) -> Option<&mut Vec> { + if depth > 64 { + return None; + } + if shapes.iter().any(|s| s.id == id) { + return Some(shapes); + } + for s in shapes.iter_mut() { + if let ShapeKind::Group { children, .. } = &mut s.kind + && let Some(f) = rec(children, id, depth + 1) + { + return Some(f); + } + } + None + } + rec(shapes, id, 0) +} + +/// The group directly holding `id`: `Some(None)` for a top-level shape, `None` when absent. +pub fn parent_of(shapes: &[Shape], id: ShapeId) -> Option> { + fn rec(shapes: &[Shape], parent: Option, id: ShapeId, depth: usize) -> Option> { + if depth > 64 { + return None; + } + for s in shapes { + if s.id == id { + return Some(parent); + } + if let Some(f) = rec(s.children(), Some(s.id), id, depth + 1) { + return Some(f); + } + } + None + } + rec(shapes, None, id, 0) +} + +/// Background of a slide, layout or master. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", tag = "kind")] +#[allow(clippy::large_enum_variant)] // the native JSON shape is fixed; boxing would not change it but would change the API +pub enum Background { + Fill { + fill: Fill, + }, + /// Theme background style reference (1001–1003) with a colour. + Ref { + idx: u32, + color: ColorRef, + }, +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Comment { + pub author: String, + pub initials: String, + pub text: String, + /// ISO 8601. + pub date: String, + pub x: f64, + pub y: f64, + pub resolved: bool, + pub replies: Vec, + /// The shape the comment is attached to. + pub shape: Option, +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Slide { + pub id: SlideId, + pub layout: LayoutId, + pub name: String, + pub shapes: Vec, + pub background: Option, + pub hidden: bool, + pub show_master_shapes: bool, + pub transition: Option, + pub animations: Vec, + /// Speaker notes. + pub notes: TextBody, + pub comments: Vec, + /// `clrMapOvr`: a colour map replacing the layout's or master's (`bg1`→`dk1`…); None follows + /// the layout. + #[serde(skip_serializing_if = "Option::is_none")] + pub color_map: Option>, + /// Add-in tags (`p:custDataLst` → `ppt/tags/tagN.xml`): name/value pairs kept for round trip. + #[serde(skip_serializing_if = "Vec::is_empty")] + pub tags: Vec<(String, String)>, + /// Unmodelled XML kept for round-trip (custom data). + #[serde(skip_serializing_if = "Option::is_none")] + pub raw_ext: Option, +} + +impl Default for Slide { + fn default() -> Self { + Slide { + id: SlideId(0), + layout: LayoutId(0), + name: String::new(), + shapes: vec![], + background: None, + hidden: false, + show_master_shapes: true, + transition: None, + animations: vec![], + notes: TextBody::default(), + comments: vec![], + color_map: None, + tags: vec![], + raw_ext: None, + } + } +} + +impl Slide { + pub fn shape(&self, id: ShapeId) -> Option<&Shape> { + find_shape(&self.shapes, id) + } + pub fn shape_mut(&mut self, id: ShapeId) -> Option<&mut Shape> { + find_shape_mut(&mut self.shapes, id) + } + /// The slide's title text (first title placeholder), for thumbnails, outline and navigation. + pub fn title(&self) -> String { + self.shapes + .iter() + .find(|s| s.ph_type().is_some_and(PhType::is_title)) + .and_then(|s| s.text.as_ref()) + .map(|t| t.text().replace('\u{b}', " ")) + .unwrap_or_default() + } + pub fn notes_text(&self) -> String { + self.notes.text() + } +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum LayoutType { + Title, + #[default] + TitleAndContent, + SectionHeader, + TwoContent, + Comparison, + TitleOnly, + Blank, + ContentWithCaption, + PictureWithCaption, + TitleAndVerticalText, + VerticalTitleAndText, + Custom, +} + +impl LayoutType { + pub fn xml(self) -> &'static str { + match self { + LayoutType::Title => "title", + LayoutType::TitleAndContent => "obj", + LayoutType::SectionHeader => "secHead", + LayoutType::TwoContent => "twoObj", + LayoutType::Comparison => "twoTxTwoObj", + LayoutType::TitleOnly => "titleOnly", + LayoutType::Blank => "blank", + LayoutType::ContentWithCaption => "objTx", + LayoutType::PictureWithCaption => "picTx", + LayoutType::TitleAndVerticalText => "vertTx", + LayoutType::VerticalTitleAndText => "vertTitleAndTx", + LayoutType::Custom => "cust", + } + } + pub fn from_xml(s: &str) -> Self { + match s { + "title" => LayoutType::Title, + "obj" | "tx" => LayoutType::TitleAndContent, + "secHead" => LayoutType::SectionHeader, + "twoObj" | "twoColTx" => LayoutType::TwoContent, + "twoTxTwoObj" => LayoutType::Comparison, + "titleOnly" => LayoutType::TitleOnly, + "blank" => LayoutType::Blank, + "objTx" => LayoutType::ContentWithCaption, + "picTx" => LayoutType::PictureWithCaption, + "vertTx" => LayoutType::TitleAndVerticalText, + "vertTitleAndTx" => LayoutType::VerticalTitleAndText, + _ => LayoutType::Custom, + } + } +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Layout { + pub id: LayoutId, + pub name: String, + pub kind: LayoutType, + pub shapes: Vec, + pub background: Option, + pub show_master_shapes: bool, + pub preserve: bool, + /// `clrMapOvr`: a colour map replacing the master's for this layout and the slides using it + /// (dark "section" layouts map `bg1` to `dk1`); None follows the master. + #[serde(skip_serializing_if = "Option::is_none")] + pub color_map: Option>, + #[serde(skip_serializing_if = "Option::is_none")] + pub raw_ext: Option, +} + +/// Which placeholders slides show in the footer row (Header & Footer dialog). +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct HeaderFooter { + pub date: bool, + pub slide_number: bool, + pub footer: bool, + pub header: bool, + pub footer_text: String, + /// Fixed date text; empty = automatic date. + pub date_text: String, + /// `datetime1`…`datetime13` format for automatic dates. + pub date_format: String, + pub hide_on_title: bool, +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Master { + pub id: MasterId, + pub name: String, + pub theme: Theme, + pub shapes: Vec, + pub background: Option, + pub title_style: text::ListStyle, + pub body_style: text::ListStyle, + pub other_style: text::ListStyle, + pub layouts: Vec, + /// `bg1`→`lt1` etc. Default mapping when empty. + pub color_map: Vec<(String, String)>, + pub preserve: bool, + #[serde(skip_serializing_if = "Option::is_none")] + pub raw_ext: Option, +} + +impl Master { + pub fn layout(&self, id: LayoutId) -> Option<&Layout> { + self.layouts.iter().find(|l| l.id == id) + } + /// Resolve `bg1`/`tx1`/`bg2`/`tx2` through the colour map. + pub fn map_slot(&self, slot: SchemeSlot) -> SchemeSlot { + map_slot(&self.color_map, slot) + } + /// The theme's colour scheme with the colour map applied (`bg1`… resolve to mapped colours). + pub fn scheme(&self) -> ColorScheme { + self.scheme_with(&self.color_map) + } + /// The theme's colour scheme with `map` (a layout's or slide's override, or the master's own) + /// applied. + pub fn scheme_with(&self, map: &[(String, String)]) -> ColorScheme { + let mut s = self.theme.colors.clone(); + if map.is_empty() { + return s; + } + let base = self.theme.colors.clone(); + for (alias, slot) in [ + (SchemeSlot::Bg1, SchemeSlot::Lt1), + (SchemeSlot::Tx1, SchemeSlot::Dk1), + (SchemeSlot::Bg2, SchemeSlot::Lt2), + (SchemeSlot::Tx2, SchemeSlot::Dk2), + ] { + s.set(slot, base.get(map_slot(map, alias))); + } + s + } +} + +/// Resolve `bg1`/`tx1`/`bg2`/`tx2` through the colour map `map` (the default mapping when it has +/// no entry). +pub fn map_slot(map: &[(String, String)], slot: SchemeSlot) -> SchemeSlot { + let key = match slot { + SchemeSlot::Bg1 => "bg1", + SchemeSlot::Tx1 => "tx1", + SchemeSlot::Bg2 => "bg2", + SchemeSlot::Tx2 => "tx2", + other => return other, + }; + map.iter().find(|(k, _)| k == key).and_then(|(_, v)| SchemeSlot::from_xml(v)).unwrap_or(match slot { + SchemeSlot::Bg1 => SchemeSlot::Lt1, + SchemeSlot::Tx1 => SchemeSlot::Dk1, + SchemeSlot::Bg2 => SchemeSlot::Lt2, + _ => SchemeSlot::Dk2, + }) +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Section { + pub name: String, + pub slides: Vec, +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct CustomShow { + pub name: String, + pub slides: Vec, +} + +/// Slide Show ▸ Set Up Slide Show. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct ShowSettings { + /// `speaker`, `browsed`, `kiosk`. + pub show_type: String, + pub loop_until_esc: bool, + pub without_narration: bool, + pub without_animation: bool, + pub use_timings: bool, + /// 1-based inclusive range; `None` = all. + pub range: Option<(u32, u32)>, + pub custom_show: Option, + pub pen_color: Rgba, + pub laser_color: Rgba, + pub show_media_controls: bool, +} + +impl Default for ShowSettings { + fn default() -> Self { + ShowSettings { + show_type: "speaker".into(), + loop_until_esc: false, + without_narration: false, + without_animation: false, + use_timings: true, + range: None, + custom_show: None, + pen_color: Rgba::rgb(0xFF, 0x00, 0x00), + laser_color: Rgba::rgb(0xFF, 0x00, 0x00), + show_media_controls: true, + } + } +} + +/// Document properties (File ▸ Properties). +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Properties { + pub title: String, + pub subject: String, + pub author: String, + pub keywords: String, + pub comments: String, + pub category: String, + pub company: String, + pub created: String, + pub modified: String, + pub last_modified_by: String, + pub revision: u32, +} + +/// One typeface embedded in the presentation (`p:embeddedFont`). Each slot is a font file in +/// [`Presentation::media`]: regular, bold, italic, bold italic. A slot is absent when the file +/// didn't include that face. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct EmbeddedFont { + pub typeface: String, + /// `panose` on `p:font`, when the file gave the 20 hex digits. + #[serde(skip_serializing_if = "Option::is_none")] + pub panose: Option, + /// `pitchFamily` on `p:font`. + #[serde(skip_serializing_if = "Option::is_none")] + pub pitch_family: Option, + /// `charset` on `p:font`. + #[serde(skip_serializing_if = "Option::is_none")] + pub charset: Option, + pub regular: Option, + pub bold: Option, + pub italic: Option, + pub bold_italic: Option, +} + +impl EmbeddedFont { + /// The four face slots, in the order the package stores them. + pub fn faces(&self) -> [Option; 4] { + [self.regular, self.bold, self.italic, self.bold_italic] + } +} + +/// An embedded file (image, audio, video, font, workbook). +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct MediaItem { + pub id: MediaId, + /// Original file name, e.g. `image1.png`. + pub name: String, + pub content_type: String, + /// Bytes; kept out of the JSON manifest and stored as a separate file in native files. + #[serde(skip)] + pub data: Arc>, + /// External link instead of embedded bytes. + pub link: Option, +} + +/// A size whose sides may be missing, null or not numbers (NaN then); see `Presentation::slide_size`. +fn lenient_size<'de, D: serde::Deserializer<'de>>(d: D) -> std::result::Result { + let v = serde_json::Value::deserialize(d)?; + let side = |k: &str| v.get(k).and_then(serde_json::Value::as_f64).unwrap_or(f64::NAN); + Ok(Size::new(side("width"), side("height"))) +} + +/// An XML package part the reader does not model, kept as written. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct KeptPart { + /// Part name from the package root (`customXml/item1.xml`). + pub name: String, + /// Content type; empty for a relationships part. + pub content_type: String, + pub xml: String, + /// Full relationship type linking it, empty when only its sibling parts point at it. + pub rel_type: String, + /// Linked from the package root rather than from the presentation part. + pub from_root: bool, +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(default, rename_all = "camelCase")] +pub struct Presentation { + /// Slide size in points (default 16:9, 960 × 540). Read leniently: a missing, null or + /// non-numeric side becomes NaN for `the caller` to replace, instead of the + /// whole file failing to open. + #[serde(deserialize_with = "lenient_size")] + pub slide_size: Size, + #[serde(deserialize_with = "lenient_size")] + pub notes_size: Size, + pub slides: Vec>, + pub masters: Vec>, + pub notes_master: Option>, + pub handout_master: Option>, + pub sections: Vec
, + pub custom_shows: Vec, + pub show: ShowSettings, + pub header_footer: HeaderFooter, + pub props: Properties, + /// Presentation-wide default text style (shapes that aren't placeholders). + pub default_text_style: text::ListStyle, + pub media: Vec, + pub first_slide_number: u32, + /// Next free shape/slide/layout/media id. + pub next_id: u32, + /// Embedded fonts. The face bytes live in [`Self::media`]. + pub embedded_fonts: Vec, + /// Table styles defined in the file (`ppt/tableStyles.xml`). + #[serde(skip_serializing_if = "Vec::is_empty")] + pub table_styles: Vec, + /// Add-in tags (`p:custDataLst` → `ppt/tags/tagN.xml`): name/value pairs kept for round trip. + #[serde(skip_serializing_if = "Vec::is_empty")] + pub tags: Vec<(String, String)>, + /// Package parts kept verbatim for round trip: custom XML data (SharePoint metadata), the + /// sensitivity label and custom document properties. + #[serde(skip_serializing_if = "Vec::is_empty")] + pub kept_parts: Vec, + /// Drawing guides (View › Guides), shared by every slide (`p:guideLst` in viewProps). + #[serde(skip_serializing_if = "Vec::is_empty")] + pub guides: Vec, + /// Unmodelled presentation-level XML kept for round-trip. + #[serde(skip_serializing_if = "Option::is_none")] + pub raw_ext: Option, +} + +/// A drawing guide: a vertical line at `pos` points from the slide's left edge, or a horizontal +/// one from its top. +#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct DrawGuide { + pub vertical: bool, + pub pos: f64, +} + +impl Default for Presentation { + /// An empty deck: 16:9 slides (960 × 540 points), no masters and no slides. An application + /// builds its own starting deck (masters, layouts, text styles) on top of this. + fn default() -> Self { + Presentation { + slide_size: Size::new(960.0, 540.0), + notes_size: Size::new(540.0, 720.0), + slides: vec![], + masters: vec![], + notes_master: None, + handout_master: None, + sections: vec![], + custom_shows: vec![], + show: ShowSettings::default(), + header_footer: HeaderFooter::default(), + props: Properties::default(), + default_text_style: text::ListStyle::default(), + media: vec![], + first_slide_number: 1, + next_id: 0, + embedded_fonts: vec![], + table_styles: vec![], + tags: vec![], + kept_parts: vec![], + guides: vec![], + raw_ext: None, + } + } +} + +impl Presentation { + /// A fresh id, unique across slides, shapes, layouts, masters and media in this deck. `None` + /// once every id is in use (a hostile file can arrive that way): the deck cannot grow + /// further, and callers must refuse the change rather than duplicate ids. + pub fn alloc_id(&mut self) -> Option { + let next = self.next_id.max(1).checked_add(1)?; + self.next_id = next; + Some(next) + } + /// Make sure `next_id` is above every id in use (after loading a file). + pub fn fix_next_id(&mut self) { + let mut m = self.next_id; + for s in &self.slides { + m = m.max(s.id.0); + walk(&s.shapes, &mut |sh, _| m = m.max(sh.id.0)); + } + for ma in &self.masters { + m = m.max(ma.id.0); + walk(&ma.shapes, &mut |sh, _| m = m.max(sh.id.0)); + for l in &ma.layouts { + m = m.max(l.id.0); + walk(&l.shapes, &mut |sh, _| m = m.max(sh.id.0)); + } + } + for md in &self.media { + m = m.max(md.id.0); + } + for extra in [self.notes_master.as_deref(), self.handout_master.as_deref()].into_iter().flatten() { + m = m.max(extra.id.0); + walk(&extra.shapes, &mut |sh, _| m = m.max(sh.id.0)); + } + self.next_id = m; + } + pub fn slide_index(&self, id: SlideId) -> Option { + self.slides.iter().position(|s| s.id == id) + } + pub fn slide(&self, id: SlideId) -> Option<&Slide> { + self.slides.iter().find(|s| s.id == id).map(AsRef::as_ref) + } + pub fn slide_mut(&mut self, id: SlideId) -> Option<&mut Slide> { + self.slides.iter_mut().find(|s| s.id == id).map(Arc::make_mut) + } + pub fn slide_at(&self, i: usize) -> Option<&Slide> { + self.slides.get(i).map(AsRef::as_ref) + } + /// Master and layout for a layout id. + pub fn layout(&self, id: LayoutId) -> Option<(&Master, &Layout)> { + self.masters.iter().find_map(|m| m.layout(id).map(|l| (m.as_ref(), l))) + } + /// The master (and layout) a slide uses, falling back to the first. + pub fn master_for(&self, slide: &Slide) -> Option<(&Master, Option<&Layout>)> { + if let Some((m, l)) = self.layout(slide.layout) { + return Some((m, Some(l))); + } + self.masters.first().map(|m| (m.as_ref(), m.layouts.first())) + } + pub fn first_master(&self) -> Option<&Master> { + self.masters.first().map(AsRef::as_ref) + } + pub fn media(&self, id: MediaId) -> Option<&MediaItem> { + self.media.iter().find(|m| m.id == id) + } + /// Add media bytes, reusing an identical existing item. + pub fn add_media(&mut self, name: &str, content_type: &str, data: Vec) -> Option { + if let Some(m) = self.media.iter().find(|m| m.data.len() == data.len() && *m.data == data) { + return Some(m.id); + } + let id = MediaId(self.alloc_id()?); + self.media.push(MediaItem { id, name: name.into(), content_type: content_type.into(), data: Arc::new(data), link: None }); + Some(id) + } + /// Section index containing a slide (by position), when sections exist. + pub fn section_of(&self, slide: SlideId) -> Option { + self.sections.iter().position(|s| s.slides.contains(&slide)) + } + /// 1-based slide number as displayed (respects `first_slide_number`). + pub fn slide_number(&self, index: usize) -> u32 { + self.first_slide_number.saturating_add(u32::try_from(index).unwrap_or(u32::MAX)) + } + /// Basic consistency check: unique slide/shape ids, layouts exist. Returns problems found. + pub fn validate(&self) -> Vec { + let mut out = vec![]; + let mut seen = std::collections::HashSet::new(); + // Indexed once: a per-slide scan of every layout is slides × layouts work. + let layouts: std::collections::HashSet<_> = self.masters.iter().flat_map(|m| m.layouts.iter().map(|l| l.id)).collect(); + for s in &self.slides { + if !seen.insert(s.id) { + out.push(format!("duplicate slide id {}", s.id)); + } + if !layouts.contains(&s.layout) { + out.push(format!("slide {} uses missing layout {}", s.id, s.layout)); + } + let mut ids = std::collections::HashSet::new(); + walk(&s.shapes, &mut |sh, _| { + if !ids.insert(sh.id) { + out.push(format!("slide {}: duplicate shape id {}", s.id, sh.id)); + } + }); + } + out + } +} + +#[cfg(test)] +mod tests; diff --git a/crates/pptx/src/placeholder.rs b/crates/pptx/src/placeholder.rs new file mode 100644 index 000000000..3ee82846e --- /dev/null +++ b/crates/pptx/src/placeholder.rs @@ -0,0 +1,106 @@ +//! Placeholder inheritance: which layout and master placeholders a placeholder takes its +//! position, formatting and text styles from (`p:ph` `type` and `idx`, ECMA-376 part 1, 19.3.1.36). +//! +//! Only the matching lives here. What a caller does with the parents (effective position, fill, +//! text style) is its own business. + +use crate::{Layout, Master, PhType, Shape}; + +/// Where a shape lives. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Owner { + Slide, + Layout, + Master, +} + +/// Can a slide placeholder of type `slide` inherit from a layout placeholder of type `other`? +fn compatible(slide: PhType, other: PhType) -> bool { + use PhType::*; + match slide { + Title | CtrTitle => matches!(other, Title | CtrTitle), + SubTitle | Body | Obj | Chart | Table | ClipArt | Diagram | Media | Picture => { + !matches!(other, Title | CtrTitle | Date | Footer | SlideNum | Header) + } + k => k == other, + } +} + +/// The placeholder in `shapes` that `shape` matches: by `idx` (when `by_idx`) and a compatible +/// type, else the same type, else a compatible one. +pub fn find_ph<'s>(shapes: &'s [Shape], shape: &Shape, by_idx: bool) -> Option<&'s Shape> { + let ph = shape.ph.as_ref()?; + if by_idx + && ph.idx != 0 + && let Some(s) = shapes.iter().find(|s| s.ph.as_ref().is_some_and(|p| p.idx == ph.idx && compatible(ph.kind, p.kind))) + { + return Some(s); + } + // Same type first, then a compatible one. + shapes + .iter() + .find(|s| s.ph.as_ref().is_some_and(|p| p.kind == ph.kind && (!by_idx || ph.idx == 0 || p.idx == ph.idx || ph.kind.is_title()))) + .or_else(|| shapes.iter().find(|s| s.ph.as_ref().is_some_and(|p| p.kind == ph.kind))) + .or_else(|| shapes.iter().find(|s| s.ph.as_ref().is_some_and(|p| compatible(ph.kind, p.kind)))) +} + +/// The master placeholder a placeholder of type `kind` inherits from: the title, the matching +/// footer-row placeholder, or the body. +pub fn master_ph(master: &Master, kind: PhType) -> Option<&Shape> { + let want = match kind { + PhType::Title | PhType::CtrTitle => PhType::Title, + PhType::Date | PhType::Footer | PhType::SlideNum | PhType::Header => kind, + _ => PhType::Body, + }; + master.shapes.iter().find(|s| s.ph_type() == Some(want)) +} + +/// The layout and master placeholders `shape` inherits from. A slide's placeholder matches one on +/// `layout`, and the master placeholder follows the layout one's type; a layout's placeholder +/// inherits from the master only; a master's from nothing. +pub fn parents<'a>(master: &'a Master, layout: Option<&'a Layout>, owner: Owner, shape: &Shape) -> (Option<&'a Shape>, Option<&'a Shape>) { + let Some(ph) = shape.ph.as_ref() else { return (None, None) }; + match owner { + Owner::Slide => { + let lp = layout.and_then(|l| find_ph(&l.shapes, shape, true)); + let kind = lp.and_then(Shape::ph_type).unwrap_or(ph.kind); + (lp, master_ph(master, kind)) + } + Owner::Layout => (None, master_ph(master, ph.kind)), + Owner::Master => (None, None), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::{Placeholder, ShapeId}; + + fn ph(id: u32, kind: PhType, idx: u32) -> Shape { + Shape { id: ShapeId(id), ph: Some(Placeholder { kind, idx, ..Default::default() }), ..Default::default() } + } + + #[test] + fn slide_placeholders_match_by_index_then_type() { + let master = Master { shapes: vec![ph(1, PhType::Title, 0), ph(2, PhType::Body, 1), ph(3, PhType::SlideNum, 12)], ..Default::default() }; + let layout = Layout { shapes: vec![ph(10, PhType::Title, 0), ph(11, PhType::Body, 1), ph(12, PhType::Obj, 2)], ..Default::default() }; + let id = |s: Option<&Shape>| s.map(|s| s.id.0); + // By idx: idx 2 is the layout's obj, whose master parent is the body. + let (l, m) = parents(&master, Some(&layout), Owner::Slide, &ph(20, PhType::Body, 2)); + assert_eq!((id(l), id(m)), (Some(12), Some(2))); + // A centred title takes the layout's title and the master's title. + let (l, m) = parents(&master, Some(&layout), Owner::Slide, &ph(21, PhType::CtrTitle, 0)); + assert_eq!((id(l), id(m)), (Some(10), Some(1))); + // An unknown idx falls back to the same type. + let (l, _) = parents(&master, Some(&layout), Owner::Slide, &ph(22, PhType::Body, 9)); + assert_eq!(id(l), Some(11)); + // A title never inherits from a body, nor a slide number from a title. + assert_eq!(find_ph(&layout.shapes[1..], &ph(23, PhType::Title, 0), true).map(|s| s.id.0), None); + let (l, m) = parents(&master, Some(&layout), Owner::Slide, &ph(24, PhType::SlideNum, 12)); + assert_eq!((id(l), id(m)), (None, Some(3))); + // Layouts inherit from the master only, masters from nothing, other shapes from nothing. + assert_eq!(parents(&master, None, Owner::Layout, &ph(25, PhType::SubTitle, 1)), (None, master.shapes.get(1))); + assert_eq!(parents(&master, None, Owner::Master, &ph(26, PhType::Title, 0)), (None, None)); + assert_eq!(parents(&master, Some(&layout), Owner::Slide, &Shape::default()), (None, None)); + } +} diff --git a/crates/pptx/src/tests.rs b/crates/pptx/src/tests.rs new file mode 100644 index 000000000..dececf12f --- /dev/null +++ b/crates/pptx/src/tests.rs @@ -0,0 +1,131 @@ +use super::*; + +#[test] +fn serde_roundtrip() { + let mut p = Presentation::default(); + p.masters.push(Arc::new(Master { layouts: vec![Layout { id: LayoutId(2), ..Default::default() }], ..Default::default() })); + p.slides.push(Arc::new(Slide { id: SlideId(3), layout: LayoutId(2), ..Default::default() })); + let j = serde_json::to_string(&p).unwrap(); + let q: Presentation = serde_json::from_str(&j).unwrap(); + assert_eq!(p, q); + // Unknown and missing fields are tolerated. + let q: Presentation = serde_json::from_str(r#"{"slides":[],"zzz":1}"#).unwrap(); + assert!(q.slides.is_empty()); + assert_eq!(q.slide_size, Size::new(960.0, 540.0)); + // A side that is missing, null or not a number reads as NaN instead of failing the file. + let q: Presentation = serde_json::from_str(r#"{"slideSize":{"width":720,"height":null}}"#).unwrap(); + assert_eq!(q.slide_size.width, 720.0); + assert!(q.slide_size.height.is_nan()); +} + +/// A deck whose ids are used up (a hostile file can arrive that way) yields no further ids: +/// allocating never repeats one. +#[test] +fn exhausted_id_space_allocates_nothing() { + let mut p = Presentation { next_id: u32::MAX, ..Default::default() }; + assert_eq!(p.alloc_id(), None); + assert_eq!(p.alloc_id(), None); + assert!(p.add_media("x.png", "image/png", vec![9]).is_none(), "fresh media bytes are refused too"); +} + +#[test] +fn ids_are_unique_and_fix_next_id() { + let mut p = Presentation::default(); + let shape = Shape { id: ShapeId(40), ..Default::default() }; + p.slides.push(Arc::new(Slide { id: SlideId(30), layout: LayoutId(50), shapes: vec![shape], ..Default::default() })); + p.masters.push(Arc::new(Master { id: MasterId(7), layouts: vec![Layout { id: LayoutId(50), ..Default::default() }], ..Default::default() })); + p.fix_next_id(); + assert_eq!(p.next_id, 50); + assert_eq!(p.alloc_id(), Some(51)); + let m = p.add_media("a.png", "image/png", vec![1, 2]).unwrap(); + assert_eq!(p.add_media("b.png", "image/png", vec![1, 2]), Some(m), "identical bytes are shared"); + assert!(p.validate().is_empty(), "{:?}", p.validate()); + Arc::make_mut(&mut p.slides[0]).layout = LayoutId(99); + assert_eq!(p.validate(), ["slide 30 uses missing layout 99"]); +} + +#[test] +fn group_find_and_walk() { + let child = Shape { id: ShapeId(900), ..Default::default() }; + let g = Shape { id: ShapeId(901), kind: ShapeKind::Group { children: vec![child], child: Xfrm::new(0.0, 0.0, 1.0, 1.0) }, ..Default::default() }; + let mut slide = Slide { shapes: vec![g], ..Default::default() }; + assert!(slide.shape(ShapeId(900)).is_some()); + assert!(slide.shape_mut(ShapeId(900)).is_some()); + let mut n = 0; + walk(&slide.shapes, &mut |_, _| n += 1); + assert_eq!(n, 2); +} + +#[test] +fn walks_stop_at_depth_64() { + let mut s = Shape { id: ShapeId(1), ..Default::default() }; + for i in 2..100 { + s = Shape { id: ShapeId(i), kind: ShapeKind::Group { children: vec![s], child: Xfrm::default() }, ..Default::default() }; + } + let tree = vec![s]; + let mut n = 0; + walk(&tree, &mut |_, _| n += 1); + assert_eq!(n, 65); + assert!(find_shape(&tree, ShapeId(1)).is_none()); + assert!(find_shape(&tree, ShapeId(40)).is_some()); +} + +#[test] +fn sibling_lists_and_parents_reach_into_groups() { + let leaf = |id: u32| Shape { id: ShapeId(id), ..Default::default() }; + let group = |id: u32, children: Vec| Shape { + id: ShapeId(id), + kind: ShapeKind::Group { children, child: Default::default() }, + ..Default::default() + }; + let mut tree = vec![leaf(1), group(2, vec![leaf(3), group(4, vec![leaf(5)])]), leaf(6)]; + assert_eq!(parent_of(&tree, ShapeId(1)), Some(None)); + assert_eq!(parent_of(&tree, ShapeId(3)), Some(Some(ShapeId(2)))); + assert_eq!(parent_of(&tree, ShapeId(5)), Some(Some(ShapeId(4)))); + assert_eq!(parent_of(&tree, ShapeId(9)), None); + let ids = |v: &Vec| v.iter().map(|s| s.id.0).collect::>(); + assert_eq!(sibling_list_mut(&mut tree, ShapeId(6)).map(|v| ids(v)), Some(vec![1, 2, 6])); + assert_eq!(sibling_list_mut(&mut tree, ShapeId(4)).map(|v| ids(v)), Some(vec![3, 4])); + assert_eq!(sibling_list_mut(&mut tree, ShapeId(5)).map(|v| ids(v)), Some(vec![5])); + assert!(sibling_list_mut(&mut tree, ShapeId(9)).is_none()); +} + +#[test] +fn ink_width_follows_pressure_and_keeps_the_old_placeholder() { + assert!(ink_pressure_is_legacy(&[(0.0, 0.0, 0.5), (1.0, 1.0, 0.5)])); + assert!(!ink_pressure_is_legacy(&[(0.0, 0.0, 0.5), (1.0, 1.0, 1.0)])); + assert!(!ink_pressure_is_legacy(&[])); + assert!((ink_sample_width(8.0, 0.5, true) - 8.0).abs() < 1e-9); + assert!((ink_sample_width(8.0, 1.0, false) - 8.0).abs() < 1e-9); + assert!((ink_sample_width(8.0, 0.25, false) - 2.0).abs() < 1e-9); + let floor = ink_sample_width(8.0, 0.0, false); + assert!((floor - 0.64).abs() < 1e-4, "a light touch stays visible, got {floor}"); + assert!((ink_sample_width(8.0, f32::NAN, false) - 8.0).abs() < 1e-9); +} + +#[test] +fn colour_maps_alias_the_background_and_text_slots() { + let mut m = Master::default(); + let base = m.theme.colors.clone(); + assert_eq!(m.map_slot(SchemeSlot::Bg1), SchemeSlot::Lt1, "the default mapping when empty"); + assert_eq!(m.map_slot(SchemeSlot::Accent2), SchemeSlot::Accent2); + let dark: Vec<(String, String)> = [("bg1", "dk1"), ("tx1", "lt1")].iter().map(|(a, b)| (a.to_string(), b.to_string())).collect(); + let s = m.scheme_with(&dark); + assert_eq!(s.get(SchemeSlot::Lt1), base.get(SchemeSlot::Dk1)); + assert_eq!(s.get(SchemeSlot::Dk1), base.get(SchemeSlot::Lt1)); + assert_eq!(s.get(SchemeSlot::Lt2), base.get(SchemeSlot::Lt2), "bg2 keeps its default mapping"); + m.color_map = dark; + assert_eq!(m.scheme(), s); +} + +#[test] +fn enum_names_round_trip_through_their_xml() { + for k in [PhType::Title, PhType::CtrTitle, PhType::Table, PhType::Diagram, PhType::SlideImage, PhType::SlideNum] { + assert_eq!(PhType::from_xml(k.xml()), k); + } + assert_eq!(PhType::from_xml("zzz"), PhType::Body); + for k in [LayoutType::Title, LayoutType::Comparison, LayoutType::VerticalTitleAndText, LayoutType::Custom] { + assert_eq!(LayoutType::from_xml(k.xml()), k); + } + assert_eq!(LayoutType::from_xml("twoColTx"), LayoutType::TwoContent); +} From 3540cf0a7f753608fe65f9eabfdf3d216922b37a Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:06:06 -0500 Subject: [PATCH 04/10] craft-metadata: read the Extended XMP a JPEG links, from LightCraft (#1046) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * craft-metadata: EXIF, IPTC-IIM, XMP and GPX from LightCraft lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif (read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse, write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers. They share Metadata, DateTime, Gps and Orientation. Changed in the move: - TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own. - The app namespace is a parameter: parse_xmp_with, write_xmp_with and MergeRules::namespaces take it, and XmpData no longer has lc_settings. - The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and craft-raw read those containers; from_embedded takes the blocks they find. * craft-metadata: a RW2 reads ISO and the rest of its Exif from the JpgFromRaw A RW2's own Exif IFD is a short subset. What it lacks is filled from the Exif of the embedded JpgFromRaw, then from the ISO tags of the Panasonic IFD0 (0x0017, 0x0037), each only in its recorded field type. 65535 in a SHORT ISOSpeedRatings is the overflow marker and states no ISO. The private tags are read only in a RW2 stream, under the caller's parse options. Ported from storytold/lightcraft#3518. * craft-metadata: the Exif rating is written, and 65535 reads as rejected write_exif writes Rating (0x4746), with 0xFFFF for a rejected photo, and from_tiff reads 65535 as -1 instead of clamping it to 5. Ported from storytold/lightcraft#1995. * craft-metadata: one shutter-speed formatter, exact to four digits format_exposure_time shows a duration within 0.1% of 1/n second as 1/n and anything else to four significant digits, so 0.3 s, 1/2.8 s and 0.625 s are no longer rounded to a reciprocal or one decimal. Metadata::exposure_display and the ExposureTime tag row both use it. Ported from storytold/lightcraft#1692. * craft-metadata: JPEG XL containers give their Exif and xml boxes The first Exif and xml box of a JPEG XL container, plain or Brotli-wrapped in brob, honouring the Exif box's TIFF header offset. A brob payload is capped at 32 MiB. A bare codestream has no metadata. Ported from storytold/lightcraft#3756. * craft-metadata: XMP Resource structs flatten their attribute fields A property with rdf:parseType="Resource" may give its fields as attributes as well as child elements. Only the children were flattened into XmpData::properties; the attributes now are too, as for a nested rdf:Description. Ported from storytold/lightcraft#1226 (the XMP part). * craft-metadata: a capture date of just a year or a month stays as stated XMP dates may stop at the year (1983) or the month (1983-04). DateTime now has a DatePrecision; such a date writes back to XMP as stated and only in photoshop:DateCreated, and write_exif leaves it out, since an Exif date cannot say it. Ported from storytold/lightcraft#3502. * craft-metadata: Fujifilm in-camera recipes from the maker note fuji module: the film simulation and the rest of the in-camera recipe (grain, colour chrome, tone, dynamic range, white balance and its shift, sharpness, noise reduction, clarity) from a Fujifilm maker note. recipe_from_exif, recipe_from_bytes (TIFF, JPEG, RAF) and read_recipe_file, which reads only as far as the metadata. from_embedded fills Metadata::fuji_recipe and file_tag_rows lists the recipe under Fujifilm. Ported from storytold/lightcraft#1209. * craft-metadata: the linear-time scan test compares times, not a fixed bound The test asserted that 200 000 attributes scan in under 5 s, which failed on a loaded machine. It now scans 25 000 and 100 000 attributes, takes the best of three runs of each, and asserts that four times the attributes take less than ten times as long. Linear scans give about 4; a quadratic duplicate check gave 18.6 and fails. * craft-metadata: interleave the linear-time scan test's runs The best-of-three ratio still failed once under load. The test now alternates five runs of 50 000 and 200 000 attributes and compares the fastest of each, so a burst of load slows both sizes. Six runs at load average 55 gave ratios of 3.9 to 5.4 against the bound of 10. * craft-metadata: XmpData::toolkit, the x:xmptk of the packet The name of the program that wrote a packet, from x:xmptk on x:xmpmeta (or x:xaptk on the older x:xapmeta), whatever prefix the packet binds to adobe:ns:meta/. write_xmp_with writes XmpWrite::toolkit there, and the test reads it back, escaped characters included. * craft-metadata: mpf, the Multi-Picture index of a JPEG mpf::images lists the images an APP2 MPF index (CIPA DC-007) puts after the primary image, with their file offsets; the index is read with craft-tiff, in either byte order, up to 16 entries. mpf::write inserts an index after the primary's leading APPn segments and appends the other images, with the primary listed as the representative baseline image: the layout LightCraft's gain map JPEG writer uses, for any number of images. mpf::markers lists the marker segments before the first scan, for APPn payloads such as ISO 21496-1 gain map metadata. * craft-metadata: edit, byte-preserving Exif and XMP edits from PhotoCraft PhotoCraft's orientation, resolution and Exif colour-space code (codecs orientation.rs without the pixel transforms, resolution.rs, exif_color.rs), at photocraft f1b5e7d72, as the edit module: - upright_exif and upright_xmp set Orientation to 1 in place, in the entry's own type and width, every well-formed duplicate included. - exif_with_resolution and xmp_with_resolution set the resolution in place and remove entries that cannot be rewritten; the _adding_missing variant appends a new IFD0 when there are none. - export_exif and export_xmp do both and drop the source Exif thumbnail, with the overlap scan bounded. - exif_orientation, exif_resolution, xmp_resolution, photoshop_resolution, exif_color_space and resolution read the same values with the editors' rules (first entry wins, SHORT or LONG only, BigTIFF accepted). The tests that call these functions directly move with them (the codec round trips stay in PhotoCraft): the orientation, resolution, thumbnail and compaction tests, the comparison-counting complexity test and the colour space tests. * craft-metadata: exif_with_pixel_size, the written size in PixelX/YDimension A camera records its sensor's size in the Exif IFD's PixelXDimension and PixelYDimension, which after a crop, resize or rotation names a size the exported file does not have. exif_with_pixel_size rewrites each one that holds one SHORT or LONG, in place and as a LONG, in classic and BigTIFF blocks, with the JPEG Exif prefix kept. Ported from storytold/photocraft#12407 (the format part). * craft-metadata: an ISO 8601 clock needs two-digit hours and minutes Ported from storytold/lightcraft#3523. * craft-metadata: a UTC offset needs a two-digit minute field Ported from storytold/lightcraft#3467. * craft-metadata: tiff:Orientation is read only as an integral code 1 to 8 A fractional or out-of-range value stays a raw property and does not become the photo's orientation. Ported from storytold/lightcraft#3495. * craft-metadata: an XMP ISO is read only when it is a whole number in range A fractional, non-positive or out-of-range ISO is not truncated or saturated into one; the next candidate property is tried instead. Ported from storytold/lightcraft#3633. * craft-metadata: XMP pixel dimensions are read only as exact whole numbers A malformed, fractional, negative or out-of-range tiff:ImageWidth or ImageLength is no longer truncated or saturated, and lets the exif:Pixel dimension take its place. Numbers are compared as exact decimal digits, so wide rationals stay exact. The ISO reader now uses the same exact reader (both PRs added one). Ported from storytold/lightcraft#3690. * craft-metadata: XMP GPS minutes and seconds must be under 60; tell when a read was cut short A coordinate such as 45,60N is no longer normalised to a plausible point; the raw property stays. XmpData::properties_truncated says the flattened property map ran past its key budget, Embedded::chunk_walk_incomplete says a PNG or WebP chunk walk stopped early, and jpeg_segments_with_all_xmp returns every ordinary XMP packet of a JPEG, so a caller that must prove a file carries no location can refuse what it could not inspect. Ported from storytold/lightcraft#3054. * craft-metadata: IPTC Time Created counts only when it is a whole, valid time A date with a six-digit clock and no zone stays date-only; HHMMSS±HHMM carries an offset; clocks and offsets are range-checked. A partial offset, extra digits, a leap second or trailing text no longer becomes a capture time, so it cannot block valid XMP or Exif. Ported from storytold/lightcraft#3448. * craft-metadata: test hostile XMP attribute references The one-pass reference scan (entity_end) already covers the bug that storytold/lightcraft#3007 fixed another way; its test is added here and passes on this implementation: unknown, unterminated and far-away references stay verbatim, a value of a million ampersands or of a million zeros is read once, and multi-byte text around a reference stays whole. Test from storytold/lightcraft#3007. * craft-metadata: parse_gps_position; an XMP coordinate's sign comes from its letter parse_gps_position reads an exif:GPSLatitude and GPSLongitude pair and checks both axes. A negative degree, minute or second part, or a letter of the other axis (10,30E as a latitude), is malformed; the letter may be either case. Ported from storytold/lightcraft#2088 (the format part). * craft-metadata: bound the XMP namespace declarations in scope; merge into an empty rdf:RDF Prefix lookup scans every declaration in scope, so one start tag with thousands of xmlns declarations made parsing and merging quadratic. More than 512 declarations in scope is now TooComplex, in parse_xmp and in merge_xmp's scan. merge_xmp also opens a self-closing around the new description instead of failing. Ported from storytold/lightcraft#3696. * craft-metadata: XmpData::has_location_metadata, every embedded XMP location has_location_metadata answers whether a packet names any Exif GPS property or a non-blank place (IPTC Core location fields, photoshop City/State/Country, IPTC Extension LocationShown and LocationCreated with a place or coordinates), including inside array items and repeated structs that the flattened map and the structured values keep only once (XmpData::names_location, from the parsed tree). The IPTC Extension namespace is now canonicalised and written. extended_xmp_guid reports the announced Extended XMP GUID. A caller that must prove a file has no location checks properties_truncated first. The source also listed LightCraft's own lc:location; a caller with its own namespace checks that itself. Ported from storytold/lightcraft#4121 (the format part). * craft-metadata: is_lens_spec_name, the inverse check of lens_spec_name Whether a lens name has the ' mm f/' shape lens_spec_name writes, so a caller can tell a made-up name from a camera's. The test is new. Ported from storytold/lightcraft#2644 (the format part). * craft-metadata: Sony focus points for 44 bodies SONY_MODELS grows from the alpha 1 and A7S III to the 44 bodies whose focus tags were checked on the public raw.pixls.us samples against ExifTool's reading; bodies whose 0x201b is not the focus mode stay out. This also covers the ILCE-6400 and ILCE-7M3 that storytold/lightcraft#3109 adds. Ported from storytold/lightcraft#1891 (the focus part). * craft-metadata: GPX points say which track they are on; tracks keep their names TrackPoint::track counts the a point came from, Tracklog::names holds each track's (trimmed, at most 200 characters), and only_track gives one track's points, so a map can draw a file's tracks apart and tag from one. At most 10 000 tracks per file (TooComplex past that); a non-finite elevation is dropped. Ported from storytold/lightcraft#921 (the GPX part; #916 carries the same). * craft-metadata: test that from_embedded keeps the EXIF, IPTC, XMP precedence storytold/lightcraft#2211 adds the same function as extract_embedded; this crate already has it as from_embedded. Its tests are added: EXIF camera fields stay, IPTC fills gaps, XMP overrides the user fields, and damaged blocks read as nothing. Tests from storytold/lightcraft#2211. * craft-metadata: the XMP writer keeps a carriage return as XML end-of-line handling turns a literal CR into a line feed when a packet is read, so text with one did not round-trip through other readers. Element text now writes it as a character reference. Ported from storytold/lightcraft#4119 (the writer fix only; that PR also moves LightCraft to quick-xml 0.42, which this crate does not take here, as craft-xml uses 0.41). * craft-metadata: README rows for the ported validation and location checks * craft-metadata: read the Extended XMP a JPEG's standard packet links A JPEG whose XMP is too large for one APP1 segment puts the rest in Extended XMP chunks, linked by xmpNote:HasExtendedXMP. When the standard packet links a GUID, only that GUID's chunks are read: they must be complete segments, declare one full length of at most 16 MiB, number at most 4096, and cover that length exactly once (Embedded::xmp_extended_error says why not). parse_xmp_pair reads the two packets as one (standard properties win; arrays and structs stated in both are combined), so extract and from_embedded now see the extension's GPS, caption and other fields. embedded_xmp reports a missing, malformed or unlinked extension as an EmbeddedXmpError that keeps the standard packet, for a caller that must see all of a file's metadata. An extension the standard packet does not link stays visible in Embedded::xmp_extended as before, and is not merged. Ported from storytold/lightcraft#2581, onto the current JPEG walker. --------- Co-authored-by: radialmonster Co-authored-by: Sebastian Berschneider Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Michael Anderson <1427851+endthestart@users.noreply.github.com> Co-authored-by: Ilya Dorman Co-authored-by: Clyde C Brown Co-authored-by: MohammadTeimori Co-authored-by: vangel76 Co-authored-by: Max Mittelstädt <62167861+TheGalitube@users.noreply.github.com> Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com> Co-authored-by: ronibalthazar <61232335+ronibalthazar@users.noreply.github.com> Co-authored-by: Andrew Frahn Co-authored-by: jkalkstein <76634771+jkalkstein@users.noreply.github.com> Co-authored-by: d-rouge Co-authored-by: espoon Co-authored-by: Yoann Laissus Co-authored-by: Jason Baldwin <33934381+bitshiftza@users.noreply.github.com> Co-authored-by: ingolfteetz Co-authored-by: smi-leonivkovic --- crates/metadata/README.md | 1 + crates/metadata/src/container.rs | 96 +++++++++- crates/metadata/src/lib.rs | 308 ++++++++++++++++++++++++++++++- crates/metadata/src/xmp.rs | 89 ++++++++- 4 files changed, 481 insertions(+), 13 deletions(-) diff --git a/crates/metadata/README.md b/crates/metadata/README.md index 72ea0063d..080f36eb8 100644 --- a/crates/metadata/README.md +++ b/crates/metadata/README.md @@ -19,6 +19,7 @@ are one crate. | `parse_xmp(&str) -> Result` | `Metadata`, every property (`prefix:name`, structs flattened as `a:b/c:d`) and the structured values. `parse_xmp_with` also canonicalises the caller's own namespaces. | | `write_xmp(&Metadata) -> String` | A complete `` packet. `write_xmp_with` sets the toolkit name and adds simple properties in the caller's namespace. | | `XmpData::has_location_metadata`, `XmpData::properties_truncated` | Whether a packet names GPS or a place anywhere (array items and repeated structs included), and whether the flattened map is complete, for a caller that must prove a file carries no location. `parse_gps_position` reads an XMP coordinate pair. | +| `embedded_xmp(&Embedded)`, `parse_xmp_pair` | A JPEG's standard XMP with the Extended XMP it links (`xmpNote:HasExtendedXMP`) read as one packet; an `EmbeddedXmpError` (keeping the standard packet) when the link or the extension cannot be read whole. `extract` and `from_embedded` use it best-effort. Extended XMP: 16 MiB, 4096 chunks, only the linked GUID's chunks counted. | | `merge_xmp(existing, fresh, MergeRules)` | Updates a packet another program wrote: owned properties are replaced, everything else stays byte for byte. | | `parse_gpx(&str) -> Result` | GPX track points with their track and segment; `Tracklog::locate` gives the position at a capture time, `Tracklog::only_track` one track. | | `embedded(&[u8]) -> Embedded` | Sniffs JPEG, PNG, WebP, TIFF-based files, Fujifilm RAF, Canon CR3 and JPEG XL containers and returns the Exif, XMP (and extended XMP), ICC and IPTC blocks. `jpeg_segments`, `png_chunks`, `webp_chunks` are the walkers. | diff --git a/crates/metadata/src/container.rs b/crates/metadata/src/container.rs index bb787c545..842020683 100644 --- a/crates/metadata/src/container.rs +++ b/crates/metadata/src/container.rs @@ -33,6 +33,10 @@ pub struct Embedded { pub iptc: Option>, /// True when the PNG/WebP chunk stream could not be fully inspected. pub chunk_walk_incomplete: bool, + /// Why the Extended XMP the standard packet links to (`xmpNote:HasExtendedXMP`) could not be + /// read: its chunks are missing, truncated, too large, or do not cover their declared length + /// exactly once. [`Embedded::xmp_extended`] is `None` then. + pub xmp_extended_error: Option, } /// Sniff the container and collect its metadata blocks. Unknown containers yield an empty result. @@ -70,6 +74,7 @@ pub fn embedded(bytes: &[u8]) -> Embedded { other => craft_tiff::writer::encode(t.order, other), }), chunk_walk_incomplete: false, + xmp_extended_error: None, } } else { Embedded::default() @@ -209,12 +214,97 @@ fn jpeg_segments_inner(b: &[u8], collect_all_xmp: bool) -> (Embedded, Vec)>, ext_headers: &[(&[u8], u32)]) { + // a standard packet that links an extension: only that GUID's chunks count, checked strictly + let reference = (!ext.is_empty()) + .then(|| out.xmp.as_deref().and_then(|x| crate::parse_xmp(x).ok()).and_then(|d| crate::xmp::extended_xmp_reference(&d).ok().flatten())) + .flatten(); + if let Some(guid) = reference { + match extended_xmp(b, &guid) { + Ok(packet) => { + out.xmp_extended = Some(packet); + out.xmp_extended_guid = Some(guid); + } + Err(error) => out.xmp_extended_error = Some(error), + } + } else if !ext.is_empty() { ext.sort_by_key(|p| p.0); - out.xmp_extended_guid = verified_extension_guid(&ext_headers, &ext); + out.xmp_extended_guid = verified_extension_guid(ext_headers, &ext); out.xmp_extended = Some(String::from_utf8_lossy(&ext.into_iter().flat_map(|p| p.1).collect::>()).into_owned()); } - (out, xmp_packets) +} + +/// Most bytes an Extended XMP packet may have, and most chunks it may be split into. +const MAX_EXTENDED_XMP_BYTES: usize = 16 * 1024 * 1024; +const MAX_EXTENDED_XMP_CHUNKS: usize = 4096; + +/// The Extended XMP packet of a JPEG for `guid` (upper-case hex, from the standard packet's +/// `xmpNote:HasExtendedXMP`; Adobe XMP Specification Part 3, 1.1.3.1): the APP1 chunks carrying +/// that GUID, which must all declare one full length of at most [`MAX_EXTENDED_XMP_BYTES`] and +/// together cover it exactly once, in at most [`MAX_EXTENDED_XMP_CHUNKS`] complete segments. +/// Chunks of any other GUID are not looked at, so they cannot exhaust the limits. +fn extended_xmp(b: &[u8], guid: &str) -> Result { + let mut chunks: Vec<(u32, &[u8])> = Vec::new(); + let (mut total, mut received) = (None, 0usize); + let mut i = 2usize; + while let (Some(&0xff), Some(&marker)) = (b.get(i), b.get(i + 1)) { + if marker == 0xff { + i += 1; + continue; + } + if marker == 0xd8 || (0xd0..=0xd7).contains(&marker) || marker == 0x01 { + i += 2; + continue; + } + if marker == 0xd9 || marker == 0xda { + break; + } + let Some(len) = b.get(i + 2..i + 4).map(|l| usize::from(u16::from_be_bytes([l[0], l[1]]))).filter(|&l| l >= 2) else { break }; + let (start, end) = (i + 4, i + 2 + len); + let payload = b.get(start..end.min(b.len())).unwrap_or_default(); + if let Some(p) = payload.strip_prefix(XMP_EXT_SIG).filter(|_| marker == 0xe1) + && p.get(..32).is_some_and(|g| g.eq_ignore_ascii_case(guid.as_bytes())) + { + if end > b.len() { + return Err("Extended XMP APP1 segment is truncated".into()); + } + let (Some(full), Some(offset), Some(data)) = (p.get(32..36), p.get(36..40), p.get(40..)) else { + return Err("Extended XMP APP1 header is truncated".into()); + }; + let full = u32::from_be_bytes([full[0], full[1], full[2], full[3]]); + if *total.get_or_insert(full) != full { + return Err("Extended XMP chunks declare different full lengths".into()); + } + received = received.saturating_add(data.len()); + if usize::try_from(full).map_or(true, |f| f > MAX_EXTENDED_XMP_BYTES) || received > MAX_EXTENDED_XMP_BYTES { + return Err("Extended XMP is larger than 16 MiB".into()); + } + if chunks.len() >= MAX_EXTENDED_XMP_CHUNKS { + return Err("Extended XMP has too many chunks".into()); + } + chunks.push((u32::from_be_bytes([offset[0], offset[1], offset[2], offset[3]]), data)); + } + i = end; + } + let total = total.ok_or("the referenced Extended XMP is missing")?; + chunks.sort_by_key(|c| c.0); + let mut packet = Vec::with_capacity(received); + for (offset, data) in chunks { + if usize::try_from(offset).ok() != Some(packet.len()) { + return Err("Extended XMP chunks have a gap, an overlap or a repeated offset".into()); + } + packet.extend_from_slice(data); + } + if u32::try_from(packet.len()).ok() != Some(total) || total == 0 { + return Err("Extended XMP chunks do not cover their declared full length".into()); + } + String::from_utf8(packet).map_err(|_| "Extended XMP is not UTF-8".into()) } /// The GUID shared by all Extended XMP segments when their offset-sorted chunks are contiguous from diff --git a/crates/metadata/src/lib.rs b/crates/metadata/src/lib.rs index baf9fe18c..1bc9660b8 100644 --- a/crates/metadata/src/lib.rs +++ b/crates/metadata/src/lib.rs @@ -332,7 +332,7 @@ pub fn extract(bytes: &[u8]) -> Metadata { /// "unreadable" uses [`try_read_exif`] and [`parse_xmp`] on the blocks first. pub fn from_embedded(e: &Embedded) -> Metadata { let mut m = e.exif.as_deref().map(read_exif).unwrap_or_default(); - let xmp = e.xmp.as_deref().and_then(|xmp| parse_xmp(xmp).ok()); + let xmp = embedded_xmp(e).unwrap_or_else(|error| error.standard.map(|x| *x)); // the size the Exif states: a size only the XMP states may be the upright one, so it can't tell how the photo is stored let stored = m.width.zip(m.height); // and the orientation the Exif states: the photo is displayed by that alone, so an orientation only the XMP @@ -355,6 +355,54 @@ pub fn from_embedded(e: &Embedded) -> Metadata { m } +/// The XMP of `e` read as one packet: the standard packet, with the Extended XMP it links to +/// (`xmpNote:HasExtendedXMP`) merged in (see [`xmp::parse_xmp_pair`]). An error when the standard +/// packet is not XMP, when its link is malformed, when the linked extension is missing or +/// unreadable ([`Embedded::xmp_extended_error`]), or when there is an extension nothing links to: +/// a caller that must see all of a file's metadata (to prove it carries no location, say) refuses +/// the file then. [`from_embedded`] falls back to [`EmbeddedXmpError::standard`]. +pub fn embedded_xmp(e: &Embedded) -> Result, EmbeddedXmpError> { + let fail = |standard: Option, message: String| EmbeddedXmpError { standard: standard.map(Box::new), message }; + let Some(packet) = e.xmp.as_deref() else { + return match (&e.xmp_extended, &e.xmp_extended_error) { + (None, None) => Ok(None), + _ => Err(fail(None, "Extended XMP is present without a standard packet".into())), + }; + }; + let parsed = parse_xmp(packet).map_err(|error| fail(None, format!("embedded XMP is invalid: {error}")))?; + let mut standard = parsed.clone(); + standard.properties.remove("xmpNote:HasExtendedXMP"); + standard.values.remove("xmpNote:HasExtendedXMP"); + let reference = xmp::extended_xmp_reference(&parsed).map_err(|error| fail(Some(standard.clone()), error))?; + match (reference, e.xmp_extended.as_deref()) { + (Some(guid), Some(extended)) if e.xmp_extended_guid.as_deref() == Some(guid.as_str()) => xmp::parse_xmp_pair(packet, extended, &[]) + .map(Some) + .map_err(|error| fail(Some(standard), format!("the linked Extended XMP is invalid: {error}"))), + (Some(_), _) => { + let why = e.xmp_extended_error.clone().unwrap_or_else(|| "the linked Extended XMP is missing".into()); + Err(fail(Some(standard), why)) + } + (None, Some(_)) => Err(fail(Some(standard), "Extended XMP is present but the standard packet does not link it".into())), + (None, None) => Ok(Some(standard)), + } +} + +/// Why [`embedded_xmp`] could not read a file's XMP whole. +#[derive(Clone, Debug, PartialEq)] +pub struct EmbeddedXmpError { + /// The standard packet, when it could be read: what a best-effort reader uses. + pub standard: Option>, + pub message: String, +} + +impl std::fmt::Display for EmbeddedXmpError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.message) + } +} + +impl std::error::Error for EmbeddedXmpError {} + /// Turn regions drawn on a photo as it is stored onto the upright photo. MWG gives each box as a fraction of /// `AppliedToDimensions` (`drawn_on`); for a photo whose Exif `orientation` turns it a quarter turn, some apps give the /// stored, sideways size there and boxes on the stored pixels to match (Lightroom says so per box with @@ -752,4 +800,262 @@ mod tests { assert_eq!(m.orientation, Some(Orientation::Rotate90)); assert!(near(m.regions[0].rect, DRAWN), "TIFF: {:?}", m.regions[0].rect); } + + const JPEG_XMP_SIG: &[u8] = b"http://ns.adobe.com/xap/1.0/\0"; + + const JPEG_XMP_EXT_SIG: &[u8] = b"http://ns.adobe.com/xmp/extension/\0"; + + fn jpeg_segment(marker: u8, payload: &[u8]) -> Vec { + let len = u16::try_from(payload.len() + 2).expect("test APP1 payload fits u16"); + let mut segment = vec![0xff, marker]; + segment.extend_from_slice(&len.to_be_bytes()); + segment.extend_from_slice(payload); + segment + } + + fn xmp_jpeg(standard: &str, guid: &str, extension: &str, chunks: &[(u32, &[u8])]) -> Vec { + let mut jpeg = vec![0xff, 0xd8]; + let mut packet = JPEG_XMP_SIG.to_vec(); + packet.extend_from_slice(standard.as_bytes()); + jpeg.extend(jpeg_segment(0xe1, &packet)); + for (offset, data) in chunks { + let mut payload = JPEG_XMP_EXT_SIG.to_vec(); + payload.extend_from_slice(guid.as_bytes()); + payload.extend_from_slice(&(extension.len() as u32).to_be_bytes()); + payload.extend_from_slice(&offset.to_be_bytes()); + payload.extend_from_slice(data); + jpeg.extend(jpeg_segment(0xe1, &payload)); + } + jpeg + } + + fn append_extended_chunk(jpeg: &mut Vec, guid: &str, total_length: u32, offset: u32, data: &[u8]) { + let mut payload = JPEG_XMP_EXT_SIG.to_vec(); + payload.extend_from_slice(guid.as_bytes()); + payload.extend_from_slice(&total_length.to_be_bytes()); + payload.extend_from_slice(&offset.to_be_bytes()); + payload.extend_from_slice(data); + jpeg.extend(jpeg_segment(0xe1, &payload)); + } + + fn xmp_document(body: &str) -> String { + format!( + r#""# + ) + } + + fn referenced_standard(guid: &str, extra: &str) -> String { + xmp_document(&format!(r#"xmlns:reference="http://ns.adobe.com/xmp/note/" reference:HasExtendedXMP="{guid}" {extra}"#)) + } + + fn split_chunks(packet: &str) -> [(u32, &[u8]); 2] { + let split = packet.len() / 2; + let (first, second) = packet.as_bytes().split_at(split); + [(split as u32, second), (0, first)] + } + + fn raf_with_jpeg(jpeg: &[u8]) -> Vec { + let offset = 120u32; + let length = u32::try_from(jpeg.len()).expect("test JPEG fits RAF length"); + let mut raf = b"FUJIFILMCCD-RAW 0201FF000000TEST".to_vec(); + raf.resize(offset as usize, 0); + raf[84..88].copy_from_slice(&offset.to_be_bytes()); + raf[88..92].copy_from_slice(&length.to_be_bytes()); + raf.extend_from_slice(jpeg); + raf + } + + #[test] + fn extended_xmp_gps_reaches_extractor() { + let guid = "A1234567890123456789012345678901"; + let extension = xmp_document(r#"xmlns:exif="http://ns.adobe.com/exif/1.0/" exif:GPSLatitude="45,0N" exif:GPSLongitude="7,0E""#); + let standard = referenced_standard(guid, ""); + let jpeg = xmp_jpeg(&standard, guid, &extension, &split_chunks(&extension)); + let gps = extract(&jpeg).gps.expect("the extension GPS reaches extraction"); + assert_eq!((gps.latitude, gps.longitude), (45.0, 7.0)); + } + + #[test] + fn extended_xmp_caption_reaches_extractor() { + let guid = "B1234567890123456789012345678901"; + let caption = "multi-chunk caption ".repeat(160); + let extension = xmp_document(&format!(r#"xmlns:dc="http://purl.org/dc/elements/1.1/">{caption}"#)); + let standard = referenced_standard(guid, ""); + let jpeg = xmp_jpeg(&standard, guid, &extension, &split_chunks(&extension)); + assert_eq!(extract(&jpeg).caption.as_deref(), Some(caption.trim())); + } + + #[test] + fn extended_xmp_split_gps_properties_are_combined() { + let guid = "C1234567890123456789012345678901"; + let standard = xmp_document(&format!( + r#"xmlns:note="http://ns.adobe.com/xmp/note/" xmlns:exif="http://ns.adobe.com/exif/1.0/" note:HasExtendedXMP="{guid}" exif:GPSLatitude="45,0N""# + )); + let extension = xmp_document(r#"xmlns:exif="http://ns.adobe.com/exif/1.0/" exif:GPSLongitude="7,0E""#); + let jpeg = xmp_jpeg(&standard, guid, &extension, &split_chunks(&extension)); + let gps = extract(&jpeg).gps.expect("GPS properties across packets compose"); + assert_eq!((gps.latitude, gps.longitude), (45.0, 7.0)); + } + + #[test] + fn extended_xmp_structured_values_are_composed() { + let guid = "F1234567890123456789012345678901"; + let standard = xmp_document(&format!( + r#"xmlns:note="http://ns.adobe.com/xmp/note/" xmlns:exif="http://ns.adobe.com/exif/1.0/" note:HasExtendedXMP="{guid}">True"# + )); + let extension = + xmp_document(r#"xmlns:exif="http://ns.adobe.com/exif/1.0/">2"#); + let jpeg = xmp_jpeg(&standard, guid, &extension, &split_chunks(&extension)); + assert_eq!(extract(&jpeg).flash.map(|flash| flash.raw), Some(17)); + } + + #[test] + fn extended_xmp_ignores_malformed_chunks_for_unreferenced_guids() { + let guid = "A1234567890123456789012345678901"; + let other_guid = "B1234567890123456789012345678901"; + let extension = xmp_document(r#"xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Referenced title""#); + let mut jpeg = xmp_jpeg(&referenced_standard(guid, ""), guid, &extension, &split_chunks(&extension)); + let mut malformed = JPEG_XMP_EXT_SIG.to_vec(); + malformed.extend_from_slice(other_guid.as_bytes()); + jpeg.extend(jpeg_segment(0xe1, &malformed)); + + assert_eq!(extract(&jpeg).title.as_deref(), Some("Referenced title")); + assert_eq!(embedded_xmp(&embedded(&jpeg)).unwrap().unwrap().metadata.title.as_deref(), Some("Referenced title")); + } + + #[test] + fn extended_xmp_ignores_chunk_limit_for_unreferenced_guids() { + let guid = "C1234567890123456789012345678901"; + let other_guid = "D1234567890123456789012345678901"; + let extension = xmp_document(r#"xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Referenced title""#); + let mut jpeg = xmp_jpeg(&referenced_standard(guid, ""), guid, &extension, &split_chunks(&extension)); + for _ in 0..4097 { + append_extended_chunk(&mut jpeg, other_guid, 1, 0, &[]); + } + + assert_eq!(embedded_xmp(&embedded(&jpeg)).unwrap().unwrap().metadata.title.as_deref(), Some("Referenced title")); + } + + #[test] + fn extended_xmp_ignores_received_byte_limit_for_unreferenced_guids() { + let guid = "E1234567890123456789012345678901"; + let other_guid = "F1234567890123456789012345678901"; + let extension = xmp_document(r#"xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Referenced title""#); + let mut jpeg = xmp_jpeg(&referenced_standard(guid, ""), guid, &extension, &split_chunks(&extension)); + let data = vec![0; 65_200]; + for _ in 0..258 { + append_extended_chunk(&mut jpeg, other_guid, 65_200, 0, &data); + } + + assert_eq!(embedded_xmp(&embedded(&jpeg)).unwrap().unwrap().metadata.title.as_deref(), Some("Referenced title")); + } + + #[test] + fn extended_xmp_rejects_mismatched_guid_and_bad_coverage() { + let guid = "D1234567890123456789012345678901"; + let extension = xmp_document(r#"xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Extension title""#); + let standard = referenced_standard(guid, r#"xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Standard title""#); + let bytes = extension.as_bytes(); + let middle = bytes.len() / 2; + let inconsistent_lengths = { + let mut jpeg = vec![0xff, 0xd8]; + let mut standard_payload = JPEG_XMP_SIG.to_vec(); + standard_payload.extend_from_slice(standard.as_bytes()); + jpeg.extend(jpeg_segment(0xe1, &standard_payload)); + for (offset, declared, part) in + [(0u32, bytes.len() as u32, &bytes[..middle]), (middle as u32, (bytes.len() + 1) as u32, &bytes[middle..])] + { + let mut payload = JPEG_XMP_EXT_SIG.to_vec(); + payload.extend_from_slice(guid.as_bytes()); + payload.extend_from_slice(&declared.to_be_bytes()); + payload.extend_from_slice(&offset.to_be_bytes()); + payload.extend_from_slice(part); + jpeg.extend(jpeg_segment(0xe1, &payload)); + } + jpeg + }; + let oversized_length = { + let mut jpeg = vec![0xff, 0xd8]; + let mut standard_payload = JPEG_XMP_SIG.to_vec(); + standard_payload.extend_from_slice(standard.as_bytes()); + jpeg.extend(jpeg_segment(0xe1, &standard_payload)); + let mut payload = JPEG_XMP_EXT_SIG.to_vec(); + payload.extend_from_slice(guid.as_bytes()); + payload.extend_from_slice(&(16u32 * 1024 * 1024 + 1).to_be_bytes()); + payload.extend_from_slice(&0u32.to_be_bytes()); + payload.extend_from_slice(bytes); + jpeg.extend(jpeg_segment(0xe1, &payload)); + jpeg + }; + let foreign_namespace_reference = { + let foreign_standard = xmp_document(&format!( + r#"xmlns:note="urn:example:foreign-note" note:HasExtendedXMP="{guid}" xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Standard title""# + )); + xmp_jpeg(&foreign_standard, guid, &extension, &split_chunks(&extension)) + }; + let mut truncated_segment = xmp_jpeg(&standard, guid, &extension, &split_chunks(&extension)); + truncated_segment.pop(); + + let cases = [ + xmp_jpeg(&standard, "E1234567890123456789012345678901", &extension, &split_chunks(&extension)), + xmp_jpeg(&standard, guid, &extension, &[(0, &bytes[..middle]), ((middle + 1) as u32, &bytes[middle..])]), + xmp_jpeg(&standard, guid, &extension, &[(0, &bytes[..middle]), ((middle - 1) as u32, &bytes[middle..])]), + xmp_jpeg(&standard, guid, &extension, &[(0, bytes), (0, bytes)]), + xmp_jpeg(&standard, guid, &extension, &[(0, &bytes[..middle])]), + inconsistent_lengths, + oversized_length, + truncated_segment, + foreign_namespace_reference, + ]; + for (index, jpeg) in cases.iter().enumerate() { + // the last case links nothing (a foreign HasExtendedXMP): its chunks stay visible as an unlinked + // extension, as before, and embedded_xmp still refuses it below + if index + 1 < cases.len() { + assert!(embedded(jpeg).xmp_extended.is_none(), "malformed extension case {index} was exposed"); + } + assert_eq!(extract(jpeg).title.as_deref(), Some("Standard title"), "standard XMP remains readable"); + assert!(embedded_xmp(&embedded(jpeg)).is_err(), "embedded_xmp must reject malformed extension case {index}"); + let error = embedded_xmp(&embedded(jpeg)).expect_err("invalid referenced extension is reported"); + assert_eq!(error.standard.as_deref().and_then(|xmp| xmp.metadata.title.as_deref()), Some("Standard title")); + } + } + + #[test] + fn standard_only_jpeg_xmp_still_extracts() { + let standard = xmp_document(r#"xmlns:dc="http://purl.org/dc/elements/1.1/" dc:title="Standard-only title""#); + let mut jpeg = vec![0xff, 0xd8]; + let mut packet = JPEG_XMP_SIG.to_vec(); + packet.extend_from_slice(standard.as_bytes()); + jpeg.extend(jpeg_segment(0xe1, &packet)); + assert_eq!(extract(&jpeg).title.as_deref(), Some("Standard-only title")); + } + + #[test] + fn embedded_xmp_rejects_malformed_xmp_but_extract_remains_best_effort() { + let mut jpeg = vec![0xff, 0xd8]; + let mut packet = JPEG_XMP_SIG.to_vec(); + packet.extend_from_slice(b" Result>, values: &BTreeMap) -> Metadata { let first = |k: &str| props.get(k).and_then(|v| v.first()).map(|s| s.trim().to_string()).filter(|s| !s.is_empty()); let num = |k: &str| first(k).and_then(|s| parse_number(&s)); let mut m = Metadata { @@ -1107,13 +1120,67 @@ pub fn parse_xmp_with(s: &str, extra: &[XmpNamespace]) -> Result Result { + let ext = parse_xmp_with(extended, extra)?; + let std = parse_xmp_with(standard, extra)?; + let mut properties = ext.properties; + properties.extend(std.properties); + let mut values = ext.values; + for (key, preferred) in std.values { + match values.get_mut(&key) { + Some(fallback) => *fallback = merge_xmp_value(&preferred, fallback), + None => { + values.insert(key, preferred); + } + } + } + properties.remove("xmpNote:HasExtendedXMP"); + values.remove("xmpNote:HasExtendedXMP"); + let metadata = project(&properties, &values); + Ok(XmpData { + metadata, + properties, + properties_truncated: std.properties_truncated || ext.properties_truncated, + values, + toolkit: std.toolkit, + names_location: std.names_location || ext.names_location, + }) +} + +/// `preferred` with what `fallback` adds: array items appended, struct fields merged. +fn merge_xmp_value(preferred: &XmpValue, fallback: &XmpValue) -> XmpValue { + match (preferred, fallback) { + (XmpValue::Array(preferred), XmpValue::Array(fallback)) => XmpValue::Array(preferred.iter().chain(fallback).cloned().collect()), + (XmpValue::Struct(preferred), XmpValue::Struct(fallback)) => { + let mut merged = fallback.clone(); + for (key, value) in preferred { + let combined = merged.get(key).map_or_else(|| value.clone(), |fallback| merge_xmp_value(value, fallback)); + merged.insert(key.clone(), combined); + } + XmpValue::Struct(merged) + } + _ => preferred.clone(), + } +} + +/// The GUID a standard packet's `xmpNote:HasExtendedXMP` names, upper-cased; `Ok(None)` without +/// one. Only the XMP Note namespace counts (a foreign `HasExtendedXMP` is not a link), and the +/// property must occur once, as 32 hexadecimal digits. +pub fn extended_xmp_reference(data: &XmpData) -> Result, String> { + let Some(values) = data.properties.get("xmpNote:HasExtendedXMP") else { return Ok(None) }; + let [value] = values.as_slice() else { + return Err("XMP HasExtendedXMP reference must occur exactly once".into()); + }; + if value.len() != 32 || !value.bytes().all(|b| b.is_ascii_hexdigit()) { + return Err("XMP HasExtendedXMP reference must be 32 hexadecimal characters".into()); + } + Ok(Some(value.to_ascii_uppercase())) } fn frac(v: f64) -> String { @@ -1189,9 +1256,13 @@ pub fn write_xmp_with(meta: &Metadata, with: &XmpWrite) -> String { x.push_str(&format!("\n", escape(xml_safe(with.toolkit)))); x.push_str(" \n"); x.push_str(" own, + "xmpNote" => None, + _ => Some(ns), + }) { x.push_str(&format!("\n xmlns:{p}=\"{}\"", escape(xml_safe(u)))); } x.push_str(">\n"); From 54b8aa8fb3ec36b2e3605f5c02ff45b9db1ffa00 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:06:29 -0500 Subject: [PATCH 05/10] craft-psd: whole .atn libraries with several sets, and their byte spans (#1047) A version-16 .atn file can hold several sets after one version header. - `atn::parse_sets` / `parse_sets_with_encoding` read every set to the end of the file (a malformed later set fails the whole result); `write_sets` writes them with one header. Archive-wide limits: `MAX_SETS`, `MAX_TOTAL_ACTIONS`, `MAX_TOTAL_ITEMS`, on top of the per-set and per-action ones. - `parse_sets_with_spans` returns, for each set, action and step, the byte ranges of its header fields and descriptor in the input, so a library can edit one field and keep every other byte as it was. - `parse` and `write` (one set) are unchanged. Ported from storytold/photocraft#11021 (its crates/psd part). Co-authored-by: yoriswind --- crates/psd/src/atn.rs | 299 +++++++++++++++++++++++++++++--- crates/psd/src/atn/sets.rs | 135 ++++++++++++++ crates/psd/tests/atn_sidecar.rs | 233 +++++++++++++++++++++++++ 3 files changed, 642 insertions(+), 25 deletions(-) create mode 100644 crates/psd/src/atn/sets.rs create mode 100644 crates/psd/tests/atn_sidecar.rs diff --git a/crates/psd/src/atn.rs b/crates/psd/src/atn.rs index 09a187686..32f3bce31 100644 --- a/crates/psd/src/atn.rs +++ b/crates/psd/src/atn.rs @@ -1,4 +1,4 @@ -//! Photoshop Actions files (`.atn`): one action set, its actions and their steps. +//! Photoshop Actions files (`.atn`): action sets, their actions and their steps. //! //! Implemented clean-room from Adobe's "Photoshop File Formats" specification (the *Actions* //! and *Descriptor structure* sections) and observation of files Photoshop writes. No code was @@ -6,7 +6,8 @@ //! code-unit count + UTF-16BE, usually NUL-terminated) unless noted. //! //! ```text -//! file := version u32 (12 or 16), set name UnicodeString, expanded u8, count u32, count × action +//! file := version u32 (12 or 16), set (version 16 may hold further sets until EOF) +//! set := name UnicodeString, expanded u8, count u32, count × action //! action := function key u16 (0 = none, n = Fn), shift u8, command u8, colour u16, //! name UnicodeString, expanded u8, count u32, count × item //! item := expanded u8, enabled u8, with dialog u8, dialog options u8, @@ -31,8 +32,9 @@ //! //! [`parse`] never panics: every count is bounds-checked against the bytes left, descriptor //! nesting is bounded, and allocations are capped. [`write`] always produces version 16, so -//! `parse(write(f)) == f` for files [`parse`] read, and `write(parse(b)) == b` for version 16 files -//! Photoshop wrote; a version 12 file's legacy descriptors keep their meaning, not their layout. +//! `parse(write(f)) == f` for sets [`parse`] read. The existing single-set API retains trailing-byte +//! warnings. [`parse_sets`] and [`write_sets`] preserve complete version-16 multi-set files with +//! one version header; a version-12 file's legacy descriptors keep their meaning, not their layout. use crate::codepage; use crate::descriptor::{Descriptor, UnicodeString}; @@ -40,6 +42,9 @@ use crate::error::{PsdError, Result}; use crate::io::{Reader, WriteExt, decode_legacy_name}; mod legacy; +mod sets; + +pub use sets::{MAX_SETS, MAX_TOTAL_ACTIONS, MAX_TOTAL_ITEMS, parse_sets, parse_sets_with_encoding, parse_sets_with_spans, write_sets}; /// The version written by this module (Photoshop 6 and every later release). pub const VERSION: u32 = 16; @@ -48,6 +53,70 @@ pub const MAX_ACTIONS: usize = 10_000; /// Most steps read from one action. pub const MAX_ITEMS: usize = 100_000; +/// A complete version16 archive and byte provenance into its original input. +/// Ranges address the caller's input, including original names/flags/descriptor encodings. +/// No source bytes are copied or rewritten by this parser. +#[derive(Debug, Clone, PartialEq)] +pub struct SpannedAtnArchive { + /// Parsed sets in physical file order. + pub sets: Vec, + /// Byte spans for the sets, in the same order. + pub spans: Vec, +} + +/// One physical set, including its original header fields and action occurrences. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct AtnSetSpan { + /// Half-open input range of the complete set, including its actions. + pub range: std::ops::Range, + /// Set header through the action count, before the first action. + pub header: std::ops::Range, + /// Length-prefixed Unicode set name bytes. + pub name: std::ops::Range, + /// Original expanded-flag byte. + pub expanded: std::ops::Range, + /// Original four-byte action count. + pub count: std::ops::Range, + /// Action spans in physical file order. + pub actions: Vec, +} + +/// One physical action. Header ranges let a library edit one field without normalizing others. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct AtnActionSpan { + /// Half-open input range of the complete action, including its steps. + pub range: std::ops::Range, + /// Action header through the step count, before the first step. + pub header: std::ops::Range, + /// Original function-key code bytes. + pub function_key: std::ops::Range, + /// Original Shift-modifier flag byte. + pub shift: std::ops::Range, + /// Original command-modifier flag byte. + pub command: std::ops::Range, + /// Original color-label code bytes. + pub color: std::ops::Range, + /// Length-prefixed Unicode action name bytes. + pub name: std::ops::Range, + /// Original expanded-flag byte. + pub expanded: std::ops::Range, + /// Original four-byte step count. + pub count: std::ops::Range, + /// Step spans in physical file order. + pub items: Vec, +} + +/// One original event, even when it maps to several native command steps. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct AtnItemSpan { + /// Half-open input range of the complete original step. + pub range: std::ops::Range, + /// Expanded, enabled, dialog, dialog-options bytes, in that order. + pub flags: std::ops::Range, + /// Descriptor body after its -1 flag; absent for a descriptor-free event. + pub descriptor: Option>, +} + /// A parsed action set. #[derive(Debug, Clone, PartialEq, Default)] #[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] @@ -217,7 +286,8 @@ fn write_ascii(out: &mut Vec, s: &[u8]) -> Result<()> { } /// Read one step with the descriptor dialect selected by the file version. -fn read_item(r: &mut Reader<'_>, legacy: bool) -> Result { +fn read_item(r: &mut Reader<'_>, legacy: bool, mut span: Option<&mut AtnItemSpan>) -> Result { + let start = r.pos(); let expanded = r.u8()? != 0; let enabled = r.u8()? != 0; let with_dialog = r.u8()? != 0; @@ -230,29 +300,67 @@ fn read_item(r: &mut Reader<'_>, legacy: bool) -> Result { let dictionary = read_ascii(r)?; let descriptor = match r.i32()? { 0 => None, - -1 => Some(if legacy { legacy::read(r)? } else { Descriptor::read(r)? }), + -1 => { + let start = r.pos(); + let descriptor = if legacy { legacy::read(r)? } else { Descriptor::read(r)? }; + if let Some(span) = span.as_deref_mut() { + span.descriptor = Some(start..r.pos()); + } + Some(descriptor) + } n => return Err(PsdError::invalid(format!("bad descriptor flag {n}"))), }; + if let Some(span) = span { + span.range = start..r.pos(); + span.flags = start..start.saturating_add(4); + } Ok(AtnItem { expanded, enabled, with_dialog, dialog_options, event, dictionary, descriptor }) } /// Read an action and its steps using the file's descriptor dialect. -fn read_action(r: &mut Reader<'_>, legacy: bool) -> Result { +fn read_action(r: &mut Reader<'_>, legacy: bool, budget: Option<&mut sets::Budget>, mut span: Option<&mut AtnActionSpan>) -> Result { + let start = r.pos(); let function_key = r.u16()?; + let shift_start = r.pos(); let shift = r.u8()? != 0; + let command_start = r.pos(); let command = r.u8()? != 0; + let color_start = r.pos(); let color = r.u16()?; + let name_start = r.pos(); let name = UnicodeString::read(r)?; + let expanded_start = r.pos(); let expanded = r.u8()? != 0; + let count_start = r.pos(); let n = r.u32()?; + if let Some(span) = span.as_deref_mut() { + span.header = start..r.pos(); + span.function_key = start..shift_start; + span.shift = shift_start..command_start; + span.command = command_start..color_start; + span.color = color_start..name_start; + span.name = name_start..expanded_start; + span.expanded = expanded_start..count_start; + span.count = count_start..r.pos(); + } // Each item needs at least its 4 flags, the event kind, two empty names and the descriptor flag. r.check_count(u64::from(n), 20)?; if n as usize > MAX_ITEMS { return Err(PsdError::LimitExceeded("too many steps in one action")); } + if let Some(budget) = budget { + budget.items(n as usize)?; + } let mut items = Vec::with_capacity((n as usize).min(1024)); for _ in 0..n { - items.push(read_item(r, legacy)?); + let mut item_span = span.as_ref().map(|_| AtnItemSpan::default()); + items.push(read_item(r, legacy, item_span.as_mut())?); + if let (Some(span), Some(item)) = (span.as_deref_mut(), item_span) { + span.items.push(item); + } + } + if let Some(span) = span { + span.range = start..r.pos(); } Ok(AtnAction { function_key, shift, command, color, name, expanded, items }) } @@ -278,33 +386,74 @@ pub fn parse_with_encoding(data: &[u8], encoding: Option<&str>) -> Result Result { let mut r = Reader::new(data); + let version = read_version(&mut r)?; + let mut file = read_set(&mut r, version == 12, None)?; + if !r.is_empty() { + file.warnings.push(format!("{} trailing bytes ignored", r.remaining())); + } + Ok(file) +} + +fn read_version(r: &mut Reader<'_>) -> Result { let version = r.u32()?; if !matches!(version, 12 | VERSION) { return Err(PsdError::Unsupported(format!("actions file version {version} (supported versions: 12 and 16)"))); } - let name = UnicodeString::read(&mut r)?; + Ok(version) +} + +fn read_set(r: &mut Reader<'_>, legacy: bool, budget: Option<&mut sets::Budget>) -> Result { + read_set_with_spans(r, legacy, budget, None) +} + +fn read_set_with_spans( + r: &mut Reader<'_>, + legacy: bool, + mut budget: Option<&mut sets::Budget>, + mut span: Option<&mut AtnSetSpan>, +) -> Result { + let start = r.pos(); + let name = UnicodeString::read(r)?; + let expanded_start = r.pos(); let expanded = r.u8()? != 0; + let count_start = r.pos(); let n = r.u32()?; + if let Some(span) = span.as_deref_mut() { + span.header = start..r.pos(); + span.name = start..expanded_start; + span.expanded = expanded_start..count_start; + span.count = count_start..r.pos(); + } // Each action needs at least its key, modifiers, colour, an empty name, the flag and a count. r.check_count(u64::from(n), 15)?; if n as usize > MAX_ACTIONS { return Err(PsdError::LimitExceeded("too many actions in one set")); } + if let Some(budget) = budget.as_deref_mut() { + budget.actions(n as usize)?; + } let mut actions = Vec::with_capacity((n as usize).min(256)); for _ in 0..n { - actions.push(read_action(&mut r, version == 12)?); + let mut action_span = span.as_ref().map(|_| AtnActionSpan::default()); + actions.push(read_action(r, legacy, budget.as_deref_mut(), action_span.as_mut())?); + if let (Some(span), Some(action)) = (span.as_deref_mut(), action_span) { + span.actions.push(action); + } } - let mut warnings = Vec::new(); - if !r.is_empty() { - warnings.push(format!("{} trailing bytes ignored", r.remaining())); + if let Some(span) = span { + span.range = start..r.pos(); } - Ok(AtnFile { name, expanded, actions, warnings, encoding: None }) + Ok(AtnFile { name, expanded, actions, warnings: Vec::new(), encoding: None }) } /// The encoding of `f`'s 8-bit names that are not ASCII or UTF-8, or `None` if no name shown /// from them needs one; adds a warning when the guess is uncertain. fn name_encoding(f: &AtnFile, warnings: &mut Vec) -> Option<&'static encoding_rs::Encoding> { - let items = || f.actions.iter().flat_map(|a| &a.items); + name_encoding_sets(std::slice::from_ref(f), warnings) +} + +fn name_encoding_sets(files: &[AtnFile], warnings: &mut Vec) -> Option<&'static encoding_rs::Encoding> { + let items = || files.iter().flat_map(|file| &file.actions).flat_map(|action| &action.items); let legacy = |b: &[u8]| !b.is_ascii(); // Steps that store their name as Unicode too: the encoding that turns one into the other. let pairs: Vec<(&[u8], String)> = items() @@ -333,13 +482,26 @@ fn name_encoding(f: &AtnFile, warnings: &mut Vec) -> Option<&'static enc return None; } // The set, action and class names are UTF-16: their script points at the writer's locale. - let mut hint = f.name.to_string_lossy(); - for name in f.actions.iter().map(|a| a.name.to_string_lossy()).chain(items().filter_map(class_name)) { - if hint.len() > codepage::GUESS_SAMPLE { - break; + let mut hint = String::new(); + for (index, file) in files.iter().enumerate() { + if index == 0 { + hint = file.name.to_string_lossy(); + } else if hint.len() <= codepage::GUESS_SAMPLE { + hint.push('\n'); + hint.push_str(&file.name.to_string_lossy()); + } + for name in file + .actions + .iter() + .map(|action| action.name.to_string_lossy()) + .chain(file.actions.iter().flat_map(|action| &action.items).filter_map(class_name)) + { + if hint.len() > codepage::GUESS_SAMPLE { + break; + } + hint.push('\n'); + hint.push_str(&name); } - hint.push('\n'); - hint.push_str(&name); } let guess = codepage::guess(&orphans, &hint); if guess.uncertain { @@ -385,7 +547,12 @@ fn count(n: usize, what: &'static str) -> Result { pub fn write(f: &AtnFile) -> Result> { let mut out = Vec::new(); out.put_u32(VERSION); - f.name.write(&mut out); + write_set(&mut out, f)?; + Ok(out) +} + +fn write_set(out: &mut Vec, f: &AtnFile) -> Result<()> { + f.name.write(out); out.put_u8(u8::from(f.expanded)); out.put_u32(count(f.actions.len(), "too many actions")?); for a in &f.actions { @@ -393,14 +560,14 @@ pub fn write(f: &AtnFile) -> Result> { out.put_u8(u8::from(a.shift)); out.put_u8(u8::from(a.command)); out.put_u16(a.color); - a.name.write(&mut out); + a.name.write(out); out.put_u8(u8::from(a.expanded)); out.put_u32(count(a.items.len(), "too many steps")?); for item in &a.items { - write_item(&mut out, item)?; + write_item(out, item)?; } } - Ok(out) + Ok(()) } #[cfg(test)] @@ -863,4 +1030,86 @@ mod tests { let _ = parse(&bytes); } } + + /// Independent wire input: duplicate raw UTF16 names, noncanonical true bytes, and a + /// descriptor boolean whose semantic value cannot preserve its original byte by itself. + fn spanned_wire_archive() -> Vec { + fn ascii(out: &mut Vec, bytes: &[u8]) { + out.extend_from_slice(&(bytes.len() as u32).to_be_bytes()); + out.extend_from_slice(bytes); + } + let mut out = 16u32.to_be_bytes().to_vec(); + let raw_name = [0, 0, 0, 2, 0xd8, 0, 0, 0]; // unpaired high surrogate, then NUL + for _ in 0..2 { + out.extend_from_slice(&raw_name); + out.push(2); // expanded=true, retaining its original byte matters + out.extend_from_slice(&1u32.to_be_bytes()); + out.extend_from_slice(&[0, 3, 2, 0, 0, 12]); // F3, Shift byte2, no Command, colour12 + out.extend_from_slice(&raw_name); + out.push(2); + out.extend_from_slice(&2u32.to_be_bytes()); + out.extend_from_slice(&[2, 2, 0, 3]); + out.extend_from_slice(b"longFltn"); + ascii(&mut out, b"Flatten"); + out.extend_from_slice(&0i32.to_be_bytes()); + out.extend_from_slice(&[0, 1, 2, 2]); + out.extend_from_slice(b"TEXT"); + ascii(&mut out, b"futureAction"); + ascii(&mut out, b"Future"); + out.extend_from_slice(&(-1i32).to_be_bytes()); + out.extend_from_slice(&0u32.to_be_bytes()); // empty descriptor Unicode name + out.extend_from_slice(&0u32.to_be_bytes()); + out.extend_from_slice(b"null"); + out.extend_from_slice(&1u32.to_be_bytes()); + out.extend_from_slice(&0u32.to_be_bytes()); + out.extend_from_slice(b"flagbool"); + out.push(2); + } + out + } + + #[test] + fn archive_spans_retain_physical_occurrences_and_original_wire_fields() { + let bytes = spanned_wire_archive(); + let archive = parse_sets_with_spans(&bytes).unwrap(); + assert_eq!(archive.sets.len(), 2); + assert_eq!(archive.spans.len(), 2); + assert_eq!(archive.spans[0].range.start, 4); + assert_eq!(archive.spans[0].range.end, archive.spans[1].range.start); + assert_eq!(archive.spans[1].range.end, bytes.len()); + let mut untouched = bytes[..4].to_vec(); + for (file, set) in archive.sets.iter().zip(&archive.spans) { + assert_eq!(file.name.0, [0xd800, 0]); + assert_eq!(bytes[set.expanded.clone()], [2]); + assert_eq!(bytes[set.count.clone()], 1u32.to_be_bytes()); + assert_eq!(set.header.end, set.actions[0].range.start); + let action = &set.actions[0]; + assert_eq!(bytes[action.function_key.clone()], [0, 3]); + assert_eq!(bytes[action.shift.clone()], [2]); + assert_eq!(bytes[action.command.clone()], [0]); + assert_eq!(bytes[action.color.clone()], [0, 12]); + assert_eq!(bytes[action.expanded.clone()], [2]); + assert_eq!(bytes[action.count.clone()], 2u32.to_be_bytes()); + assert_eq!(action.header.end, action.items[0].range.start); + assert_eq!(bytes[action.items[0].flags.clone()], [2, 2, 0, 3]); + assert_eq!(bytes[action.items[1].flags.clone()], [0, 1, 2, 2]); + assert_eq!(action.items[0].range.end, action.items[1].range.start); + assert_eq!(action.items[1].range.end, action.range.end); + assert_eq!(action.range.end, set.range.end); + assert_eq!(bytes[action.items[1].range.clone()].last(), Some(&2)); + untouched.extend_from_slice(&bytes[set.range.clone()]); + } + assert_eq!(untouched, bytes, "no semantic writer generates the expected raw bytes"); + assert_eq!(parse_sets(&bytes).unwrap(), archive.sets, "ordinary parsing keeps its semantic API"); + assert_ne!(write_sets(&archive.sets).unwrap(), bytes, "typed boolean rewriting is not a raw library round trip"); + } + + #[test] + fn archive_spans_reject_partial_later_records_and_legacy_layout() { + let mut bytes = spanned_wire_archive(); + bytes.pop(); // the second set's descriptor value is incomplete + assert!(parse_sets_with_spans(&bytes).is_err(), "no partial archive is admitted"); + assert!(parse_sets_with_spans(&16u32.to_be_bytes()).is_err(), "a version prefix is not an invented empty ATN"); + assert!(matches!(parse_sets_with_spans(&12u32.to_be_bytes()), Err(PsdError::Unsupported(_)))); + } } diff --git a/crates/psd/src/atn/sets.rs b/crates/psd/src/atn/sets.rs new file mode 100644 index 000000000..974b6b245 --- /dev/null +++ b/crates/psd/src/atn/sets.rs @@ -0,0 +1,135 @@ +//! Whole-file version-16 set streams. The original single-set/legacy API remains compatible. +use super::{ + AtnFile, AtnSetSpan, MAX_ACTIONS, MAX_ITEMS, SpannedAtnArchive, VERSION, codepage, name_encoding_sets, read_set_with_spans, read_version, + write_set, +}; +use crate::{ + error::{PsdError, Result}, + io::{Reader, WriteExt}, +}; + +/// Whole-archive resource ceilings, not Photoshop format maxima. Per-set/action ceilings still apply. +pub const MAX_SETS: usize = MAX_ACTIONS; +/// Maximum action headers across the complete archive, including any metadata sidecar. +pub const MAX_TOTAL_ACTIONS: usize = MAX_ACTIONS; +/// Maximum steps across the complete archive, including any metadata sidecar. +pub const MAX_TOTAL_ITEMS: usize = MAX_ITEMS; + +#[derive(Default)] +pub(super) struct Budget { + actions: usize, + items: usize, +} + +impl Budget { + pub(super) fn actions(&mut self, count: usize) -> Result<()> { + if count > MAX_ACTIONS { + return Err(PsdError::LimitExceeded("too many actions in one set")); + } + self.actions = self + .actions + .checked_add(count) + .filter(|sum| *sum <= MAX_TOTAL_ACTIONS) + .ok_or(PsdError::LimitExceeded("too many actions in actions archive"))?; + Ok(()) + } + + pub(super) fn items(&mut self, count: usize) -> Result<()> { + if count > MAX_ITEMS { + return Err(PsdError::LimitExceeded("too many steps in one action")); + } + self.items = self + .items + .checked_add(count) + .filter(|sum| *sum <= MAX_TOTAL_ITEMS) + .ok_or(PsdError::LimitExceeded("too many steps in actions archive"))?; + Ok(()) + } +} + +/// Read every set in a version-16 file, in order, with one version prefix and strict EOF. +/// A malformed later set fails the whole result. Version12 retains the existing one-set/tail contract. +pub fn parse_sets(data: &[u8]) -> Result> { + parse_sets_with_encoding(data, None) +} + +/// [`parse_sets`] with one legacy name encoding for the complete file. Automatic detection also +/// uses the complete stream; its diagnostic is attached once to the first set. +pub fn parse_sets_with_encoding(data: &[u8], encoding: Option<&str>) -> Result> { + parse_archive(data, encoding, false).map(|archive| archive.sets) +} + +/// Read a strict complete version16 archive, retaining physical record/field ranges. +/// Legacy version12 conversion is deliberately excluded: its descriptor dialect cannot be +/// spliced unchanged into a version16 library. Existing import APIs still accept it. +/// A header without any real set remains invalid; an empty logical library needs a valid carrier. +pub fn parse_sets_with_spans(data: &[u8]) -> Result { + parse_archive(data, None, true) +} + +fn parse_archive(data: &[u8], encoding: Option<&str>, capture: bool) -> Result { + let forced = encoding.map(codepage::resolve_label).transpose()?; + let mut reader = Reader::new(data); + let version = read_version(&mut reader)?; + let legacy = version == 12; + if capture && legacy { + return Err(PsdError::Unsupported("raw Actions library provenance requires version16".into())); + } + let mut files = Vec::new(); + let mut spans = Vec::new(); + let mut budget = Budget::default(); + loop { + if files.len() >= MAX_SETS { + return Err(PsdError::LimitExceeded("too many sets in actions archive")); + } + let before = reader.remaining(); + let mut span = capture.then(AtnSetSpan::default); + let mut file = read_set_with_spans(&mut reader, legacy, Some(&mut budget), span.as_mut())?; + if let Some(span) = span { + spans.push(span); + } + if legacy && !reader.is_empty() { + file.warnings.push(format!("{} trailing bytes ignored", reader.remaining())); + } + if reader.remaining() >= before { + return Err(PsdError::invalid("actions set made no progress")); + } + files.push(file); + if legacy || reader.is_empty() { + break; + } + } + let mut warnings = Vec::new(); + let encoding = forced.or_else(|| name_encoding_sets(&files, &mut warnings)).map(|encoding| encoding.name().to_string()); + for file in &mut files { + file.encoding = encoding.clone(); + } + if let Some(first) = files.first_mut() { + first.warnings.extend(warnings); + } + Ok(SpannedAtnArchive { sets: files, spans }) +} + +/// Write all sets under one version16 prefix. Validate aggregate counts before allocating output. +/// At least one set is required; an empty set is valid. Stored names/descriptors stay verbatim. +pub fn write_sets(files: &[AtnFile]) -> Result> { + if files.is_empty() { + return Err(PsdError::invalid("actions archive must contain at least one set")); + } + if files.len() > MAX_SETS { + return Err(PsdError::LimitExceeded("too many sets in actions archive")); + } + let mut budget = Budget::default(); + for file in files { + budget.actions(file.actions.len())?; + for action in &file.actions { + budget.items(action.items.len())?; + } + } + let mut out = Vec::new(); + out.put_u32(VERSION); + for file in files { + write_set(&mut out, file)?; + } + Ok(out) +} diff --git a/crates/psd/tests/atn_sidecar.rs b/crates/psd/tests/atn_sidecar.rs new file mode 100644 index 000000000..101f352d6 --- /dev/null +++ b/crates/psd/tests/atn_sidecar.rs @@ -0,0 +1,233 @@ +//! Independently specified version16 wire streams: one header, consecutive set bodies. +use craft_psd::{atn, descriptor::Value, error::PsdError}; +use proptest::prelude::*; + +fn u32be(out: &mut Vec, value: u32) { + out.extend_from_slice(&value.to_be_bytes()); +} +fn unicode(out: &mut Vec, units: &[u16]) { + u32be(out, units.len() as u32); + for unit in units { + out.extend_from_slice(&unit.to_be_bytes()); + } +} +fn text(out: &mut Vec, value: &str) { + unicode(out, &value.encode_utf16().collect::>()); +} +fn id(out: &mut Vec, key: &str) { + if key.len() == 4 { + u32be(out, 0); + } else { + u32be(out, key.len() as u32); + } + out.extend_from_slice(key.as_bytes()); +} +fn descriptor(items: &[(&str, &[u8; 4], Vec)]) -> Vec { + let mut out = Vec::new(); + unicode(&mut out, &[]); + id(&mut out, "null"); + u32be(&mut out, items.len() as u32); + for (key, kind, value) in items { + id(&mut out, key); + out.extend_from_slice(*kind); + out.extend(value); + } + out +} +fn list(descriptors: &[Vec]) -> Vec { + let mut out = Vec::new(); + u32be(&mut out, descriptors.len() as u32); + for descriptor in descriptors { + out.extend_from_slice(b"Objc"); + out.extend(descriptor); + } + out +} +fn string(value: &str) -> Vec { + let mut out = Vec::new(); + text(&mut out, value); + out +} +fn set_header(out: &mut Vec, name: &[u16], actions: u32) { + unicode(out, name); + out.push(0); + u32be(out, actions); +} +fn action_header(out: &mut Vec, name: &str, items: u32) { + out.extend_from_slice(&[0, 0, 0, 0, 0, 0]); // Fn, Shift, Control and colour. + text(out, name); + out.push(0); + u32be(out, items); +} +fn bare_item(out: &mut Vec) { + out.extend_from_slice(&[0, 1, 0, 0]); + out.extend_from_slice(b"longInvr"); + u32be(out, 0); + u32be(out, 0); +} + +fn fixture() -> (Vec, usize, Vec, Vec) { + let metadata = descriptor(&[ + ("actionDescriptionKey", b"TEXT", string("Description\0")), + ("actionThumbnailTypeKey", b"long", 3i32.to_be_bytes().to_vec()), + ("actionIconKey", b"TEXT", string("future-icon\0")), + ("actionPreviewKey", b"bool", vec![0]), + ("actionGenerativeKey", b"bool", vec![0]), + ("actionPreconditionVal", b"long", (-1i32).to_be_bytes().to_vec()), + ("futureDouble", b"doub", 0x7ff8_0000_0000_0042u64.to_be_bytes().to_vec()), + ]); + let first = descriptor(&[("setColorIndex", b"long", 7i32.to_be_bytes().to_vec()), ("actions", b"VlLs", list(&[metadata]))]); + let empty = descriptor(&[("setColorIndex", b"long", 19i32.to_be_bytes().to_vec()), ("actions", b"VlLs", list(&[]))]); + let root = descriptor(&[("sets", b"VlLs", list(&[first, empty])), ("futureRaw", b"tdta", vec![0, 0, 0, 3, 0, 255, 7])]); + let mut payload = 16u32.to_be_bytes().to_vec(); + payload.extend(root); + let mut raw = Vec::new(); + u32be(&mut raw, payload.len() as u32); + raw.extend(&payload); + let outer = descriptor(&[("Msge", b"TEXT", string("Original note\0")), ("Cntn", b"bool", vec![1]), ("modernizedActionData", b"tdta", raw)]); + let mut bytes = 16u32.to_be_bytes().to_vec(); + set_header(&mut bytes, &"Real\0".encode_utf16().collect::>(), 1); + action_header(&mut bytes, "Invert\0", 1); + bare_item(&mut bytes); + let first_end = bytes.len(); + set_header(&mut bytes, &[0xd800, 0], 0); // Empty real set and unpaired UTF16 remain exact. + set_header(&mut bytes, &"Action sidecar\0".encode_utf16().collect::>(), 1); + action_header(&mut bytes, "Play for details\0", 1); + bytes.extend_from_slice(&[0, 1, 1, 2]); + bytes.extend_from_slice(b"TEXT"); + u32be(&mut bytes, 4); + bytes.extend_from_slice(b"stop"); + u32be(&mut bytes, 4); + bytes.extend_from_slice(b"Stop"); + bytes.extend_from_slice(&(-1i32).to_be_bytes()); + bytes.extend(&outer); + (bytes, first_end, outer, payload) +} + +#[test] +fn one_version_and_three_set_bodies_preserve_raw_sidecar_descriptor_separately() { + let (bytes, first_end, outer, payload) = fixture(); + assert_eq!(bytes.get(first_end..first_end + 4).unwrap(), &2u32.to_be_bytes()); + let sets = atn::parse_sets(&bytes).unwrap(); + assert_eq!(sets.len(), 3); + assert_eq!(sets[0].name.to_string_lossy(), "Real"); + assert_eq!(sets[1].name.0, vec![0xd800, 0]); + assert!(sets[1].actions.is_empty()); + let sidecar = sets[2].actions[0].items[0].descriptor.as_ref().unwrap(); + assert_eq!(sidecar.to_bytes(), outer, "descriptor bytes have their own independent oracle"); + assert_eq!(sidecar.get("modernizedActionData"), Some(&Value::RawData(payload))); + assert_eq!(atn::write_sets(&sets).unwrap(), bytes); + let single = atn::parse(&bytes).unwrap(); + assert_eq!(single.name, sets[0].name); + assert_eq!(single.warnings, [format!("{} trailing bytes ignored", bytes.len() - first_end)]); +} + +#[test] +fn every_truncated_later_set_fails_without_returning_partial_sets() { + let (bytes, first_end, _, _) = fixture(); + // The first complete set alone is valid. The second complete empty set is another valid prefix. + let second_end = first_end + 13; + for cut in first_end + 1..bytes.len() { + if cut != second_end { + assert!(atn::parse_sets(&bytes[..cut]).is_err(), "cut {cut}"); + } + } + assert_eq!(atn::parse_sets(&bytes[..first_end]).unwrap().len(), 1); + assert_eq!(atn::parse_sets(&bytes[..second_end]).unwrap().len(), 2); + let mut repeated = bytes[..first_end].to_vec(); + repeated.extend_from_slice(&16u32.to_be_bytes()); + set_header(&mut repeated, &[], 0); + assert!(atn::parse_sets(&repeated).is_err(), "a repeated version is not the measured set boundary"); + assert!(atn::parse_sets(&16u32.to_be_bytes()).is_err()); + assert!(atn::write_sets(&[]).is_err()); +} + +#[test] +fn forced_encoding_applies_to_all_sets_and_legacy_tails_remain_compatible() { + let (bytes, _, _, _) = fixture(); + let sets = atn::parse_sets_with_encoding(&bytes, Some("shift_jis")).unwrap(); + assert!(sets.iter().all(|set| set.encoding.as_deref() == Some("Shift_JIS"))); + assert_eq!(atn::write_sets(&sets).unwrap(), bytes); + assert!(atn::parse_sets_with_encoding(&bytes, Some("utf-16")).is_err()); + let mut legacy = 12u32.to_be_bytes().to_vec(); + set_header(&mut legacy, &[], 0); + legacy.extend_from_slice(&[99, 1, 4]); + let old = atn::parse(&legacy).unwrap(); + assert_eq!(atn::parse_sets(&legacy).unwrap(), vec![old.clone()]); + assert_eq!(old.warnings, ["3 trailing bytes ignored"]); +} + +#[test] +fn whole_archive_set_and_action_budgets_are_checked_across_boundaries() { + let mut bytes = 16u32.to_be_bytes().to_vec(); + for _ in 0..atn::MAX_SETS { + set_header(&mut bytes, &[], 0); + } + let sets = atn::parse_sets(&bytes).unwrap(); + assert_eq!(sets.len(), atn::MAX_SETS); + assert_eq!(atn::write_sets(&sets).unwrap(), bytes); + set_header(&mut bytes, &[], 0); + assert!(matches!(atn::parse_sets(&bytes), Err(PsdError::LimitExceeded(_)))); + let mut excess = sets; + excess.push(Default::default()); + assert!(matches!(atn::write_sets(&excess), Err(PsdError::LimitExceeded(_)))); + + let mut bytes = 16u32.to_be_bytes().to_vec(); + for _ in 0..2 { + set_header(&mut bytes, &[], (atn::MAX_TOTAL_ACTIONS / 2) as u32); + for _ in 0..atn::MAX_TOTAL_ACTIONS / 2 { + action_header(&mut bytes, "", 0); + } + } + let sets = atn::parse_sets(&bytes).unwrap(); + assert_eq!(sets.iter().map(|set| set.actions.len()).sum::(), atn::MAX_TOTAL_ACTIONS); + assert_eq!(atn::write_sets(&sets).unwrap(), bytes); + set_header(&mut bytes, &[], 1); + action_header(&mut bytes, "", 0); + assert!(matches!(atn::parse_sets(&bytes), Err(PsdError::LimitExceeded(_)))); +} + +#[test] +fn aggregate_steps_are_bounded_before_a_later_action_allocates() { + let mut bytes = 16u32.to_be_bytes().to_vec(); + set_header(&mut bytes, &[], 2); + for _ in 0..2 { + action_header(&mut bytes, "", (atn::MAX_TOTAL_ITEMS / 2) as u32); + for _ in 0..atn::MAX_TOTAL_ITEMS / 2 { + bare_item(&mut bytes); + } + } + let sets = atn::parse_sets(&bytes).unwrap(); + assert_eq!(atn::write_sets(&sets).unwrap(), bytes); + set_header(&mut bytes, &[], 1); + action_header(&mut bytes, "", 1); + bare_item(&mut bytes); + assert!(matches!(atn::parse_sets(&bytes), Err(PsdError::LimitExceeded(_)))); + let mut excess = sets; + excess[0].actions[0].items.push(Default::default()); + assert!(matches!(atn::write_sets(&excess), Err(PsdError::LimitExceeded(_)))); +} + +proptest! { + #![proptest_config(ProptestConfig::with_cases(64))] + + #[test] + fn arbitrary_streams_reject_or_reach_a_stable_writer(data in prop::collection::vec(any::(), 0..2048)) { + if let Ok(files) = atn::parse_sets(&data) { + let encoded = atn::write_sets(&files).unwrap(); + let again = atn::parse_sets(&encoded).unwrap(); + prop_assert_eq!(atn::write_sets(&again).unwrap(), encoded); + } + } + + #[test] + fn mutated_independent_multiset_stream_rejects_or_reaches_a_stable_writer(index in any::(), value in any::()) { + let (mut bytes, _, _, _) = fixture(); + let at = index % bytes.len(); bytes[at] = value; + if let Ok(files) = atn::parse_sets(&bytes) { + let encoded = atn::write_sets(&files).unwrap(); + let again = atn::parse_sets(&encoded).unwrap(); + prop_assert_eq!(atn::write_sets(&again).unwrap(), encoded); + } + } +} From 7b936bf293566856c7de351c5038a2731f197c94 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:07:34 -0500 Subject: [PATCH 06/10] craft-xlsx: typed worksheet features (views, print, validation, conditional formats, hyperlinks, comments) (#1048) * craft-xlsx: read parts with canonical prefixes so kept trees can be written back The part layer read every part by local name, so extLst children (x14:, xr:, xm:), mc:AlternateContent and r:id lost their namespaces and could not be written back. Parts are now read with one fixed prefix per namespace (the main namespace unprefixed), mc:Ignorable and mc:Choice/@Requires are rewritten to match, and ns::declare prepares a kept tree for writing. craft-xml: a namespace listed with the empty prefix reads unprefixed. * craft-xlsx: write SpreadsheetML workbooks (Document read and write) Document holds a whole workbook: the workbook part, shared strings, styles, each worksheet's rows and the rest of its part, and every other part kept as bytes. write regenerates the workbook part, worksheets, shared strings and styles from the model, keeps unknown children in place (schema order for regenerated ones), keeps part names and relationship ids, and writes kept parts unchanged. Writers: Styles::to_xml and to_el for each table entry, StringItem::to_el (phonetic runs checked as on reading), SheetPart::to_xml with rows streamed, Workbook::to_el, XlsxKind. Row and DefinedName keep their other attributes; CellStyle reads iLevel. craft-opc: LazyPackage::canonical gives a part's stored name. * craft-xlsx: typed worksheet features and comments, written back SheetPart::features lifts sheet views, print setup, data validation, conditional formats, hyperlinks and the legacy drawing into types that mirror their elements (optional attributes, unnamed ones kept) and writes them in schema order. Comments come from the comments part, note shapes from the VML drawing; a sheet that gains comments without a VML drawing gets one. A hyperlink to a new URL gets its own relationship. craft-sheet: Range derives Default. --- crates/sheet/src/a1.rs | 5 +- crates/xlsx/README.md | 9 + crates/xlsx/src/document.rs | 80 +++- crates/xlsx/src/features.rs | 919 ++++++++++++++++++++++++++++++++++++ crates/xlsx/src/lib.rs | 5 + crates/xlsx/src/package.rs | 13 + crates/xlsx/src/sheet.rs | 20 +- crates/xlsx/src/write.rs | 21 +- crates/xlsx/tests/write.rs | 109 ++++- 9 files changed, 1153 insertions(+), 28 deletions(-) create mode 100644 crates/xlsx/src/features.rs diff --git a/crates/sheet/src/a1.rs b/crates/sheet/src/a1.rs index be5943661..95586da9e 100644 --- a/crates/sheet/src/a1.rs +++ b/crates/sheet/src/a1.rs @@ -66,8 +66,9 @@ impl std::fmt::Display for CellRef { } } -/// A rectangle of cells, both corners included; `first` is the top left one. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +/// A rectangle of cells, both corners included; `first` is the top left one. The default is +/// `A1`. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] pub struct Range { pub first: CellRef, pub last: CellRef, diff --git a/crates/xlsx/README.md b/crates/xlsx/README.md index 8d831eec2..953a7ee5a 100644 --- a/crates/xlsx/README.md +++ b/crates/xlsx/README.md @@ -35,6 +35,15 @@ from those fields, keeps the rest of each part's tree in place, and writes the k were. Part names and relationship ids are kept, so a file read and written without changes points where it did, and writing it twice gives the same bytes. The calculation chain is dropped (Excel rebuilds it). `XlsxKind` is the workbook part's content type (`.xlsx`, `.xlsm`, `.xltx`, `.xltm`). +Typed worksheet features (`SheetPart::features`), each mirroring its element with optional +attributes and the unnamed ones kept: sheet views (panes, selections), print setup (options, +margins, page setup, header and footer, page breaks), data validation (also read from the `x14` +extension), conditional formats (colour scales, data bars, icon sets, rule extensions kept), +hyperlinks (with their URLs), the legacy drawing. Comments (`Comments`, from the comments part) +and the note shapes of the VML drawing (`NoteShape`). A sheet that gains comments and has no VML +drawing gets one (`notes_vml`, in a shape block no kept drawing uses); an existing VML drawing is +kept as read. Extensions in `extLst` are carried through unchanged. + The parts this crate writes use the transitional namespaces; a strict file's kept parts stay as they were. A binary workbook is not written. diff --git a/crates/xlsx/src/document.rs b/crates/xlsx/src/document.rs index 25b75daeb..859630301 100644 --- a/crates/xlsx/src/document.rs +++ b/crates/xlsx/src/document.rs @@ -3,6 +3,7 @@ use craft_opc::{ContentTypes, PackageOut, RT_BASE, Rel, RelOut, Rels, RelsOut, relative, rels_path}; +use crate::features::{Comments, NoteShape, notes_vml}; use crate::sheet::{CellValue, Row, SheetPart}; use crate::strings::StringItem; use crate::styles::Styles; @@ -12,6 +13,8 @@ use crate::{Error, Limits, Package}; const CT_WORKSHEET: &str = "application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"; const CT_STRINGS: &str = "application/vnd.openxmlformats-officedocument.spreadsheetml.sharedStrings+xml"; const CT_STYLES: &str = "application/vnd.openxmlformats-officedocument.spreadsheetml.styles+xml"; +const CT_COMMENTS: &str = "application/vnd.openxmlformats-officedocument.spreadsheetml.comments+xml"; +const CT_VML: &str = "application/vnd.openxmlformats-officedocument.vmlDrawing"; /// The relationship type namespace of strict files: kept relationships are written with the /// transitional one, as the parts this crate writes are. const RT_STRICT: &str = "http://purl.oclc.org/ooxml/officeDocument/relationships/"; @@ -64,11 +67,16 @@ impl XlsxKind { } } -/// One worksheet: its rows and the rest of its part. +/// One worksheet: its rows, the rest of its part, and its comments. #[derive(Clone, Debug, Default)] pub struct Worksheet { pub rows: Vec, pub part: SheetPart, + /// The comments part, written from here. `None` removes it. + pub comments: Option, + /// How the comments are shown, from the legacy VML drawing. The drawing itself is kept as + /// read; these are used only to draw the comments of a sheet that has no VML drawing. + pub notes: Vec, } /// A part carried through unchanged: drawings, charts, the theme, document properties, a VBA @@ -131,7 +139,11 @@ impl Document { rows.push(row); Ok(()) }) { - Ok(part) => worksheets.push(Some(Worksheet { rows, part })), + Ok(part) => { + let comments = p.comments(&part)?; + let notes = if comments.is_some() { p.note_shapes(&part)? } else { Vec::new() }; + worksheets.push(Some(Worksheet { rows, part, comments, notes })); + } Err(Error::TooLarge(m)) => return Err(Error::TooLarge(m)), Err(e) => { p.warn(format!("sheet \"{}\" kept unread: {e}", entry.name)); @@ -144,6 +156,9 @@ impl Document { for ws in worksheets.iter().flatten() { made.push(ws.part.part.clone()); made.push(rels_path(&ws.part.part)); + if ws.comments.is_some() { + made.extend(ws.part.rels.first("comments").filter(|r| !r.external).map(|r| r.target.clone())); + } } made.extend(workbook_rels.list.iter().filter(|r| !r.external && regenerated(r)).map(|r| r.target.clone())); let opc = p.opc(); @@ -223,11 +238,9 @@ impl Document { package.add("officeDocument", book_part, false); out.rels("", &package); } + let mut blocks = vml_blocks(&self.parts); for (name, ws) in &sheet_parts { - out.xml_part(name, CT_WORKSHEET, ws.part.to_xml(&ws.rows)); - let mut sheet_rels = RelsOut::default(); - sheet_rels.items.extend(ws.part.rels.list.iter().map(kept)); - out.rels(name, &sheet_rels); + write_sheet(&mut out, name, ws, &mut taken, &mut blocks); } if let Some(name) = &strings { let count: usize = @@ -252,6 +265,61 @@ impl Document { } } +/// A worksheet, its relationships, and its comments with a VML drawing to show them when it has +/// none. A hyperlink to a URL its relationship does not lead to gets a relationship of its own. +fn write_sheet(out: &mut PackageOut, name: &str, ws: &Worksheet, taken: &mut Vec, blocks: &mut Vec) { + let mut part = ws.part.clone(); + let mut rels = RelsOut::default(); + rels.items.extend(ws.part.rels.list.iter().filter(|r| ws.comments.is_some() || r.kind() != "comments").map(kept)); + for h in &mut part.features.hyperlinks { + let Some(url) = &h.url else { continue }; + let ok = h.rel_id.as_deref().and_then(|id| ws.part.rels.by_id(id)).is_some_and(|r| r.external && r.target == *url); + if !ok { + let id = fresh_id(&rels); + rels.items.push(RelOut { id: id.clone(), ty: format!("{RT_BASE}hyperlink"), target: url.clone(), external: true }); + h.rel_id = Some(id); + } + } + if let Some(c) = &ws.comments { + let target = match ws.part.rels.first("comments").filter(|r| !r.external) { + Some(r) => r.target.clone(), + None => { + let target = fresh(taken, |n| format!("xl/comments{n}.xml")); + let id = fresh_id(&rels); + rels.items.push(RelOut { id, ty: format!("{RT_BASE}comments"), target: relative(name, &target), external: false }); + target + } + }; + taken.push(target.to_ascii_lowercase()); + out.xml_part(&target, CT_COMMENTS, c.to_xml()); + if !rels.items.iter().any(|r| r.ty.ends_with("/vmlDrawing")) && !c.comments.is_empty() { + let block = (1..).find(|b| !blocks.contains(b)).unwrap_or(1); + blocks.push(block); + let vml = fresh(taken, |n| format!("xl/drawings/vmlDrawing{n}.vml")); + taken.push(vml.to_ascii_lowercase()); + let id = fresh_id(&rels); + rels.items.push(RelOut { id: id.clone(), ty: format!("{RT_BASE}vmlDrawing"), target: relative(name, &vml), external: false }); + part.features.legacy_drawing = Some(id); + out.binary(&vml, CT_VML, notes_vml(&c.comments, &ws.notes, block)); + } + } + out.xml_part(name, CT_WORKSHEET, part.to_xml(&ws.rows)); + out.rels(name, &rels); +} + +/// The shape-id blocks (`o:idmap`) the kept VML drawings use, so a new drawing takes another. +fn vml_blocks(parts: &[KeptPart]) -> Vec { + let mut used = Vec::new(); + for p in parts.iter().filter(|p| p.name.to_ascii_lowercase().ends_with(".vml")) { + let text = String::from_utf8_lossy(&p.bytes); + for (_, after) in text.match_indices("idmap").map(|(i, _)| text.split_at(i)) { + let data = after.split("data=\"").nth(1).and_then(|d| d.split('"').next()).unwrap_or(""); + used.extend(data.split(',').filter_map(|n| n.trim().parse::().ok())); + } + } + used +} + /// A workbook relationship to a part that [`Document::write`] makes again (or drops). fn regenerated(r: &Rel) -> bool { matches!(r.kind(), "sharedStrings" | "styles" | "calcChain") diff --git a/crates/xlsx/src/features.rs b/crates/xlsx/src/features.rs new file mode 100644 index 000000000..19fd56401 --- /dev/null +++ b/crates/xlsx/src/features.rs @@ -0,0 +1,919 @@ +//! The worksheet's typed features, each mirroring its SpreadsheetML element: sheet views, print +//! setup, data validation, conditional formatting, hyperlinks (ECMA-376 Part 1, 18.3), and the +//! comments part (18.7) with the legacy note shapes Excel draws comments with. +//! +//! An attribute the file may leave out is an `Option` (`None`: not written, the schema's +//! default applies), so a feature reads and writes back as it was. Attributes a type does not +//! name are kept in its `attrs` (`xr:uid` and the like). + +use craft_sheet::{CellRef, MAX_COLS, MAX_ROWS, Range, col_letters}; +use craft_xml::{El, Node}; + +use crate::strings::StringItem; +use crate::styles::Color; +use crate::write::num; + +/// An attribute value type. +trait Attr: Sized { + fn parse(s: &str) -> Option; + fn spell(&self) -> String; +} + +impl Attr for bool { + fn parse(s: &str) -> Option { + craft_xml::value::to_bool(s) + } + fn spell(&self) -> String { + if *self { "1" } else { "0" }.into() + } +} +impl Attr for u32 { + fn parse(s: &str) -> Option { + craft_xml::value::to_u32(s) + } + fn spell(&self) -> String { + self.to_string() + } +} +impl Attr for i32 { + fn parse(s: &str) -> Option { + craft_xml::value::to_i32(s) + } + fn spell(&self) -> String { + self.to_string() + } +} +impl Attr for f64 { + fn parse(s: &str) -> Option { + craft_xml::value::to_f64(s) + } + fn spell(&self) -> String { + num(*self) + } +} +impl Attr for String { + fn parse(s: &str) -> Option { + Some(s.to_string()) + } + fn spell(&self) -> String { + self.clone() + } +} + +/// The attributes of `e` not in `named`. A container's `count` (written from the items) and +/// the `ref` and `sqref` each type reads into a field of its own are left out. +fn rest(e: &El, named: &[&str]) -> Vec<(String, String)> { + e.attrs.iter().filter(|(k, _)| !matches!(k.as_str(), "count" | "ref" | "sqref") && !named.contains(&k.as_str())).cloned().collect() +} + +/// A struct whose `Option` fields are the element's attributes (`field: type = "xmlName"`), with +/// further fields after `+` that the element's own reader fills, and `attrs` for the rest. +macro_rules! element { + ( + $(#[$m:meta])* pub struct $name:ident { + $($(#[$fm:meta])* $f:ident: $t:ty = $x:literal,)* + } + $(+ { $($(#[$em:meta])* $e:ident: $et:ty,)* })? + ) => { + $(#[$m])* + #[derive(Clone, Debug, Default, PartialEq)] + pub struct $name { + $($(#[$fm])* pub $f: Option<$t>,)* + $($($(#[$em])* pub $e: $et,)*)? + /// The other attributes as written. + pub attrs: Vec<(String, String)>, + } + + impl $name { + const NAMED: &'static [&'static str] = &[$($x),*]; + + #[allow(clippy::needless_update)] + fn read_attrs(el: &El) -> $name { + $name { $($f: el.attr($x).and_then(<$t as Attr>::parse),)* attrs: rest(el, Self::NAMED), ..$name::default() } + } + + fn write_attrs(&self, el: &mut El) { + $(if let Some(v) = &self.$f { el.set_attr($x, v.spell()); })* + el.attrs.extend(self.attrs.iter().cloned()); + } + } + }; +} + +/// `sqref`: ranges separated by spaces. Unreadable ranges are dropped. +pub fn parse_sqref(s: &str) -> Vec { + s.split_whitespace().filter_map(Range::parse).collect() +} + +/// A range as a reference writes it: whole columns as `A:C`, whole rows as `1:3`. +pub fn range_ref(r: &Range) -> String { + let full_cols = r.first.row == 0 && r.last.row == MAX_ROWS - 1; + let full_rows = r.first.col == 0 && r.last.col == MAX_COLS - 1; + match (full_cols, full_rows) { + (true, false) => format!("{}:{}", col_letters(r.first.col), col_letters(r.last.col)), + (false, true) => format!("{}:{}", r.first.row + 1, r.last.row + 1), + _ => r.a1(), + } +} + +fn sqref(ranges: &[Range]) -> String { + ranges.iter().map(range_ref).collect::>().join(" ") +} + +fn kids(e: &El, except: &[&str]) -> Vec { + e.elements().filter(|c| !except.contains(&c.name.as_str())).cloned().collect() +} + +// ------------------------------------------------------------------------------- sheet views + +element! { + /// ``: the split or frozen panes of a view. + pub struct Pane { + x_split: f64 = "xSplit", + y_split: f64 = "ySplit", + top_left_cell: String = "topLeftCell", + active_pane: String = "activePane", + state: String = "state", + } +} + +element! { + /// ``. + pub struct Selection { + pane: String = "pane", + active_cell: String = "activeCell", + active_cell_id: u32 = "activeCellId", + } + + { + sqref: Vec, + } +} + +element! { + /// `` (18.3.1.87). + pub struct SheetView { + window_protection: bool = "windowProtection", + show_formulas: bool = "showFormulas", + show_grid_lines: bool = "showGridLines", + show_row_col_headers: bool = "showRowColHeaders", + show_zeros: bool = "showZeros", + right_to_left: bool = "rightToLeft", + tab_selected: bool = "tabSelected", + show_ruler: bool = "showRuler", + show_outline_symbols: bool = "showOutlineSymbols", + default_grid_color: bool = "defaultGridColor", + show_white_space: bool = "showWhiteSpace", + /// `normal`, `pageBreakPreview`, `pageLayout`. + view: String = "view", + top_left_cell: String = "topLeftCell", + color_id: u32 = "colorId", + zoom_scale: u32 = "zoomScale", + zoom_scale_normal: u32 = "zoomScaleNormal", + zoom_scale_sheet_layout_view: u32 = "zoomScaleSheetLayoutView", + zoom_scale_page_layout_view: u32 = "zoomScalePageLayoutView", + workbook_view_id: u32 = "workbookViewId", + } + + { + pane: Option, + selections: Vec, + /// `pivotSelection` and `extLst`, as written. + other: Vec, + } +} + +impl SheetView { + pub fn read(e: &El) -> SheetView { + let selection = |s: &El| Selection { sqref: s.attr("sqref").map(parse_sqref).unwrap_or_default(), ..Selection::read_attrs(s) }; + SheetView { + pane: e.child("pane").map(Pane::read_attrs), + selections: e.children("selection").map(selection).collect(), + other: kids(e, &["pane", "selection"]), + ..SheetView::read_attrs(e) + } + } + + pub fn to_el(&self) -> El { + let mut e = El::new("sheetView"); + self.write_attrs(&mut e); + if let Some(p) = &self.pane { + let mut pe = El::new("pane"); + p.write_attrs(&mut pe); + e.push(pe); + } + for s in &self.selections { + let mut se = El::new("selection"); + s.write_attrs(&mut se); + if !s.sqref.is_empty() { + se.set_attr("sqref", sqref(&s.sqref)); + } + e.push(se); + } + e.children.extend(self.other.iter().cloned().map(Node::El)); + e + } +} + +// ------------------------------------------------------------------------------- printing + +element! { + /// ``. + pub struct PrintOptions { + horizontal_centered: bool = "horizontalCentered", + vertical_centered: bool = "verticalCentered", + headings: bool = "headings", + grid_lines: bool = "gridLines", + grid_lines_set: bool = "gridLinesSet", + } +} + +element! { + /// ``, inches. + pub struct PageMargins { + left: f64 = "left", + right: f64 = "right", + top: f64 = "top", + bottom: f64 = "bottom", + header: f64 = "header", + footer: f64 = "footer", + } +} + +element! { + /// ``. + pub struct PageSetup { + paper_size: u32 = "paperSize", + /// With its unit (`297mm`). + paper_height: String = "paperHeight", + paper_width: String = "paperWidth", + scale: u32 = "scale", + first_page_number: u32 = "firstPageNumber", + fit_to_width: u32 = "fitToWidth", + fit_to_height: u32 = "fitToHeight", + /// `downThenOver`, `overThenDown`. + page_order: String = "pageOrder", + /// `default`, `portrait`, `landscape`. + orientation: String = "orientation", + use_printer_defaults: bool = "usePrinterDefaults", + black_and_white: bool = "blackAndWhite", + draft: bool = "draft", + /// `none`, `asDisplayed`, `atEnd`. + cell_comments: String = "cellComments", + use_first_page_number: bool = "useFirstPageNumber", + /// `displayed`, `blank`, `dash`, `NA`. + errors: String = "errors", + horizontal_dpi: u32 = "horizontalDpi", + vertical_dpi: u32 = "verticalDpi", + copies: u32 = "copies", + /// The printer settings part. + rel_id: String = "r:id", + } +} + +element! { + /// ``: the header and footer texts with their `&` codes as written. + pub struct HeaderFooter { + different_odd_even: bool = "differentOddEven", + different_first: bool = "differentFirst", + scale_with_doc: bool = "scaleWithDoc", + align_with_margins: bool = "alignWithMargins", + } + + { + odd_header: Option, + odd_footer: Option, + even_header: Option, + even_footer: Option, + first_header: Option, + first_footer: Option, + } +} + +const HEADER_FOOTER: [&str; 6] = ["oddHeader", "oddFooter", "evenHeader", "evenFooter", "firstHeader", "firstFooter"]; + +impl HeaderFooter { + fn texts(&self) -> [&Option; 6] { + [&self.odd_header, &self.odd_footer, &self.even_header, &self.even_footer, &self.first_header, &self.first_footer] + } + + pub fn read(e: &El) -> HeaderFooter { + let t = |n: &str| e.child(n).map(|c| c.text().into_owned()); + HeaderFooter { + odd_header: t("oddHeader"), + odd_footer: t("oddFooter"), + even_header: t("evenHeader"), + even_footer: t("evenFooter"), + first_header: t("firstHeader"), + first_footer: t("firstFooter"), + ..HeaderFooter::read_attrs(e) + } + } + + pub fn to_el(&self) -> El { + let mut e = El::new("headerFooter"); + self.write_attrs(&mut e); + for (name, text) in HEADER_FOOTER.iter().zip(self.texts()) { + if let Some(t) = text { + let mut c = El::new(*name); + if t.starts_with(' ') || t.ends_with(' ') { + c.set_attr("xml:space", "preserve"); + } + e.push(if t.is_empty() { c } else { c.with_text(t.as_str()) }); + } + } + e + } +} + +element! { + /// ``: a page break before row or column `id` (zero-based is `id - 1`: the break + /// follows that many rows or columns). + pub struct Break { + id: u32 = "id", + min: u32 = "min", + max: u32 = "max", + /// Inserted by hand. + man: bool = "man", + /// Inserted for a pivot table. + pt: bool = "pt", + } +} + +fn breaks_el(name: &str, breaks: &[Break]) -> Option { + if breaks.is_empty() { + return None; + } + let manual = breaks.iter().filter(|b| b.man == Some(true)).count(); + let mut e = El::new(name).with_attr("count", breaks.len()).with_attr("manualBreakCount", manual); + for b in breaks { + let mut be = El::new("brk"); + b.write_attrs(&mut be); + e.push(be); + } + Some(e) +} + +/// How a worksheet prints. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct Print { + pub options: Option, + pub margins: Option, + pub setup: Option, + pub header_footer: Option, + pub row_breaks: Vec, + pub col_breaks: Vec, +} + +// ------------------------------------------------------------------------------- data validation + +element! { + /// `` (18.3.1.32). The formulas are as the file spells them, without `=`. + pub struct DataValidation { + /// `none`, `whole`, `decimal`, `list`, `date`, `time`, `textLength`, `custom`. + kind: String = "type", + /// `stop`, `warning`, `information`. + error_style: String = "errorStyle", + ime_mode: String = "imeMode", + operator: String = "operator", + allow_blank: bool = "allowBlank", + /// Set: the in-cell list arrow is *hidden*. + show_drop_down: bool = "showDropDown", + show_input_message: bool = "showInputMessage", + show_error_message: bool = "showErrorMessage", + error_title: String = "errorTitle", + error: String = "error", + prompt_title: String = "promptTitle", + prompt: String = "prompt", + } + + { + sqref: Vec, + formula1: Option, + formula2: Option, + } +} + +impl DataValidation { + /// A `dataValidation` of the main namespace, or of an `x14:dataValidations` extension (its + /// formulas in `xm:f` and its ranges in `xm:sqref`). + pub fn read(v: &El) -> DataValidation { + let formula = |n: &str| { + let f = v.child(n).or_else(|| v.child(format!("x14:{n}").as_str()))?; + Some(f.child("xm:f").map_or_else(|| f.text(), |x| x.text()).trim().to_string()) + }; + let sq = v.attr("sqref").map(str::to_string).or_else(|| v.child("xm:sqref").map(|s| s.text().into_owned())); + DataValidation { + sqref: sq.as_deref().map(parse_sqref).unwrap_or_default(), + formula1: formula("formula1"), + formula2: formula("formula2"), + ..DataValidation::read_attrs(v) + } + } + + /// The main-namespace element. + pub fn to_el(&self) -> El { + let mut e = El::new("dataValidation"); + self.write_attrs(&mut e); + e.set_attr("sqref", sqref(&self.sqref)); + for (n, f) in [("formula1", &self.formula1), ("formula2", &self.formula2)] { + if let Some(f) = f { + e.push(El::new(n).with_text(f.as_str())); + } + } + e + } +} + +element! { + /// ``. + pub struct DataValidations { + disable_prompts: bool = "disablePrompts", + x_window: u32 = "xWindow", + y_window: u32 = "yWindow", + } + + { + items: Vec, + } +} + +// ------------------------------------------------------------------------------- conditional formats + +element! { + /// ``: a threshold of a colour scale, data bar or icon set. + pub struct Cfvo { + /// `num`, `percent`, `max`, `min`, `formula`, `percentile`. + kind: String = "type", + val: String = "val", + gte: bool = "gte", + } +} + +fn cfvos(e: &El) -> Vec { + e.children("cfvo").map(Cfvo::read_attrs).collect() +} + +fn push_cfvos(e: &mut El, cfvos: &[Cfvo]) { + for c in cfvos { + let mut ce = El::new("cfvo"); + c.write_attrs(&mut ce); + e.push(ce); + } +} + +/// ``. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct ColorScale { + pub cfvos: Vec, + pub colors: Vec, +} + +element! { + /// ``. + pub struct DataBar { + min_length: u32 = "minLength", + max_length: u32 = "maxLength", + show_value: bool = "showValue", + } + + { + cfvos: Vec, + color: Option, + } +} + +element! { + /// ``. + pub struct IconSet { + icon_set: String = "iconSet", + show_value: bool = "showValue", + percent: bool = "percent", + reverse: bool = "reverse", + } + + { + cfvos: Vec, + } +} + +element! { + /// `` (18.3.1.10). + pub struct CfRule { + /// `expression`, `cellIs`, `colorScale`, `dataBar`, `iconSet`, `top10`, `uniqueValues`, + /// `duplicateValues`, `containsText`, ..., `aboveAverage`. + kind: String = "type", + /// An index into [`crate::Styles::dxfs`]. + dxf_id: u32 = "dxfId", + priority: u32 = "priority", + stop_if_true: bool = "stopIfTrue", + above_average: bool = "aboveAverage", + percent: bool = "percent", + bottom: bool = "bottom", + operator: String = "operator", + text: String = "text", + time_period: String = "timePeriod", + rank: u32 = "rank", + std_dev: i32 = "stdDev", + equal_average: bool = "equalAverage", + } + + { + /// `` texts, as written. + formulas: Vec, + color_scale: Option, + data_bar: Option, + icon_set: Option, + /// The rule's `extLst` (the id that links a data bar to its Excel 2010 extension). + ext: Option, + } +} + +impl CfRule { + pub fn read(r: &El) -> CfRule { + CfRule { + formulas: r.children("formula").map(|f| f.text().into_owned()).collect(), + color_scale: r.child("colorScale").map(|c| ColorScale { cfvos: cfvos(c), colors: c.children("color").map(Color::read).collect() }), + data_bar: r.child("dataBar").map(|d| DataBar { cfvos: cfvos(d), color: d.child("color").map(Color::read), ..DataBar::read_attrs(d) }), + icon_set: r.child("iconSet").map(|i| IconSet { cfvos: cfvos(i), ..IconSet::read_attrs(i) }), + ext: r.child("extLst").cloned(), + ..CfRule::read_attrs(r) + } + } + + pub fn to_el(&self) -> El { + let mut e = El::new("cfRule"); + self.write_attrs(&mut e); + for f in &self.formulas { + e.push(El::new("formula").with_text(f.as_str())); + } + if let Some(c) = &self.color_scale { + let mut ce = El::new("colorScale"); + push_cfvos(&mut ce, &c.cfvos); + c.colors.iter().for_each(|col| ce.push(col.to_el("color"))); + e.push(ce); + } + if let Some(d) = &self.data_bar { + let mut de = El::new("dataBar"); + d.write_attrs(&mut de); + push_cfvos(&mut de, &d.cfvos); + if let Some(c) = &d.color { + de.push(c.to_el("color")); + } + e.push(de); + } + if let Some(i) = &self.icon_set { + let mut ie = El::new("iconSet"); + i.write_attrs(&mut ie); + push_cfvos(&mut ie, &i.cfvos); + e.push(ie); + } + if let Some(x) = &self.ext { + e.push(x.clone()); + } + e + } +} + +element! { + /// ``: rules over ranges. + pub struct ConditionalFormatting { + pivot: bool = "pivot", + } + + { + sqref: Vec, + rules: Vec, + ext: Option, + } +} + +impl ConditionalFormatting { + pub fn read(e: &El) -> ConditionalFormatting { + ConditionalFormatting { + sqref: e.attr("sqref").map(parse_sqref).unwrap_or_default(), + rules: e.children("cfRule").map(CfRule::read).collect(), + ext: e.child("extLst").cloned(), + ..ConditionalFormatting::read_attrs(e) + } + } + + pub fn to_el(&self) -> El { + let mut e = El::new("conditionalFormatting"); + self.write_attrs(&mut e); + e.set_attr("sqref", sqref(&self.sqref)); + self.rules.iter().for_each(|r| e.push(r.to_el())); + if let Some(x) = &self.ext { + e.push(x.clone()); + } + e + } +} + +// ------------------------------------------------------------------------------- hyperlinks + +element! { + /// ``: to a URL through a relationship, or to a place in the workbook. + pub struct Hyperlink { + rel_id: String = "r:id", + /// A place in the workbook (`Sheet2!A1`, a defined name). + location: String = "location", + tooltip: String = "tooltip", + display: String = "display", + } + + { + range: Range, + /// The relationship's target when it points outside the package, read from the sheet's + /// relationships. On writing, a link whose `rel_id` does not lead to this URL gets a + /// relationship of its own. + url: Option, + } +} + +impl Hyperlink { + fn read(e: &El, rels: &craft_opc::Rels) -> Option { + let range = Range::parse(e.attr("ref")?)?; + let h = Hyperlink::read_attrs(e); + let url = h.rel_id.as_deref().and_then(|id| rels.by_id(id)).filter(|r| r.external).map(|r| r.target.clone()); + Some(Hyperlink { range, url, ..h }) + } + + pub fn to_el(&self) -> El { + let mut e = El::new("hyperlink").with_attr("ref", range_ref(&self.range)); + self.write_attrs(&mut e); + e + } +} + +// ------------------------------------------------------------------------------- the lift + +/// The typed features of a worksheet, lifted from its tree. Each one listed here is written from +/// these fields; its element in [`crate::SheetPart::root`] is not. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct Features { + pub views: Vec, + pub print: Print, + pub validations: Option, + pub conditional_formats: Vec, + pub hyperlinks: Vec, + /// `legacyDrawing/@r:id`: the VML part the sheet's notes and form controls are drawn in. + pub legacy_drawing: Option, +} + +impl Features { + pub(crate) fn read(root: &El, rels: &craft_opc::Rels) -> Features { + let attrs_of = |n: &str| root.child(n); + Features { + views: root.child("sheetViews").map(|v| v.children("sheetView").map(SheetView::read).collect()).unwrap_or_default(), + print: Print { + options: attrs_of("printOptions").map(PrintOptions::read_attrs), + margins: attrs_of("pageMargins").map(PageMargins::read_attrs), + setup: attrs_of("pageSetup").map(PageSetup::read_attrs), + header_footer: attrs_of("headerFooter").map(HeaderFooter::read), + row_breaks: root.child("rowBreaks").map(|b| b.children("brk").map(Break::read_attrs).collect()).unwrap_or_default(), + col_breaks: root.child("colBreaks").map(|b| b.children("brk").map(Break::read_attrs).collect()).unwrap_or_default(), + }, + validations: root.child("dataValidations").map(|d| DataValidations { + items: d.children("dataValidation").map(DataValidation::read).collect(), + ..DataValidations::read_attrs(d) + }), + conditional_formats: root.children("conditionalFormatting").map(ConditionalFormatting::read).collect(), + hyperlinks: root + .child("hyperlinks") + .map(|h| h.children("hyperlink").filter_map(|e| Hyperlink::read(e, rels)).collect()) + .unwrap_or_default(), + legacy_drawing: root.child("legacyDrawing").and_then(|l| l.attr_or_plain("r:id")).map(str::to_string), + } + } + + /// The elements these fields own, for [`crate::write::assemble`]. Conditional formats are + /// one element per block: the first block takes the place of the file's first, the others + /// follow it. + pub(crate) fn owned(&self) -> Vec<(&'static str, Vec)> { + let mut views = El::new("sheetViews"); + self.views.iter().for_each(|v| views.push(v.to_el())); + let el = |name: &str, write: &dyn Fn(&mut El)| { + let mut e = El::new(name); + write(&mut e); + e + }; + let validations = self.validations.as_ref().filter(|v| !v.items.is_empty()).map(|v| { + let mut e = el("dataValidations", &|e| v.write_attrs(e)); + e.attrs.insert(0, ("count".into(), v.items.len().to_string())); + v.items.iter().for_each(|i| e.push(i.to_el())); + e + }); + let mut links = El::new("hyperlinks"); + self.hyperlinks.iter().for_each(|h| links.push(h.to_el())); + let p = &self.print; + vec![ + ("sheetViews", Vec::from_iter((!self.views.is_empty()).then_some(views))), + ("conditionalFormatting", self.conditional_formats.iter().map(ConditionalFormatting::to_el).collect()), + ("dataValidations", Vec::from_iter(validations)), + ("hyperlinks", Vec::from_iter((!self.hyperlinks.is_empty()).then_some(links))), + ("printOptions", Vec::from_iter(p.options.as_ref().map(|o| el("printOptions", &|e| o.write_attrs(e))))), + ("pageMargins", Vec::from_iter(p.margins.as_ref().map(|m| el("pageMargins", &|e| m.write_attrs(e))))), + ("pageSetup", Vec::from_iter(p.setup.as_ref().map(|s| el("pageSetup", &|e| s.write_attrs(e))))), + ("headerFooter", Vec::from_iter(p.header_footer.as_ref().map(HeaderFooter::to_el))), + ("rowBreaks", Vec::from_iter(breaks_el("rowBreaks", &p.row_breaks))), + ("colBreaks", Vec::from_iter(breaks_el("colBreaks", &p.col_breaks))), + ("legacyDrawing", Vec::from_iter(self.legacy_drawing.as_ref().map(|id| El::new("legacyDrawing").with_attr("r:id", id)))), + ] + } +} + +// ------------------------------------------------------------------------------- comments + +element! { + /// ``: a note on a cell. + pub struct Comment { + author_id: u32 = "authorId", + guid: String = "guid", + shape_id: u32 = "shapeId", + } + + { + at: CellRef, + text: StringItem, + /// `commentPr`, as written. + props: Option, + } +} + +/// The comments part of a worksheet (`xl/commentsN.xml`). +#[derive(Clone, Debug, Default, PartialEq)] +pub struct Comments { + pub authors: Vec, + pub comments: Vec, + /// The whole part, for what is not lifted out (extensions). + pub root: El, +} + +impl Comments { + /// Comments with no readable `ref` are dropped. + pub fn read(root: El) -> Comments { + let authors = root.child("authors").map(|a| a.children("author").map(|a| a.text().into_owned()).collect()).unwrap_or_default(); + let comments = root + .child("commentList") + .map(|l| { + l.children("comment") + .filter_map(|c| { + Some(Comment { + at: CellRef::parse(c.attr("ref")?)?, + text: c.child("text").map(StringItem::read).unwrap_or_default(), + props: c.child("commentPr").cloned(), + ..Comment::read_attrs(c) + }) + }) + .collect() + }) + .unwrap_or_default(); + Comments { authors, comments, root } + } + + /// The author of `c`. + pub fn author(&self, c: &Comment) -> Option<&str> { + self.authors.get(c.author_id? as usize).map(String::as_str) + } + + pub fn to_xml(&self) -> Vec { + let root = if self.root.name.is_empty() { El::new("comments") } else { self.root.clone() }; + let mut authors = El::new("authors"); + self.authors.iter().for_each(|a| authors.push(El::new("author").with_text(a.as_str()))); + let mut list = El::new("commentList"); + for c in &self.comments { + let mut e = El::new("comment").with_attr("ref", c.at.a1()); + c.write_attrs(&mut e); + e.push(c.text.to_el("text")); + if let Some(p) = &c.props { + e.push(p.clone()); + } + list.push(e); + } + let mut out = + crate::write::assemble(&root, &["authors", "commentList", "extLst"], vec![("authors", Some(authors)), ("commentList", Some(list))]); + crate::ns::declare(&mut out); + let mut w = craft_xml::Writer::document(); + w.tree(&out); + w.into_bytes() + } +} + +/// How a note's box is drawn, from its legacy VML shape. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct NoteShape { + pub at: CellRef, + /// Shown all the time (``), not only on hover. + pub visible: bool, + /// `x:Anchor`: left column, offset px, top row, offset px, right column, offset px, bottom + /// row, offset px. + pub anchor: Option<[u32; 8]>, + /// The shape's width and height in points, from its style. + pub size_pt: Option<(f64, f64)>, +} + +impl NoteShape { + /// The note shapes of a VML drawing read with [`crate::ns::CANONICAL`]. Shapes may sit in + /// groups; other shapes (form controls) are skipped. + pub fn read_all(vml: &El) -> Vec { + let mut out = Vec::new(); + for shape in vml.descendants().filter(|e| e.name == "v:shape") { + let Some(cd) = shape.child("x:ClientData").filter(|cd| cd.attr("ObjectType") == Some("Note")) else { continue }; + let n = |k: &str| cd.child(k).and_then(|v| v.text().trim().parse::().ok()); + let (Some(row), Some(col)) = (n("x:Row"), n("x:Column")) else { continue }; + if row >= MAX_ROWS || col >= MAX_COLS { + continue; + } + let anchor = cd.child("x:Anchor").and_then(|a| { + let v: Vec = a.text().split(',').map(|p| p.trim().parse::().ok()).collect::>()?; + <[u32; 8]>::try_from(v).ok() + }); + let style = shape.attr("style").unwrap_or(""); + let pt = |key: &str| { + style + .split(';') + .filter_map(|d| d.split_once(':')) + .find(|(k, _)| k.trim().eq_ignore_ascii_case(key)) + .and_then(|(_, v)| v.trim().strip_suffix("pt")?.trim().parse::().ok().filter(|n| n.is_finite() && *n > 0.0 && *n < 1e6)) + }; + let size_pt = pt("width").zip(pt("height")); + out.push(NoteShape { at: CellRef::new(row, col), visible: cd.child("x:Visible").is_some(), anchor, size_pt }); + } + out + } +} + +/// A legacy VML drawing that shows `comments` as notes, Excel's way, with shape ids from block +/// `block` (`o:idmap`: ids `1024 * block + 1` on). A note with a shape in `shapes` keeps its box +/// and visibility; the others get Excel's default box, two columns right of the cell. +pub fn notes_vml(comments: &[Comment], shapes: &[NoteShape], block: u32) -> Vec { + let mut s = format!( + "" + ); + for (i, c) in comments.iter().enumerate() { + let shape = shapes.iter().find(|s| s.at == c.at); + let id = u64::from(block) * 1024 + 1 + i as u64; + let visible = shape.is_some_and(|s| s.visible); + let clamp = |v: i64, max: u32| v.clamp(0, i64::from(max) - 1); + let (row, col) = (i64::from(c.at.row), i64::from(c.at.col)); + let anchor = shape.and_then(|s| s.anchor).unwrap_or_else(|| { + let a = [clamp(col + 1, MAX_COLS), 15, clamp(row, MAX_ROWS), 2, clamp(col + 3, MAX_COLS), 15, clamp(row + 4, MAX_ROWS), 16]; + a.map(|v| u32::try_from(v).unwrap_or(0)) + }); + let (w, h) = shape.and_then(|s| s.size_pt).unwrap_or((108.0, 59.25)); + let anchor = anchor.map(|v| v.to_string()).join(", "); + s.push_str(&format!( + "
{anchor}False{}{}{}", + num(w), + num(h), + i + 1, + if visible { "visible" } else { "hidden" }, + c.at.row, + c.at.col, + if visible { "" } else { "" } + )); + } + s.push_str(""); + s.into_bytes() +} + +#[cfg(test)] +mod tests { + use super::*; + + fn read(xml: &str) -> El { + craft_xml::Options::canonical(&crate::ns::CANONICAL).parse(xml.as_bytes()).unwrap_or_default() + } + + #[test] + fn rules_with_scales_bars_and_icons_write_back() { + let x = r#"{1}"#; + let cf = ConditionalFormatting::read(&read(x)); + assert_eq!(cf.sqref.len(), 2); + let [scale, bar, icons, top] = &cf.rules[..] else { panic!("{:?}", cf.rules) }; + let scale = scale.color_scale.as_ref().unwrap(); + assert_eq!((scale.cfvos[1].kind.as_deref(), scale.cfvos[1].val.as_deref(), scale.colors[1].tint), (Some("percentile"), Some("50"), 0.5)); + assert_eq!(bar.data_bar.as_ref().map(|d| d.show_value), Some(Some(false))); + assert!(bar.ext.as_ref().is_some_and(|e| e.find("x14:id").is_some())); + assert_eq!( + icons.icon_set.as_ref().map(|i| (i.icon_set.as_deref(), i.reverse, i.cfvos[1].gte)), + Some((Some("3Arrows"), Some(true), Some(false))) + ); + assert_eq!((top.rank, top.percent, top.bottom, top.stop_if_true, top.dxf_id), (Some(5), Some(true), Some(true), Some(true), Some(1))); + let mut written = cf.to_el(); + crate::ns::declare(&mut written); + assert_eq!(ConditionalFormatting::read(&read(&written.to_xml())), cf); + } + + #[test] + fn extension_validations_read_into_the_same_type() { + let x = r#"Other!$A$1:$A$3G1:G4"#; + let v = DataValidation::read(&read(x)); + assert_eq!( + (v.kind.as_deref(), v.formula1.as_deref(), v.sqref.iter().map(range_ref).collect::>()), + (Some("list"), Some("Other!$A$1:$A$3"), vec!["G1:G4".to_string()]) + ); + assert_eq!( + v.to_el().to_xml(), + r#"Other!$A$1:$A$3"# + ); + } + + #[test] + fn references() { + for r in ["A1", "B2:C9", "A:C", "3:5", "A1:XFD1048576"] { + assert_eq!(range_ref(&Range::parse(r).unwrap()), r); + } + assert_eq!(parse_sqref(" A1 B2:C3 bad 4:4").len(), 3); + } +} diff --git a/crates/xlsx/src/lib.rs b/crates/xlsx/src/lib.rs index 09d7b673e..24cce3292 100644 --- a/crates/xlsx/src/lib.rs +++ b/crates/xlsx/src/lib.rs @@ -50,6 +50,7 @@ //! ``` mod document; +mod features; pub mod ns; mod package; mod sheet; @@ -66,6 +67,10 @@ pub use craft_sheet::{ builtin_format, builtin_format_id, quote_sheet, sheet_needs_quotes, }; pub use document::{Document, KeptPart, Worksheet, XlsxKind}; +pub use features::{ + Break, CfRule, Cfvo, ColorScale, Comment, Comments, ConditionalFormatting, DataBar, DataValidation, DataValidations, Features, HeaderFooter, + Hyperlink, IconSet, NoteShape, PageMargins, PageSetup, Pane, Print, PrintOptions, Selection, SheetView, notes_vml, parse_sqref, range_ref, +}; pub use package::{Package, is_xlsb}; pub use sheet::{Cell, CellFormula, CellValue, Column, FormulaKind, Row, SheetPart, iso_date_serial}; pub use strings::{PhoneticProps, PhoneticRun, StringItem, TextRun, decode_xstring, encode_xstring, rich_text, t_el}; diff --git a/crates/xlsx/src/package.rs b/crates/xlsx/src/package.rs index 370812653..6fc9dc941 100644 --- a/crates/xlsx/src/package.rs +++ b/crates/xlsx/src/package.rs @@ -3,6 +3,7 @@ use craft_opc::{ContentTypes, LazyPackage, Rels}; use craft_xml::{El, Options, UnknownEntity}; +use crate::features::{Comments, NoteShape}; use crate::sheet::{Row, RowReader, SheetPart}; use crate::strings::StringItem; use crate::styles::Styles; @@ -257,6 +258,18 @@ impl<'a> Package<'a> { Ok(sheet) } + /// The comments of a worksheet read by [`Package::sheet`]; `None` when it has none. + pub fn comments(&mut self, sheet: &SheetPart) -> Result, Error> { + let Some(rel) = sheet.rels.first("comments").filter(|r| !r.external) else { return Ok(None) }; + Ok(self.optional_xml(&rel.target.clone())?.map(Comments::read)) + } + + /// The note shapes of a worksheet's legacy VML drawing: how its comments are shown. + pub fn note_shapes(&mut self, sheet: &SheetPart) -> Result, Error> { + let Some(rel) = sheet.rels.first("vmlDrawing").filter(|r| !r.external) else { return Ok(Vec::new()) }; + Ok(self.optional_xml(&rel.target.clone())?.map(|v| NoteShape::read_all(&v)).unwrap_or_default()) + } + fn required(&mut self, part: &str) -> Result, Error> { self.opc.read(part)?.ok_or_else(|| Error::Invalid(format!("missing part {part}"))) } diff --git a/crates/xlsx/src/sheet.rs b/crates/xlsx/src/sheet.rs index fc0e33bc9..243430f8e 100644 --- a/crates/xlsx/src/sheet.rs +++ b/crates/xlsx/src/sheet.rs @@ -7,6 +7,7 @@ use craft_opc::Rels; use craft_sheet::{CellError, CellRef, MAX_COLS, MAX_ROWS, Range, Value}; use craft_xml::{El, Writer}; +use crate::features::Features; use crate::strings::{StringItem, decode_xstring, encode_xstring}; use crate::write::{assemble, list, num, write_streamed}; @@ -190,6 +191,9 @@ pub struct SheetPart { /// Why reading stopped early, when the part is damaged; the rows before the damage were /// handed out. pub damaged: Option, + /// Views, print setup, data validation, conditional formats, hyperlinks: written from here, + /// not from [`SheetPart::root`]. + pub features: Features, } impl SheetPart { @@ -213,6 +217,7 @@ impl SheetPart { }); } } + self.features = Features::read(&self.root, &self.rels); if let Some(m) = self.root.child("mergeCells") { self.merges.extend(m.children("mergeCell").filter_map(|m| m.attr("ref").and_then(Range::parse)).filter(|r| !r.is_single())); } @@ -220,8 +225,8 @@ impl SheetPart { } impl SheetPart { - /// The worksheet part with `rows` as its `sheetData`. The dimension, the columns and the - /// merged cells come from the fields; everything else in [`SheetPart::root`] is written as + /// The worksheet part with `rows` as its `sheetData`. The dimension, the columns, the + /// merged cells and [`SheetPart::features`] come from the fields; everything else in [`SheetPart::root`] is written as /// kept, in place. pub fn to_xml(&self, rows: &[Row]) -> Vec { let root = if self.root.name.is_empty() { El::new("worksheet") } else { self.root.clone() }; @@ -230,12 +235,13 @@ impl SheetPart { cols.push(column_el(c)); } let merges = self.merges.iter().map(|m| El::new("mergeCell").with_attr("ref", m.a1())).collect(); - let owned = vec![ - ("dimension", Some(El::new("dimension").with_attr("ref", used(rows).map_or_else(|| "A1".to_string(), |r| r.a1())))), - ("cols", (!self.columns.is_empty()).then_some(cols)), - ("sheetData", Some(El::new("sheetData"))), - ("mergeCells", list(&root, "mergeCells", merges)), + let mut owned = vec![ + ("dimension", vec![El::new("dimension").with_attr("ref", used(rows).map_or_else(|| "A1".to_string(), |r| r.a1()))]), + ("cols", Vec::from_iter((!self.columns.is_empty()).then_some(cols))), + ("sheetData", vec![El::new("sheetData")]), + ("mergeCells", Vec::from_iter(list(&root, "mergeCells", merges))), ]; + owned.extend(self.features.owned()); let mut root = assemble(&root, WORKSHEET, owned); let prefixes: Vec<&str> = rows.iter().flat_map(|r| &r.attrs).filter_map(|(k, _)| k.split_once(':').map(|(p, _)| p)).collect(); crate::ns::declare_with(&mut root, &prefixes); diff --git a/crates/xlsx/src/write.rs b/crates/xlsx/src/write.rs index b9a38bd7d..132128ef2 100644 --- a/crates/xlsx/src/write.rs +++ b/crates/xlsx/src/write.rs @@ -21,18 +21,19 @@ pub fn num(n: f64) -> String { /// `root` with the children named in `owned` regenerated. Each `(name, new)` takes the place /// of the first child of that name, and later children of that name are dropped; when `root` -/// has none, `new` goes where `order` (the schema's sequence) puts it. `None` removes the -/// child. Every other child stays where it is, so elements the schema does not list +/// has none, `new` goes where `order` (the schema's sequence) puts it. No element (`None`) +/// removes the children; several (repeated blocks) go in a row. Every other child stays where it is, so elements the schema does not list /// (`mc:AlternateContent`) keep their place. -pub(crate) fn assemble(root: &El, order: &[&str], mut owned: Vec<(&str, Option)>) -> El { +pub(crate) fn assemble>(root: &El, order: &[&str], owned: Vec<(&str, I)>) -> El { + let mut owned: Vec<(&str, Vec)> = owned.into_iter().map(|(n, e)| (n, e.into_iter().collect())).collect(); let pos = |n: &str| order.iter().position(|o| *o == n).unwrap_or(order.len()); owned.sort_by_key(|(n, _)| pos(n)); // The children of `root` each owned name replaces, so a regenerated one is not also written // where `order` would put it. let present: Vec<&str> = owned.iter().map(|(n, _)| *n).filter(|n| root.child(*n).is_some()).collect(); let mut out = El { name: root.name.clone(), attrs: root.attrs.clone(), children: Vec::new() }; - let mut pending: Vec<(&str, Option)> = Vec::new(); - let mut later: Vec<(&str, Option)> = Vec::new(); + let mut pending: Vec<(&str, Vec)> = Vec::new(); + let mut later: Vec<(&str, Vec)> = Vec::new(); for (n, e) in owned { if present.contains(&n) { later.push((n, e)) } else { pending.push((n, e)) } } @@ -47,12 +48,12 @@ pub(crate) fn assemble(root: &El, order: &[&str], mut owned: Vec<(&str, Option"#); + assert_eq!(out.to_xml(), r#""#); let empty = assemble(&El::new("w"), &order, vec![("d", Some(El::new("d"))), ("a", Some(El::new("a")))]); assert_eq!(empty.to_xml(), ""); } diff --git a/crates/xlsx/tests/write.rs b/crates/xlsx/tests/write.rs index 4ada52b5e..c42fb522f 100644 --- a/crates/xlsx/tests/write.rs +++ b/crates/xlsx/tests/write.rs @@ -30,11 +30,11 @@ fn excel_like() -> Vec { r#" 010.1SUM(C1:C3)0.300000000000000041C1*20.2#N/A inline 0.4 -0.25 +0.25"a,b"&C&P x Data!C1:C3D1"# ); let sheet1_rels = format!( - r#""# + r#""# ); let styles = format!( r#" @@ -50,7 +50,7 @@ fn excel_like() -> Vec { let package_rels = format!( r#""# ); - let types = r#""#; + let types = r#""#; zip(&[ ("[Content_Types].xml", types.as_bytes()), ("_rels/.rels", package_rels.as_bytes()), @@ -66,10 +66,15 @@ fn excel_like() -> Vec { ("xl/styles.xml", styles.as_bytes()), ("xl/sharedStrings.xml", strings.as_bytes()), ("xl/calcChain.xml", b""), + ("xl/comments1.xml", COMMENTS.as_bytes()), + ("xl/drawings/vmlDrawing1.vml", VML.as_bytes()), ("xl/vbaProject.bin", &[0xD0, 0xCF, 0x11, 0xE0, 1, 2, 3]), ]) } +const COMMENTS: &str = "AnnAnn:\nnote"; +const VML: &str = "2, 15, 0, 2, 4, 15, 4, 1611"; + fn read(bytes: &[u8]) -> Document { Document::read(bytes, &Limits::default()).unwrap() } @@ -114,6 +119,12 @@ fn assert_same(a: &Document, b: &Document) { assert_eq!((&x.part.part, &x.part.columns, &x.part.merges), (&y.part.part, &y.part.columns, &y.part.merges)); assert_eq!(String::from_utf8(x.part.to_xml(&x.rows)), String::from_utf8(y.part.to_xml(&y.rows))); assert_eq!(x.part.rels.list, y.part.rels.list); + assert_eq!(x.part.features, y.part.features); + assert_eq!(x.notes, y.notes); + assert_eq!( + x.comments.as_ref().map(|c| (&c.authors, &c.comments, c.to_xml())), + y.comments.as_ref().map(|c| (&c.authors, &c.comments, c.to_xml())) + ); } let mut pa = a.parts.clone(); let mut pb = b.parts.clone(); @@ -169,6 +180,36 @@ fn a_workbook_round_trips() { assert!(types.contains("macroEnabled.main+xml") && types.contains(r#"Extension="bin""#) && !types.contains("calcChain"), "{types}"); } +#[test] +fn features_are_read() { + let doc = read(&excel_like()); + let ws = doc.worksheets[0].as_ref().unwrap(); + let f = &ws.part.features; + assert_eq!(f.views[0].pane.as_ref().and_then(|p| p.state.as_deref()), Some("frozen")); + assert_eq!(f.views[0].tab_selected, Some(true)); + let dv = &f.validations.as_ref().unwrap().items[0]; + assert_eq!((dv.kind.as_deref(), dv.formula1.as_deref(), dv.sqref.len()), (Some("list"), Some("\"a,b\""), 2)); + assert_eq!(dv.attrs, [("xr:uid".to_string(), "{00000000-0002-0000-0000-000000000000}".to_string())]); + assert_eq!(craft_xlsx::range_ref(&dv.sqref[1]), "C:C"); + assert_eq!(f.conditional_formats[0].rules[0].formulas, ["0.25"]); + assert_eq!(f.hyperlinks[0].url.as_deref(), Some("https://example.com/a?b&c")); + assert_eq!((f.hyperlinks[1].location.as_deref(), f.hyperlinks[1].url.as_deref()), (Some("Data!A1"), None)); + assert_eq!(f.print.options.as_ref().and_then(|o| o.grid_lines), Some(true)); + assert_eq!(f.print.setup.as_ref().and_then(|s| s.orientation.as_deref()), Some("landscape")); + let hf = f.print.header_footer.as_ref().unwrap(); + assert_eq!((hf.odd_header.as_deref(), hf.first_footer.as_deref(), hf.different_first), (Some("&C&P"), Some(" x "), Some(true))); + assert_eq!((f.print.row_breaks[0].id, f.print.row_breaks[0].man), (Some(10), Some(true))); + assert_eq!(f.legacy_drawing.as_deref(), Some("rId3")); + let comments = ws.comments.as_ref().unwrap(); + assert_eq!(comments.comments.len(), 1); + assert_eq!((comments.author(&comments.comments[0]), &*comments.comments[0].text.text), (Some("Ann"), "Ann:\nnote")); + assert_eq!( + ws.notes, + [craft_xlsx::NoteShape { at: CellRef::new(1, 1), visible: true, anchor: Some([2, 15, 0, 2, 4, 15, 4, 16]), size_pt: Some((120.0, 60.0)) }] + ); + assert!(doc.parts.iter().any(|p| p.name == "xl/drawings/vmlDrawing1.vml") && !doc.parts.iter().any(|p| p.name == "xl/comments1.xml")); +} + #[test] fn edits_are_written() { let mut doc = read(&excel_like()); @@ -196,6 +237,68 @@ fn edits_are_written() { assert_eq!(rels.all("worksheet").count(), 1); } +#[test] +fn features_are_edited() { + let mut doc = read(&excel_like()); + let ws = doc.worksheets[0].as_mut().unwrap(); + let f = &mut ws.part.features; + f.hyperlinks[0].url = Some("https://example.org/".into()); + f.conditional_formats.push(craft_xlsx::ConditionalFormatting { + sqref: vec![Range::parse("A1:A9").unwrap()], + rules: vec![craft_xlsx::CfRule { kind: Some("expression".into()), priority: Some(2), formulas: vec!["A1>1".into()], ..Default::default() }], + ..Default::default() + }); + f.print.setup = None; + // A comment on a sheet without a VML drawing gets one, in a shape block no other drawing uses. + let empty = doc.worksheets[2].as_mut().unwrap(); + empty.comments = Some(craft_xlsx::Comments { + authors: vec!["Bo".into()], + comments: vec![craft_xlsx::Comment { at: CellRef::new(0, 0), author_id: Some(0), text: StringItem::plain("hi"), ..Default::default() }], + ..Default::default() + }); + let bytes = doc.write().unwrap(); + let again = read(&bytes); + assert_same(&doc_after(&doc), &again); + let f = &again.worksheets[0].as_ref().unwrap().part.features; + assert_eq!(f.hyperlinks[0].url.as_deref(), Some("https://example.org/")); + assert_eq!(f.conditional_formats.len(), 2); + assert!(f.print.setup.is_none()); + let sheet2 = again.worksheets[2].as_ref().unwrap(); + assert_eq!(sheet2.comments.as_ref().map(|c| c.comments[0].text.text.to_string()), Some("hi".into())); + assert!(sheet2.part.features.legacy_drawing.is_some()); + assert_eq!(sheet2.notes.iter().map(|n| (n.at, n.visible)).collect::>(), [(CellRef::new(0, 0), false)]); + let vml = again.parts.iter().find(|p| p.name == "xl/drawings/vmlDrawing2.vml").unwrap(); + assert!(String::from_utf8_lossy(&vml.bytes).contains(r#"data="2""#)); + let sheet = + String::from_utf8(Package::open(&bytes, &Limits::default()).unwrap().opc().read("xl/worksheets/sheet1.xml").unwrap().unwrap()).unwrap(); + let order = [ + " = order.iter().map(|o| sheet.find(o).unwrap()).collect(); + assert!(at.windows(2).all(|w| w[0] < w[1]), "{sheet}"); + // Comments removed take their part and relationship with them. + let mut doc = again; + doc.worksheets[0].as_mut().unwrap().comments = None; + let bytes = doc.write().unwrap(); + let mut p = Package::open(&bytes, &Limits::default()).unwrap(); + assert!(p.opc().read("xl/comments1.xml").unwrap().is_none()); + let wb = p.workbook().unwrap(); + let part = p.sheet(&wb.sheets[0], |_| Ok(())).unwrap(); + assert!(part.rels.first("comments").is_none()); +} + +/// The document as written: a link to a new URL has a relationship of its own, a new comments +/// part a VML drawing. +fn doc_after(doc: &Document) -> Document { + read(&doc.write().unwrap()) +} + #[test] fn a_new_workbook() { let cell = |at: &str, value| Cell { at: CellRef::parse(at).unwrap(), value, ..Cell::default() }; From 59d72512eb26c43a9cb14a33a48713ab87323d21 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:29:55 -0500 Subject: [PATCH 07/10] craft-timeline: restore OpenTimelineIO after the FCPXML squash reverted it (#1069) #987 (FCPXML) was squash-merged after #1002 (OpenTimelineIO), which was stacked on it. The squash carried #987's older copies of Cargo.toml, lib.rs, model.rs, README.md and the tests, which removed the otio feature and module wiring that #1002 had added. This puts back #1002's merged state of crates/timeline exactly; it already contains every FCPXML change. --- Cargo.lock | 1 + crates/timeline/Cargo.toml | 6 +++++- crates/timeline/README.md | 3 ++- crates/timeline/src/fcpxml.rs | 2 +- crates/timeline/src/lib.rs | 7 ++++++- crates/timeline/src/model.rs | 5 ++--- crates/timeline/tests/it/fcp7.rs | 4 ++-- crates/timeline/tests/it/main.rs | 1 + crates/timeline/tests/it/support.rs | 7 +++++-- 9 files changed, 25 insertions(+), 11 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 3924f3435..7bcbe4339 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1883,6 +1883,7 @@ dependencies = [ "craft-xml", "proptest", "serde", + "serde_json", ] [[package]] diff --git a/crates/timeline/Cargo.toml b/crates/timeline/Cargo.toml index 59a79ed4b..3eca9bc5c 100644 --- a/crates/timeline/Cargo.toml +++ b/crates/timeline/Cargo.toml @@ -15,14 +15,18 @@ workspace = true serde = ["dep:serde"] # Final Cut Pro 7 XML (`fcp7`), read with craft-xml's bounded parser. xml = ["dep:craft-xml"] +# OpenTimelineIO JSON (`otio`), with serde_json. +otio = ["dep:serde_json"] [dependencies] serde = { workspace = true, optional = true } craft-xml = { workspace = true, optional = true } +serde_json = { workspace = true, optional = true } [dev-dependencies] proptest = { workspace = true } +serde_json = { workspace = true } [[test]] name = "it" -required-features = ["xml"] +required-features = ["xml", "otio"] diff --git a/crates/timeline/README.md b/crates/timeline/README.md index f53be17d6..6397d9d2e 100644 --- a/crates/timeline/README.md +++ b/crates/timeline/README.md @@ -16,6 +16,7 @@ text formats that are documents of their own. | `edl` | CMX 3600 edit decision lists: `parse` to the list as written (events, their lines and timecode fields, `M2`, `AUD`, clip-name, source-file, effect-name and locator comments), `write` for events the caller built, channel codes, reel names, the frame-rate guess. | | `fcp7` (feature `xml`) | Final Cut Pro 7 XML (`xmeml` 4 and 5, the dialect Premiere Pro reads and writes) to and from `model`. Read with craft-xml, strictly and bounded. | | `fcpxml` (feature `xml`) | FCPXML 1.9–1.11 to and from `model`: the primary storyline as V1, connected clips by lane, `ref-clip` compounds as nested sequences, rational times exact, transforms, opacity, volume and their keyframes, retiming through `timeMap`. | +| `otio` (feature `otio`) | OpenTimelineIO JSON to and from `model`: timelines, collections, stacks as nested sequences, gaps, transitions, markers, linear time warps and freeze frames. What OTIO has no schema for goes in `metadata.`, and the model's `ext` is merged into it, so an application's own data round-trips. | | crate root | `Format` (EDL, xmeml, FCPXML, OTIO, AAF, OMF), `detect_text`, `decode_text`. | No file I/O. EDL and ALE give the document as the format writes it. The formats that describe a @@ -23,7 +24,7 @@ whole edit read into and write from `model`; mapping `model` onto an application stays in the application. AAF and OMF are in `craft-aaf`. -Feature `xml`: the XML formats, on craft-xml. Feature `serde`: `Serialize` / `Deserialize` for the time, report and `Format` types. +Feature `xml`: the XML formats, on craft-xml. Feature `otio`: OpenTimelineIO, on serde_json. Feature `serde`: `Serialize` / `Deserialize` for the time, report and `Format` types. ## Specifications diff --git a/crates/timeline/src/fcpxml.rs b/crates/timeline/src/fcpxml.rs index 81342f70c..198574718 100644 --- a/crates/timeline/src/fcpxml.rs +++ b/crates/timeline/src/fcpxml.rs @@ -1325,7 +1325,7 @@ mod tests { "# ); let mut report = Report::default(); - let iopts = ImportOptions { base_dir: None, catalog: &crate::fx::Standard, namespace: "test" }; + let iopts = ImportOptions { base_dir: None, name: None, catalog: &crate::fx::Standard, namespace: "test" }; let imported = import(&xml, &iopts, &mut report).expect("import succeeds"); let xopts = ExportOptions { relative_to: None, diff --git a/crates/timeline/src/lib.rs b/crates/timeline/src/lib.rs index 4de152282..b638fd3ec 100644 --- a/crates/timeline/src/lib.rs +++ b/crates/timeline/src/lib.rs @@ -15,6 +15,7 @@ //! - [`edl`]: CMX 3600 edit decision lists. //! - [`fcp7`] (feature `xml`): Final Cut Pro 7 XML (`xmeml`). //! - [`fcpxml`] (feature `xml`): FCPXML 1.9–1.11. +//! - [`otio`] (feature `otio`): OpenTimelineIO JSON. //! - [`Format`], [`detect_text`], [`decode_text`]: which interchange format a document is. //! //! No file I/O: documents are text or bytes, and media references are strings. Each format module @@ -30,6 +31,8 @@ pub mod fcp7; pub mod fcpxml; pub mod fx; pub mod model; +#[cfg(feature = "otio")] +pub mod otio; pub mod path; pub mod rate; pub mod report; @@ -116,7 +119,7 @@ pub enum Error { } impl Error { - #[cfg_attr(not(feature = "xml"), allow(dead_code))] + #[cfg_attr(not(any(feature = "xml", feature = "otio")), allow(dead_code))] pub(crate) fn parse(format: Format, message: impl Into) -> Error { Error::Parse { format: format.name(), message: message.into() } } @@ -142,6 +145,8 @@ pub type Result = std::result::Result; pub struct ImportOptions<'a> { /// Directory relative media paths are resolved against (normally the document's directory). pub base_dir: Option<&'a str>, + /// Name for a timeline the document leaves unnamed. + pub name: Option<&'a str>, /// The application's effects and generators (see [`model::Catalog`]; [`fx::Standard`] knows /// only what the formats name themselves). pub catalog: &'a dyn model::Catalog, diff --git a/crates/timeline/src/model.rs b/crates/timeline/src/model.rs index 6adff43dd..3f8db8c1c 100644 --- a/crates/timeline/src/model.rs +++ b/crates/timeline/src/model.rs @@ -14,7 +14,7 @@ //! supplies. //! - **Generators** other than the ones every editor has: [`Generator::Other`] carries a name and //! an opaque payload. -//! - **Extensions.** `ext` on items, sequences, tracks, clips, transitions and markers is JSON +//! - **Extensions.** `ext` on items (a sequence's is its item's), tracks, clips, transitions and markers is JSON //! text the application owns. A format that can carry application metadata (OpenTimelineIO) //! writes it under the application's namespace and gives it back on reading; the others ignore it. @@ -540,12 +540,11 @@ pub struct Sequence { pub markers: Vec, /// Timecode of the first frame, in frames. pub start_timecode: i64, - pub ext: Ext, } impl Sequence { pub fn new(settings: SequenceSettings) -> Self { - Self { settings, video_tracks: Vec::new(), audio_tracks: Vec::new(), markers: Vec::new(), start_timecode: 0, ext: None } + Self { settings, video_tracks: Vec::new(), audio_tracks: Vec::new(), markers: Vec::new(), start_timecode: 0 } } /// The end of the last clip. pub fn duration(&self) -> Tick { diff --git a/crates/timeline/tests/it/fcp7.rs b/crates/timeline/tests/it/fcp7.rs index dfa3cc186..00d0704ec 100644 --- a/crates/timeline/tests/it/fcp7.rs +++ b/crates/timeline/tests/it/fcp7.rs @@ -723,7 +723,7 @@ fn absurd_timebase_is_bounded() { ); let (imported, _) = fcp7_import(&doc, None); for seq in &imported.sequences { - for format in [Format::Fcp7Xml, Format::Fcpxml] { + for format in [Format::Fcp7Xml, Format::Fcpxml, Format::Otio] { export(&imported.project, *seq, format, &xopts()).unwrap_or_else(|e| panic!("{format:?}: {e}")); } } @@ -742,7 +742,7 @@ fn absurd_start_timecode_is_bounded() { "#; let (imported, report) = fcp7_import(doc, None); for seq in &imported.sequences { - for format in [Format::Fcp7Xml, Format::Fcpxml] { + for format in [Format::Fcp7Xml, Format::Fcpxml, Format::Otio] { export(&imported.project, *seq, format, &xopts()).unwrap_or_else(|e| panic!("{format:?}: {e}")); } assert_eq!(imported.project.sequence(*seq).map(|s| s.start_timecode), Some(0)); diff --git a/crates/timeline/tests/it/main.rs b/crates/timeline/tests/it/main.rs index 12e73ea72..f8d222b28 100644 --- a/crates/timeline/tests/it/main.rs +++ b/crates/timeline/tests/it/main.rs @@ -2,4 +2,5 @@ mod fcp7; mod fcpxml; +mod otio; mod support; diff --git a/crates/timeline/tests/it/support.rs b/crates/timeline/tests/it/support.rs index 4a3247d78..1afb703a0 100644 --- a/crates/timeline/tests/it/support.rs +++ b/crates/timeline/tests/it/support.rs @@ -11,7 +11,7 @@ use craft_timeline::model::{ TransitionId, VideoStreamInfo, }; use craft_timeline::time::{FrameRate, TICKS_PER_SECOND, Tick, TimeRange}; -use craft_timeline::{Error, ExportOptions, Format, ImportOptions, Imported, Report, fcp7, fcpxml}; +use craft_timeline::{Error, ExportOptions, Format, ImportOptions, Imported, Report, fcp7, fcpxml, otio}; /// [`Standard`] and a few effects an application adds: a blur, a grade, two transitions. pub struct TestCatalog; @@ -31,6 +31,7 @@ fn extra() -> Vec { def("lumetri", "Lumetri Color", EffectKind::Video, "Color Correction", vec![float("exposure", "Exposure", 0.0)]), def("dip_to_white", "Dip to White", EffectKind::VideoTransition, "Dissolve", vec![]), def("push", "Push", EffectKind::VideoTransition, "Slide", vec![]), + def("iris_round", "Iris Round", EffectKind::VideoTransition, "Iris", vec![]), ] } @@ -49,7 +50,7 @@ impl Catalog for TestCatalog { pub const NS: &str = "testapp"; pub fn iopts(base_dir: Option<&str>) -> ImportOptions<'_> { - ImportOptions { base_dir, catalog: &TestCatalog, namespace: NS } + ImportOptions { base_dir, name: None, catalog: &TestCatalog, namespace: NS } } pub fn xopts() -> ExportOptions<'static> { @@ -307,6 +308,7 @@ pub fn import(bytes: &[u8], format: Format, base_dir: Option<&str>) -> Result<(I let imported = match format { Format::Fcp7Xml => fcp7::import(&text, &opts, &mut report), Format::Fcpxml => fcpxml::import(&text, &opts, &mut report), + Format::Otio => otio::import(&text, &opts, &mut report), _ => Err(Error::Other(format!("{} is not read here", format.name()))), }?; Ok((imported, report)) @@ -318,6 +320,7 @@ pub fn export(p: &Project, seq: ItemId, format: Format, opts: &ExportOptions) -> let text = match format { Format::Fcp7Xml => fcp7::export(p, seq, opts, &mut report), Format::Fcpxml => fcpxml::export(p, seq, opts, &mut report), + Format::Otio => otio::export(p, seq, opts, &mut report), _ => Err(Error::Other(format!("{} is not written here", format.name()))), }?; Ok((text.into_bytes(), report)) From db762f02f9e3d9806ae1b9b2e3f7ba7328cdd217 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:30:17 -0500 Subject: [PATCH 08/10] craft-rtf, craft-webp: restore the outlook and vp8 features later squash merges dropped (#1072) #991 (craft-codepage) was squash-merged from a branch that predated #961 and carried its older crates/rtf/Cargo.toml, dropping the outlook feature, its craft-mime dependency and the proptest dev-dependency; the outlook module was left behind an undefined cfg. #1028 did the same to crates/webp/Cargo.toml after #1021, dropping the vp8 feature and its craft-video dependency. --- Cargo.lock | 3 +++ crates/rtf/Cargo.toml | 10 ++++++++++ crates/webp/Cargo.toml | 3 +++ 3 files changed, 16 insertions(+) diff --git a/Cargo.lock b/Cargo.lock index 7bcbe4339..9f1beb697 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1776,7 +1776,9 @@ name = "craft-rtf" version = "0.1.0" dependencies = [ "craft-codepage", + "craft-mime", "encoding_rs", + "proptest", ] [[package]] @@ -2003,6 +2005,7 @@ dependencies = [ name = "craft-webp" version = "0.1.0" dependencies = [ + "craft-video", "image", "thiserror 2.0.21", ] diff --git a/crates/rtf/Cargo.toml b/crates/rtf/Cargo.toml index 59f139917..ecff5a2fe 100644 --- a/crates/rtf/Cargo.toml +++ b/crates/rtf/Cargo.toml @@ -10,7 +10,17 @@ repository.workspace = true [lints] workspace = true +[features] +# `outlook`: compressed RTF ([MS-OXRTFCP]) and the text or encapsulated HTML ([MS-OXRTFEX]) of a +# mail item's body. +outlook = ["dep:craft-mime"] + [dependencies] craft-codepage = { workspace = true } +# Character sets by Windows code page number, for the `outlook` module. +craft-mime = { workspace = true, optional = true } # The East Asian multibyte code pages (932, 936, 949, 950). encoding_rs = { workspace = true } + +[dev-dependencies] +proptest = { workspace = true } diff --git a/crates/webp/Cargo.toml b/crates/webp/Cargo.toml index 5a0901a3a..c5f7364b6 100644 --- a/crates/webp/Cargo.toml +++ b/crates/webp/Cargo.toml @@ -13,10 +13,13 @@ workspace = true [features] # Decoding every frame through the `image` crate's WebP decoder, with bounded memory. frames = ["dep:image"] +# Lossy animations from RGBA frames through craft-video's VP8 key-frame encoder. +vp8 = ["dep:craft-video"] [dependencies] thiserror = { workspace = true } image = { version = "0.25", default-features = false, features = ["webp"], optional = true } +craft-video = { workspace = true, optional = true, features = ["vp8-encode"] } [dev-dependencies] # Decodes the files the writer makes and writes still test files; not a product dependency. From d93991706ba27a2789a70c5151fe1feda08b210c Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:30:41 -0500 Subject: [PATCH 09/10] craft-math: operator names take their LaTeX scripts directly again (restores #1019) (#1068) #1019 (22161d2d) wrote \sin^2\theta as \sin ^{2}{\theta } so it reads back as the same tree. The squash merge of #1038, whose branch predated it, put math_latex.rs and the corpus test back as they were before. This applies #1019's change again, unchanged, with its test. --- crates/math/src/math_latex.rs | 26 +++++++++++++++++++++++++- crates/math/tests/corpus.rs | 10 ++-------- 2 files changed, 27 insertions(+), 9 deletions(-) diff --git a/crates/math/src/math_latex.rs b/crates/math/src/math_latex.rs index 5f442f437..24c0d5541 100644 --- a/crates/math/src/math_latex.rs +++ b/crates/math/src/math_latex.rs @@ -982,7 +982,12 @@ fn script_node(kind: ScriptKind, base: &[MNode], sub: &[MNode], sup: &[MNode], o braced(base, output, depth); return; } - braced(base, output, depth); + // An operator name takes its scripts directly (`\sin^{2}`): in braces it would no longer apply + // to what follows it when read back. + match base { + [MNode::Run(r)] if is_function_name(r) => run_node(r, output), + _ => braced(base, output, depth), + } if matches!(kind, ScriptKind::Sub | ScriptKind::SubSup) { output.push('_'); braced(sub, output, depth); @@ -993,6 +998,11 @@ fn script_node(kind: ScriptKind, base: &[MNode], sub: &[MNode], sup: &[MNode], o } } +/// Is `r` an operator name the writer writes as a command (`\sin`, `\log`)? +fn is_function_name(r: &MRun) -> bool { + !r.nor && r.sty == Some(MSty::Plain) && r.scr == MScr::Roman && FUNCTION_NAMES.contains(&r.text.as_str()) +} + fn radical_node(degree: &[MNode], degree_hidden: bool, expression: &[MNode], output: &mut String, depth: usize) { output.push_str("\\sqrt"); if !degree_hidden && !degree.is_empty() { @@ -1289,4 +1299,18 @@ mod tests { // The writer keeps its own names. assert_eq!(to_latex(&parse_latex(r"\infinity")), to_latex(&parse_latex(r"\infty"))); } + + /// An operator name with a script still applies to its argument when the LaTeX is read back. + #[test] + fn operator_names_take_their_scripts_directly() { + for src in [r"\sin^2\theta+\cos^2\theta=1", r"\log_2 x", r"\ln^{3}(x+1)"] { + let tree = parse_latex(src); + let tex = to_latex(&tree); + assert_eq!(crate::math::to_linear(&parse_latex(&tex)), crate::math::to_linear(&tree), "{src}: {tex}"); + assert!(!tex.contains(r"{\sin }") && !tex.contains(r"{\log }"), "{tex}"); + } + assert!(to_latex(&parse_latex(r"\sin^2\theta")).starts_with(r"\sin ^{2}")); + // Other bases keep their braces. + assert_eq!(to_latex(&parse_latex("x^2")), "{x}^{2}"); + } } diff --git a/crates/math/tests/corpus.rs b/crates/math/tests/corpus.rs index 90a7aaae7..383b77783 100644 --- a/crates/math/tests/corpus.rs +++ b/crates/math/tests/corpus.rs @@ -18,10 +18,6 @@ fn equations() -> Vec<(String, math::Arg, bool)> { v } -/// Equations whose LaTeX this crate writes in a form it reads back differently (a function -/// raised to a power: `\sin^2\theta` comes back as `(\sin)^2 θ`). Known; kept visible here. -const LATEX_ROUND_TRIP_GAPS: &[&str] = &["func"]; - #[test] fn every_corpus_equation_reads_without_unknown_commands() { for (id, tree, _) in equations() { @@ -54,10 +50,8 @@ fn every_corpus_equation_survives_mathml_and_latex() { let lin = math::to_linear(&tree); let ml = mathml::to_mathml(&tree, display, ""); assert_eq!(mathml::parse(&ml).and_then(|m| mathml::read(&m)), Some((lin.clone(), display)), "{id}: {ml}"); - if !LATEX_ROUND_TRIP_GAPS.contains(&id.as_str()) { - let tex = math_latex::to_latex(&tree); - assert_eq!(math::to_linear(&math_latex::parse_latex(&tex)), lin, "{id}: {tex}"); - } + let tex = math_latex::to_latex(&tree); + assert_eq!(math::to_linear(&math_latex::parse_latex(&tex)), lin, "{id}: {tex}"); } } From 865671e1fd938baccccfbbb93a7df61fd5786fb5 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 11 Nov 2026 21:32:31 -0500 Subject: [PATCH 10/10] craft-docx: equations through craft_math::omml (#1035) * Add craft-wordmodel: WordCraft's document model without its editing code WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX, RTF, ODT and other text-format crates can read into and write from one model. Module paths, type names and serde attributes are unchanged: an app that re-exports the crate keeps its native file format. Left in the app: range editing (edit.rs), IME text and equation editing. Not model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs. * craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs The DOCX reader and writer walk an equation's runs. The walk and the child accessors it uses were in WordCraft's equation editor (math_edit.rs), which stays in the app; they are structure, not editing. * craft-wordmodel: keep the document operations (edit.rs) with the model The DOCX and RTF readers use them: alternative-format chunks are merged into a document with adopt_blocks, and a reader formats ranges with format_range and inserts objects. They operate on the document, not on a view, so they belong beside it. The tests that drove them come back with them; the paragraph walks stay in walk.rs and edit re-exports them. base64 0.23, as WordCraft and craft-offcrypto use. * craft-docx: WordCraft's crates/docx as it is at wordcraft 07f97737 The source, unchanged, so the next commit shows only what the move changes. This commit does not build on its own (it still names wordcraft-doc and wordcraft-geom, and is not a workspace member yet). * Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath The math tree, LaTeX, linear-format, symbol and tree-walk modules come from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at 07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f / 07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in WordCraft; the rest moved with the code. mathml::parse and omml::parse/read_str/write_str are new: string entry points for callers that have no tree of their own. * craft-math: read MathLive's and other spellings of LaTeX math commands \abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel, \backprime, \infinity, \cross, \differentialD, \exponentialE and \imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox, \mathnormal and \underleftarrow / \underrightarrow / \underleftrightarrow were unknown and read as literal text. They are read as equaSHion's typed-input parser reads them (storytold/craft-libs#29). * craft-math: equaSHion's equation corpus as test data corpus/equations.tsv, corpus/latex.tsv and the OMML samples of storytold/craft-libs#29 (7a6ef6b9), unmodified. * craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README The corpus equations go through OMML, MathML and LaTeX and back; the OMML samples are written back as kept and from the tree. GridCraft's OMML to LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the same through the tree. omml::parse and read_str are tested against other prefixes, the strict namespace, kept OMML, junk, the element cap and deep nesting. * craft-docx: on craft-wordmodel, craft-opc, craft-xml and craft-offcrypto - The package (zip reading, part names, relationships, content types, zip writing) is craft-opc's; the module keeps WordCraft's bookkeeping (passthrough keys, the embedded-part manifest) and its own Rel/Rels shape. - Parts are parsed with craft-xml's canonical names (the Strict drawing names and mc:Choice/@Requires through its hooks), then given the tree shape the reader was written against. - Encryption and Restrict Editing hashes are craft-offcrypto's; this crate maps its errors and the w:documentProtection attributes. - wordcraft-doc is craft-wordmodel; wordcraft-geom's helpers are its geom. Tests that covered what now lives in craft-opc and craft-offcrypto are left to those crates; the DIFAT regression is craft-cfb's. Test-only zip use follows zip 9's fallible names. The attribute-cap test times the read only. * craft-wordmodel: the math modules, symbol_fonts and Rgb are craft-math's math, math_latex, math_linear, math_symbols, math_tree and symbol_fonts are re-exported from craft-math at the paths they had, and props::Rgb is craft-math's Rgb (TextColor stays), so the serde forms and the app's imports do not change. Their tests are in craft-math. * craft-docx: equations (OMML) are read and written by craft_math::omml read/math.rs and write/math.rs keep their functions as adapters: the reader's tree is handed to craft-math as a craft-xml tree, and the writer's output is written into this crate's writer. Their tests are in craft-math. --------- Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com> --- crates/docx/Cargo.toml | 1 + crates/docx/README.md | 3 +- crates/docx/src/read/math.rs | 507 +---------- crates/docx/src/write/math.rs | 481 +--------- crates/wordmodel/Cargo.toml | 3 + crates/wordmodel/src/lib.rs | 7 +- crates/wordmodel/src/math.rs | 1138 ----------------------- crates/wordmodel/src/math_latex.rs | 1242 -------------------------- crates/wordmodel/src/math_linear.rs | 762 ---------------- crates/wordmodel/src/math_symbols.rs | 390 -------- crates/wordmodel/src/math_tree.rs | 104 --- crates/wordmodel/src/props/color.rs | 134 +-- crates/wordmodel/src/symbol_fonts.rs | 316 ------- 13 files changed, 45 insertions(+), 5043 deletions(-) delete mode 100644 crates/wordmodel/src/math.rs delete mode 100644 crates/wordmodel/src/math_latex.rs delete mode 100644 crates/wordmodel/src/math_linear.rs delete mode 100644 crates/wordmodel/src/math_symbols.rs delete mode 100644 crates/wordmodel/src/math_tree.rs delete mode 100644 crates/wordmodel/src/symbol_fonts.rs diff --git a/crates/docx/Cargo.toml b/crates/docx/Cargo.toml index 046d81589..cdc6144eb 100644 --- a/crates/docx/Cargo.toml +++ b/crates/docx/Cargo.toml @@ -11,6 +11,7 @@ repository.workspace = true workspace = true [dependencies] +craft-math = { workspace = true } craft-offcrypto = { workspace = true } craft-opc = { workspace = true } craft-wordmodel = { workspace = true } diff --git a/crates/docx/README.md b/crates/docx/README.md index 0e5cbf11e..8dab3fc55 100644 --- a/crates/docx/README.md +++ b/crates/docx/README.md @@ -42,8 +42,9 @@ written back as it was read. `write(doc)`, `write_as(doc, Flavor)` for `.docx`, `.docm`, `.dotx` and `.dotm`; `write_flat` and `write_wordml2003` for the XML forms; `encrypt` for a password. +Equations (OMML) are read and written by `craft_math::omml`. + ## Not here yet - DrawingML (`a:`) shape fills, lines, custom geometry and colours are still read with this crate's own code; they move to `craft-drawingml`. -- Equations (OMML) are read and written by this crate; they move to `craft-math`. diff --git a/crates/docx/src/read/math.rs b/crates/docx/src/read/math.rs index 27c78122f..e42edb646 100644 --- a/crates/docx/src/read/math.rs +++ b/crates/docx/src/read/math.rs @@ -1,496 +1,41 @@ -//! Office Math (OMML, ECMA-376 Part 1 §22.1) → [`craft_wordmodel::math`] tree. +//! Office Math (OMML, ECMA-376 Part 1 §22.1) read with `craft_math::omml`. -use craft_wordmodel::math::{ - Arg, BrkBin, BrkBinSub, ColJc, FracKind, LimLoc, MAX_DEPTH, MNode, MRun, MScr, MSty, Math, MathJc, MathProps, ScriptKind, merge_runs, -}; +use craft_wordmodel::math::{Math, MathJc, MathProps}; use craft_wordmodel::props::Rgb; -use craft_wordmodel::symbol_fonts; -use crate::xml::El; +use crate::xml::{El, Node}; -/// An `m:oMath` element as an equation (structure plus its source XML). -pub fn read_omath(e: &El, jc: MathJc) -> Math { - Math { nodes: merge_runs(arg(e, 0)), omml: e.to_xml(), jc, linear: false } -} - -/// A colour every math run shares, taken off the runs: it is the equation's own colour, as -/// [`write_equation`](crate::write::math::write_equation) spreads it (#1467). The source OMML -/// is kept: the writer replaces it only when the colour changes. -pub fn lift_color(math: &mut Math) -> Option { - let (mut shared, mut uniform) = (None, true); - craft_wordmodel::math_tree::each_run_mut(&mut math.nodes, &mut |r| match shared { - None => shared = Some(r.color), - Some(c) => uniform &= c == r.color, - }); - let c = shared.flatten().filter(|_| uniform)?; - craft_wordmodel::math_tree::each_run_mut(&mut math.nodes, &mut |r| r.color = None); - Some(c) -} - -/// `m:oMathParaPr/m:jc`. -pub fn para_jc(para: &El) -> MathJc { - match para.child("m:oMathParaPr").and_then(|p| p.child("m:jc")).and_then(|j| j.attr("m:val")) { - Some("left") => MathJc::Left, - Some("right") => MathJc::Right, - Some("center") => MathJc::Center, - _ => MathJc::CenterGroup, - } -} - -/// The settings part's `m:mathPr`: how display equations wrap. -pub fn read_math_pr(e: &El) -> MathProps { - let mut m = MathProps::default(); - let pr = Some(e); - m.brk_bin = match val(pr, "m:brkBin") { - Some("after") => BrkBin::After, - Some("repeat") => BrkBin::Repeat, - _ => BrkBin::Before, - }; - m.brk_bin_sub = match val(pr, "m:brkBinSub") { - Some("-+") => BrkBinSub::MinusPlus, - Some("+-") => BrkBinSub::PlusMinus, - _ => BrkBinSub::MinusMinus, - }; - if let Some(v) = e.child("m:wrapIndent").and_then(|c| crate::units::tw(c, "m:val")).filter(|v| v.is_finite() && *v >= 0.0) { - m.wrap_indent = v.min(1584.0); - } - m.wrap_right = prop(pr, "m:wrapRight", false); - m -} - -/// OMML on/off property: absent `m:val` = on. -fn on(e: &El) -> bool { - match e.attr("m:val") { - None => true, - Some(v) => !matches!(v.trim(), "0" | "false" | "off" | "False" | "FALSE"), - } -} - -/// An on/off property `name` in the properties element `pr` (absent = `default`). -fn prop(pr: Option<&El>, name: &str, default: bool) -> bool { - pr.and_then(|p| p.child(name)).map(on).unwrap_or(default) -} - -/// A character property (`m:chr`, `m:begChr`…): `None` when absent, `Some(None)` when empty. -fn chr(pr: Option<&El>, name: &str) -> Option> { - let c = pr?.child(name)?; - match c.attr("m:val") { - Some(v) => Some(v.chars().next()), - None => Some(None), - } -} - -fn val<'a>(pr: Option<&'a El>, name: &str) -> Option<&'a str> { - pr.and_then(|p| p.child(name)).and_then(|c| c.attr("m:val")) -} - -/// The argument held by child `name` of `e`. -fn sub_arg(e: &El, name: &str, depth: usize) -> Arg { - e.child(name).map(|c| arg(c, depth + 1)).unwrap_or_default() -} - -/// Read the math content of an element (`m:oMath`, `m:e`, `m:num`…). -fn arg(e: &El, depth: usize) -> Arg { - let mut out = Vec::new(); - if depth > MAX_DEPTH { - return out; - } - for k in e.els() { - node(k, depth, &mut out); +/// This reader's tree as `craft-xml`'s, which `craft_math::omml` reads. Both use the canonical +/// prefixes (`m:`, `w:`, `mc:`). +// shortcut: copies the equation's subtree; goes when the reader reads craft_xml::El directly. +fn to_craft(e: &El) -> craft_xml::El { + let mut out = craft_xml::El { name: e.name.clone(), attrs: e.attrs.clone(), children: Vec::with_capacity(e.kids.len()) }; + for k in &e.kids { + out.children.push(match k { + Node::El(c) => craft_xml::Node::El(to_craft(c)), + Node::Text(t) => craft_xml::Node::Text(t.clone()), + }); } out } -fn node(k: &El, depth: usize, out: &mut Arg) { - let d = depth + 1; - match k.name.as_str() { - "m:r" => { - if let Some(r) = run(k) { - out.push(MNode::Run(r)); - } - } - // Word puts ordinary runs in math zones for normal text in some files. - "w:r" => { - let text = run_text(k); - if !text.is_empty() { - let mut r = MRun { text, nor: true, sty: Some(MSty::Plain), ..Default::default() }; - wrpr(k.child("w:rPr"), &mut r); - out.push(MNode::Run(r)); - } - } - "m:f" => { - let pr = k.child("m:fPr"); - let kind = match val(pr, "m:type") { - Some("skw") => FracKind::Skewed, - Some("lin") => FracKind::Linear, - Some("noBar") => FracKind::NoBar, - _ => FracKind::Bar, - }; - out.push(MNode::Frac { kind, num: sub_arg(k, "m:num", d), den: sub_arg(k, "m:den", d) }); - } - "m:sSup" => out.push(MNode::Script { kind: ScriptKind::Sup, base: sub_arg(k, "m:e", d), sub: Vec::new(), sup: sub_arg(k, "m:sup", d) }), - "m:sSub" => out.push(MNode::Script { kind: ScriptKind::Sub, base: sub_arg(k, "m:e", d), sub: sub_arg(k, "m:sub", d), sup: Vec::new() }), - "m:sSubSup" => { - out.push(MNode::Script { kind: ScriptKind::SubSup, base: sub_arg(k, "m:e", d), sub: sub_arg(k, "m:sub", d), sup: sub_arg(k, "m:sup", d) }) - } - "m:sPre" => { - out.push(MNode::Script { kind: ScriptKind::Pre, base: sub_arg(k, "m:e", d), sub: sub_arg(k, "m:sub", d), sup: sub_arg(k, "m:sup", d) }) - } - "m:rad" => { - let pr = k.child("m:radPr"); - out.push(MNode::Rad { deg: sub_arg(k, "m:deg", d), deg_hide: prop(pr, "m:degHide", false), e: sub_arg(k, "m:e", d) }); - } - "m:nary" => { - let pr = k.child("m:naryPr"); - let chr = chr(pr, "m:chr").map(|c| c.unwrap_or('∫')).unwrap_or('∫'); - let lim_loc = match val(pr, "m:limLoc") { - Some("undOvr") => Some(LimLoc::UndOvr), - Some("subSup") => Some(LimLoc::SubSup), - _ => None, - }; - out.push(MNode::Nary { - chr, - lim_loc, - grow: prop(pr, "m:grow", false), - sub_hide: prop(pr, "m:subHide", false), - sup_hide: prop(pr, "m:supHide", false), - sub: sub_arg(k, "m:sub", d), - sup: sub_arg(k, "m:sup", d), - e: sub_arg(k, "m:e", d), - }); - } - "m:d" => { - let pr = k.child("m:dPr"); - let beg = chr(pr, "m:begChr").unwrap_or(Some('(')); - let end = chr(pr, "m:endChr").unwrap_or(Some(')')); - let sep = chr(pr, "m:sepChr").unwrap_or(Some('|')); - let elems: Vec = k.children("m:e").map(|e| arg(e, d)).collect(); - out.push(MNode::Delim { - beg, - end, - sep, - grow: prop(pr, "m:grow", true), - shp_match: val(pr, "m:shp") == Some("match"), - elems: if elems.is_empty() { vec![Vec::new()] } else { elems }, - }); - } - "m:func" => out.push(MNode::Func { name: sub_arg(k, "m:fName", d), e: sub_arg(k, "m:e", d) }), - "m:limLow" => out.push(MNode::Lim { upper: false, e: sub_arg(k, "m:e", d), lim: sub_arg(k, "m:lim", d) }), - "m:limUpp" => out.push(MNode::Lim { upper: true, e: sub_arg(k, "m:e", d), lim: sub_arg(k, "m:lim", d) }), - "m:acc" => { - let c = chr(k.child("m:accPr"), "m:chr").map(|c| c.unwrap_or('\u{302}')).unwrap_or('\u{302}'); - out.push(MNode::Acc { chr: c, e: sub_arg(k, "m:e", d) }); - } - "m:bar" => out.push(MNode::Bar { top: val(k.child("m:barPr"), "m:pos") == Some("top"), e: sub_arg(k, "m:e", d) }), - "m:borderBox" => { - let pr = k.child("m:borderBoxPr"); - out.push(MNode::BorderBox { - hide: [prop(pr, "m:hideTop", false), prop(pr, "m:hideBot", false), prop(pr, "m:hideLeft", false), prop(pr, "m:hideRight", false)], - strike: [ - prop(pr, "m:strikeH", false), - prop(pr, "m:strikeV", false), - prop(pr, "m:strikeBLTR", false), - prop(pr, "m:strikeTLBR", false), - ], - e: sub_arg(k, "m:e", d), - }); - } - "m:box" => out.push(MNode::Boxed { e: sub_arg(k, "m:e", d) }), - "m:groupChr" => { - let pr = k.child("m:groupChrPr"); - let c = chr(pr, "m:chr").map(|c| c.unwrap_or('\u{23DF}')).unwrap_or('\u{23DF}'); - out.push(MNode::GroupChr { chr: c, top: val(pr, "m:pos") == Some("top"), e: sub_arg(k, "m:e", d) }); - } - "m:eqArr" => out.push(MNode::EqArr { rows: k.children("m:e").map(|e| arg(e, d)).collect() }), - "m:m" => { - let mut col_jc = Vec::new(); - if let Some(mcs) = k.child("m:mPr").and_then(|p| p.child("m:mcs")) { - for mc in mcs.children("m:mc") { - let pr = mc.child("m:mcPr"); - let jc = match val(pr, "m:mcJc") { - Some("left") => ColJc::Left, - Some("right") => ColJc::Right, - _ => ColJc::Center, - }; - let count = val(pr, "m:count").and_then(|c| c.trim().parse::().ok()).unwrap_or(1).clamp(1, 64); - for _ in 0..count { - if col_jc.len() < 64 { - col_jc.push(jc); - } - } - } - } - let rows = k.children("m:mr").take(1000).map(|r| r.children("m:e").take(64).map(|e| arg(e, d)).collect()).collect(); - out.push(MNode::Matrix { rows, col_jc }); - } - "m:phant" => { - let pr = k.child("m:phantPr"); - out.push(MNode::Phant { - show: prop(pr, "m:show", true), - zero_wid: prop(pr, "m:zeroWid", false), - zero_asc: prop(pr, "m:zeroAsc", false), - zero_desc: prop(pr, "m:zeroDesc", false), - e: sub_arg(k, "m:e", d), - }); - } - // Tracked insertions show; deletions don't. Hyperlinks, simple fields and direction - // wrappers (EG_PContentMath, §22.1.2.77) keep their runs (#1273). - "w:ins" | "w:moveTo" | "w:smartTag" | "w:customXml" | "w:hyperlink" | "w:fldSimple" | "w:dir" | "w:bdo" | "m:oMath" if depth <= MAX_DEPTH => { - for c in k.els() { - node(c, d, out); - } - } - // Markup compatibility: the math choice (we read math), else the fallback. - "mc:AlternateContent" if depth <= MAX_DEPTH => { - if let Some(c) = k.child("mc:Choice").or_else(|| k.child("mc:Fallback")) { - for c in c.els() { - node(c, d, out); - } - } - } - "w:sdt" => { - if let Some(c) = k.child("w:sdtContent") { - for c in c.els() { - node(c, d, out); - } - } - } - _ => {} - } +/// An `m:oMath` element as an equation (structure plus its source XML). +pub fn read_omath(e: &El, jc: MathJc) -> Math { + craft_math::omml::read_omath(&to_craft(e), jc) } -/// `m:t` (and `w:t`) text of a run, with symbol characters (`w:sym`) and special hyphens. -/// Text in a symbol font (Symbol, Wingdings…) becomes its Unicode equivalent: the math font has -/// no glyphs at those private-use codes (#1273). -fn run_text(r: &El) -> String { - let font = r.child("w:rPr").and_then(|p| p.child("w:rFonts")).and_then(|f| f.attr("w:ascii").or_else(|| f.attr("w:hAnsi"))).unwrap_or(""); - let mut s = String::new(); - for c in r.els() { - match c.name.as_str() { - "m:t" | "w:t" => s.push_str(&symbol_fonts::map_text(font, &c.text())), - "w:tab" => s.push('\t'), - "w:noBreakHyphen" => s.push('\u{2011}'), - "w:sym" => { - let ch = c.attr("w:char").and_then(|h| u32::from_str_radix(h.trim(), 16).ok()).and_then(char::from_u32).filter(|c| *c as u32 >= 0x20); - if let Some(ch) = ch { - let f = c.attr("w:font").unwrap_or(font); - s.push(symbol_fonts::symbol_char(f, ch).unwrap_or(ch)); - } - } - _ => {} - } - } - s +/// The colour every run of `math` shares, taken off the runs (the run around the equation +/// carries it). +pub fn lift_color(math: &mut Math) -> Option { + craft_math::omml::lift_color(math) } -fn run(r: &El) -> Option { - let text = run_text(r); - if text.is_empty() { - return None; - } - let mut out = MRun { text, ..Default::default() }; - if let Some(pr) = r.child("m:rPr") { - out.sty = match pr.child("m:sty").and_then(|s| s.attr("m:val")) { - Some("p") => Some(MSty::Plain), - Some("b") => Some(MSty::Bold), - Some("i") => Some(MSty::Italic), - Some("bi") => Some(MSty::BoldItalic), - _ => None, - }; - out.scr = match pr.child("m:scr").and_then(|s| s.attr("m:val")) { - Some("script") => MScr::Script, - Some("fraktur") => MScr::Fraktur, - Some("double-struck") => MScr::DoubleStruck, - Some("sans-serif") => MScr::SansSerif, - Some("monospace") => MScr::Monospace, - _ => MScr::Roman, - }; - out.nor = pr.child("m:nor").is_some_and(on); - out.lit = pr.child("m:lit").is_some_and(on); - if let Some(b) = pr.child("m:brk") { - let aln = b.attr("m:alnAt").and_then(|v| v.trim().parse::().ok()).unwrap_or(0); - out.brk = Some(aln.min(255) as u8); - } - } - wrpr(r.child("w:rPr"), &mut out); - Some(out) +/// The justification an `m:oMathPara` gives its equations. +pub fn para_jc(para: &El) -> MathJc { + craft_math::omml::para_jc(&to_craft(para)) } -/// The run formatting a math run takes from `w:rPr`: size, colour, normal-text font and weight. -fn wrpr(pr: Option<&El>, out: &mut MRun) { - let Some(pr) = pr else { return }; - if let Some(sz) = pr.child("w:sz").and_then(|s| s.attr("w:val")).and_then(|v| v.trim().parse::().ok()) { - let pt = sz / 2.0; - if pt.is_finite() && (1.0..=1638.0).contains(&pt) { - out.size = Some(pt); - } - } - if let Some(c) = pr.child("w:color").and_then(|c| c.attr("w:val")).and_then(Rgb::parse) { - out.color = Some(c); - } - if out.nor { - // Symbol-font text was mapped to Unicode: it takes the text font. - if let Some(f) = - pr.child("w:rFonts").and_then(|f| f.attr("w:ascii").or_else(|| f.attr("w:hAnsi"))).filter(|f| !symbol_fonts::is_symbol_font(f)) - { - out.font = Some(f.to_string()); - } - let bold = pr.child("w:b").is_some_and(crate::units::on_off); - let italic = pr.child("w:i").is_some_and(crate::units::on_off); - if bold || italic { - out.sty = Some(match (bold, italic) { - (true, true) => MSty::BoldItalic, - (true, false) => MSty::Bold, - _ => MSty::Italic, - }); - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn omath(inner: &str) -> El { - let x = format!( - r#"{inner}"# - ); - crate::xml::parse(x.as_bytes()).unwrap() - } - - #[test] - fn reads_fraction_radical_and_scripts() { - let e = omath( - r#"x=-b±b2-4ac2a"#, - ); - let m = read_omath(&e, MathJc::CenterGroup); - assert_eq!(m.nodes.len(), 2); - let MNode::Frac { num, den, kind } = &m.nodes[1] else { panic!("{:#?}", m.nodes) }; - assert_eq!(*kind, FracKind::Bar); - assert_eq!(den, &vec![MNode::Run(MRun::new("2a"))]); - let MNode::Rad { deg_hide, e, .. } = &num[1] else { panic!() }; - assert!(*deg_hide); - assert!(matches!(e[0], MNode::Script { kind: ScriptKind::Sup, .. })); - assert_eq!(craft_wordmodel::math::to_linear(&m.nodes), "x=(-b±√(b^2-4ac))/2a"); - assert!(m.omml.starts_with("")); - } - - #[test] - fn reads_nary_delims_matrix_and_defaults() { - let e = omath( - r#"iiab"#, - ); - let m = read_omath(&e, MathJc::CenterGroup); - let MNode::Nary { chr, lim_loc, .. } = &m.nodes[0] else { panic!() }; - assert_eq!((*chr, *lim_loc), ('∑', Some(LimLoc::UndOvr))); - let MNode::Delim { beg, end, sep, elems, .. } = &m.nodes[1] else { panic!() }; - assert_eq!((*beg, *end, *sep, elems.len()), (Some('['), None, Some('|'), 2)); - assert!(matches!(m.nodes[2], MNode::Nary { chr: '∫', .. })); - assert!(matches!(&m.nodes[3], MNode::Matrix { rows, .. } if rows.len() == 1 && rows[0].len() == 2)); - } - - /// Every run's text in the tree, in reading order. - fn texts(m: &mut Math) -> Vec { - let mut out = Vec::new(); - craft_wordmodel::math_tree::each_run_mut(&mut m.nodes, &mut |r| out.push(r.text.clone())); - out - } - - fn t(s: &str) -> String { - format!("{s}") - } - - /// #1273: one equation using every Office Math element of ECMA-376 §22.1 (each argument - /// holding a distinct marker) keeps every argument's text: nothing is silently dropped. - #[test] - fn every_omml_element_keeps_its_content() { - let parts = [ - format!("{}", t("A1")), - format!("{}", t("B1")), - format!("{}", t("C1")), - format!("{}", t("D1")), - format!("{}{}", t("E1"), t("E2")), - format!("{}{}", t("F1"), t("F2")), - format!("{}{}", t("G1"), t("G2")), - format!("{}{}", t("H1"), t("H2")), - format!("{}", t("I1")), - format!("{}{}", t("J1"), t("J2")), - format!("{}{}", t("K1"), t("K2")), - format!("{}{}{}", t("L1"), t("L2"), t("L3")), - format!( - "{}{}{}", - t("M1"), - t("M2"), - t("M3") - ), - format!("{}", t("N1")), - format!("{}{}", t("O1"), t("O2")), - format!("{}{}{}", t("P1"), t("P2"), t("P3")), - format!("{}{}", t("Q1"), t("Q2")), - format!("{}{}{}", t("R1"), t("R2"), t("R3")), - format!("{}{}", t("S1"), t("S2")), - // Run-level wrappers allowed in math (EG_PContentMath): hyperlinks, simple fields, - // tracked insertions, content controls, markup compatibility. - format!("{}", t("T1")), - format!("{}", t("T2")), - format!("{}", t("T3")), - format!("{}", t("T4")), - format!( - "{}picture", - t("T5") - ), - "T6".to_string(), - ]; - let x = format!( - r#"{}"#, - parts.concat() - ); - let e = crate::xml::parse(x.as_bytes()).unwrap(); - let mut m = read_omath(&e, MathJc::CenterGroup); - let got = texts(&mut m).concat(); - let markers = [ - "A1", "B1", "C1", "D1", "E1", "E2", "F1", "F2", "G1", "G2", "H1", "H2", "I1", "J1", "J2", "K1", "K2", "L1", "L2", "L3", "M1", "M2", "M3", - "N1", "O1", "O2", "P1", "P2", "P3", "Q1", "Q2", "R1", "R2", "R3", "S1", "S2", "T1", "T2", "T3", "T4", "T5", "T6", - ]; - for k in markers { - assert!(got.contains(k), "{k} dropped from {got}"); - } - assert!(!got.contains("picture"), "the math choice wins over the fallback: {got}"); - } - - /// #1273: deleted text stays out; symbol characters and symbol-font text become Unicode. - #[test] - fn symbols_and_deletions_in_math() { - let e = omath( - r#"xgone"#, - ); - let mut m = read_omath(&e, MathJc::CenterGroup); - let got = texts(&mut m).concat(); - assert_eq!(got, "xαβ\u{2011}γ"); - let mut font = None; - craft_wordmodel::math_tree::each_run_mut(&mut m.nodes, &mut |r| { - if r.nor { - font = Some(r.font.clone()); - } - }); - assert_eq!(font, Some(None), "mapped symbol text takes the text font"); - } - - /// Hostile nesting through the wrappers is bounded (no stack overflow, no panic). - #[test] - fn deeply_nested_wrappers_are_bounded() { - // Deeper than the math limit, within the XML parser's own depth limit. - let n = 120; - let inner = format!("{}{}{}", "".repeat(n), t("deep"), "".repeat(n)); - let e = omath(&inner); - let _ = read_omath(&e, MathJc::CenterGroup); - let inner = format!( - "{}{}{}", - "".repeat(n), - t("deep"), - "".repeat(n) - ); - let e = omath(&inner); - let _ = read_omath(&e, MathJc::CenterGroup); - } +/// `m:mathPr` in the settings part. +pub fn read_math_pr(e: &El) -> MathProps { + craft_math::omml::read_math_pr(&to_craft(e)) } diff --git a/crates/docx/src/write/math.rs b/crates/docx/src/write/math.rs index 571c5978f..d22d548ef 100644 --- a/crates/docx/src/write/math.rs +++ b/crates/docx/src/write/math.rs @@ -1,486 +1,25 @@ -//! [`craft_wordmodel::math`] tree → Office Math (OMML). +//! Office Math (OMML) written with `craft_math::omml`. -use craft_wordmodel::math::{ - Arg, BrkBin, BrkBinSub, ColJc, FracKind, LimLoc, MAX_DEPTH, MAX_MATRIX_COLS, MAX_MATRIX_ROWS, MNode, MRun, MScr, MSty, Math, MathJc, MathProps, - ScriptKind, parse_linear, -}; +use craft_wordmodel::math::{Math, MathProps}; use craft_wordmodel::props::Rgb; use crate::xml::W; /// Write an equation: `m:oMath`, inside `m:oMathPara` for a display equation. The source OMML /// is written back unchanged when the equation came from a file. `color` (the equation's run -/// colour) goes on every math run without its own: OMML has no equation-level colour (#1467). +/// colour) goes on every math run without its own: OMML has no equation-level colour. pub fn write_equation(w: &mut W, linear: &str, display: bool, math: &Math, color: Option) { - let tinted; - let math = if color == omml_color(&math.omml) { - math - } else { - // The kept source XML has the old colour baked in: write from the nodes instead. - let mut m = math.clone(); - if m.nodes.is_empty() { - m.nodes = parse_linear(linear); - } - if let Some(c) = color { - craft_wordmodel::math_tree::each_run_mut(&mut m.nodes, &mut |r| { - r.color.get_or_insert(c); - }); - } - m.omml.clear(); - tinted = m; - &tinted - }; - if display { - w.open("m:oMathPara", &[]); - let jc = match math.jc { - MathJc::CenterGroup => None, - MathJc::Center => Some("center"), - MathJc::Left => Some("left"), - MathJc::Right => Some("right"), - }; - if let Some(jc) = jc { - w.open("m:oMathParaPr", &[]); - w.empty("m:jc", &[("m:val", jc)]); - w.close("m:oMathParaPr"); - } - } - if !math.omml.is_empty() && math.omml.starts_with(" Option { - if !display { - return None; - } - let [MNode::EqArr { rows }] = math.nodes.as_slice() else { return None }; - let mut out = rows.clone(); - let mut changed = false; - for row in out.iter_mut() { - if !craft_wordmodel::math::has_number_mark(row) { - continue; - } - *counter = counter.saturating_add(1); - // Nothing after the `#`: an automatic number. - let mut after = false; - let mut empty = true; - for n in row.iter() { - match n { - MNode::Run(r) if !after => { - if let Some(k) = r.text.find('#') { - after = true; - if !r.text.get(k + 1..).unwrap_or("").is_empty() { - empty = false; - } - } - } - _ if after => empty = false, - _ => {} - } - } - if empty { - let n = vec![MNode::Run(MRun::new(counter.to_string()))]; - row.push(MNode::Delim { beg: Some('('), end: Some(')'), sep: None, grow: true, shp_match: false, elems: vec![n] }); - changed = true; - } - } - changed.then(|| Math { nodes: vec![MNode::EqArr { rows: out }], omml: String::new(), ..math.clone() }) + craft_math::omml::resolve_numbers(math, display, counter) } -/// The equation colour the kept source OMML carries, as the reader lifts it. -fn omml_color(omml: &str) -> Option { - if omml.is_empty() { - return None; - } - let root = crate::xml::parse(wrap_ns(omml).as_bytes()).ok()?; - let e = root.child("m:oMath")?; - crate::read::math::lift_color(&mut crate::read::math::read_omath(e, MathJc::default())) -} - -/// The kept XML with namespace declarations, for validating it on its own. -fn wrap_ns(xml: &str) -> String { - let decl: String = crate::xml::body_ns().iter().filter(|(k, _)| k.starts_with("xmlns:")).map(|(k, v)| format!(" {k}=\"{v}\"")).collect(); - format!("{xml}") -} - -/// The settings part's `m:mathPr` (CT_MathPr order; `m:wrapIndent` and `m:wrapRight` are a choice). +/// `m:mathPr`. pub fn math_pr(w: &mut W, m: &MathProps) { - w.open("m:mathPr", &[]); - let bin = match m.brk_bin { - BrkBin::Before => "before", - BrkBin::After => "after", - BrkBin::Repeat => "repeat", - }; - val(w, "m:brkBin", bin); - let sub = match m.brk_bin_sub { - BrkBinSub::MinusMinus => "--", - BrkBinSub::MinusPlus => "-+", - BrkBinSub::PlusMinus => "+-", - }; - val(w, "m:brkBinSub", sub); - if m.wrap_right { - w.empty("m:wrapRight", &[]); - } else { - let indent = if m.wrap_indent.is_finite() { m.wrap_indent.clamp(0.0, 1584.0) } else { craft_wordmodel::math::DEFAULT_WRAP_INDENT }; - val(w, "m:wrapIndent", &crate::units::twips(indent)); - } - w.close("m:mathPr"); -} - -fn val(w: &mut W, name: &str, v: &str) { - w.empty(name, &[("m:val", v)]); -} - -fn arg(w: &mut W, name: &str, a: &[MNode], depth: usize) { - if a.is_empty() { - w.empty(name, &[]); - return; - } - w.open(name, &[]); - arg_inner(w, a, depth + 1); - w.close(name); -} - -fn arg_inner(w: &mut W, a: &[MNode], depth: usize) { - if depth > MAX_DEPTH { - return; - } - for n in a { - node(w, n, depth); - } -} - -fn ctrl(w: &mut W) { - w.open("m:ctrlPr", &[]); - w.open("w:rPr", &[]); - w.empty("w:rFonts", &[("w:ascii", "Cambria Math"), ("w:hAnsi", "Cambria Math")]); - w.empty("w:i", &[]); - w.close("w:rPr"); - w.close("m:ctrlPr"); -} - -fn char_prop(w: &mut W, name: &str, c: Option) { - let s = c.map(|c| c.to_string()).unwrap_or_default(); - val(w, name, &s); -} - -fn write_matrix(w: &mut W, n: &MNode, depth: usize) { - let MNode::Matrix { rows, col_jc } = n else { return }; - // Every row is padded to the widest one, so a ragged matrix is clamped to the reader's - // caps rather than written as rows × widest cells (#3643). - let rows = rows.get(..rows.len().min(MAX_MATRIX_ROWS)).unwrap_or_default(); - let cols = rows.iter().map(|r| r.len()).max().unwrap_or(0).clamp(1, MAX_MATRIX_COLS); - w.open("m:m", &[]); - w.open("m:mPr", &[]); - w.open("m:mcs", &[]); - for c in 0..cols { - let jc = match col_jc.get(c).copied().unwrap_or_default() { - ColJc::Center => "center", - ColJc::Left => "left", - ColJc::Right => "right", - }; - w.open("m:mc", &[]); - w.open("m:mcPr", &[]); - val(w, "m:count", "1"); - val(w, "m:mcJc", jc); - w.close("m:mcPr"); - w.close("m:mc"); - } - w.close("m:mcs"); - ctrl(w); - w.close("m:mPr"); - for row in rows { - w.open("m:mr", &[]); - for c in 0..cols { - arg(w, "m:e", row.get(c).map(|v| v.as_slice()).unwrap_or(&[]), depth); - } - w.close("m:mr"); - } - w.close("m:m"); -} - -fn write_nary(w: &mut W, n: &MNode, depth: usize) { - let MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub, sup, e } = n else { return }; - w.open("m:nary", &[]); - w.open("m:naryPr", &[]); - if *chr != '∫' { - char_prop(w, "m:chr", Some(*chr)); - } - match lim_loc { - Some(LimLoc::UndOvr) => val(w, "m:limLoc", "undOvr"), - Some(LimLoc::SubSup) => val(w, "m:limLoc", "subSup"), - None => {} - } - if *grow { - val(w, "m:grow", "1"); - } - if *sub_hide { - val(w, "m:subHide", "1"); - } - if *sup_hide { - val(w, "m:supHide", "1"); - } - ctrl(w); - w.close("m:naryPr"); - arg(w, "m:sub", sub, depth); - arg(w, "m:sup", sup, depth); - arg(w, "m:e", e, depth); - w.close("m:nary"); -} - -fn node(w: &mut W, n: &MNode, depth: usize) { - match n { - MNode::Run(r) => run(w, r), - MNode::Frac { kind, num, den } => { - w.open("m:f", &[]); - w.open("m:fPr", &[]); - match kind { - FracKind::Bar => {} - FracKind::Skewed => val(w, "m:type", "skw"), - FracKind::Linear => val(w, "m:type", "lin"), - FracKind::NoBar => val(w, "m:type", "noBar"), - } - ctrl(w); - w.close("m:fPr"); - arg(w, "m:num", num, depth); - arg(w, "m:den", den, depth); - w.close("m:f"); - } - MNode::Script { kind, base, sub, sup } => { - let (el, pr) = match kind { - ScriptKind::Sup => ("m:sSup", "m:sSupPr"), - ScriptKind::Sub => ("m:sSub", "m:sSubPr"), - ScriptKind::SubSup => ("m:sSubSup", "m:sSubSupPr"), - ScriptKind::Pre => ("m:sPre", "m:sPrePr"), - }; - w.open(el, &[]); - w.open(pr, &[]); - ctrl(w); - w.close(pr); - if *kind == ScriptKind::Pre { - arg(w, "m:sub", sub, depth); - arg(w, "m:sup", sup, depth); - arg(w, "m:e", base, depth); - } else { - arg(w, "m:e", base, depth); - if matches!(kind, ScriptKind::Sub | ScriptKind::SubSup) { - arg(w, "m:sub", sub, depth); - } - if matches!(kind, ScriptKind::Sup | ScriptKind::SubSup) { - arg(w, "m:sup", sup, depth); - } - } - w.close(el); - } - MNode::Rad { deg, deg_hide, e } => { - w.open("m:rad", &[]); - w.open("m:radPr", &[]); - if *deg_hide { - val(w, "m:degHide", "1"); - } - ctrl(w); - w.close("m:radPr"); - arg(w, "m:deg", deg, depth); - arg(w, "m:e", e, depth); - w.close("m:rad"); - } - MNode::Nary { .. } => write_nary(w, n, depth), - MNode::Delim { beg, end, sep, grow, shp_match, elems } => { - w.open("m:d", &[]); - w.open("m:dPr", &[]); - if *beg != Some('(') { - char_prop(w, "m:begChr", *beg); - } - if *sep != Some('|') { - char_prop(w, "m:sepChr", *sep); - } - if *end != Some(')') { - char_prop(w, "m:endChr", *end); - } - if !grow { - val(w, "m:grow", "0"); - } - if *shp_match { - val(w, "m:shp", "match"); - } - ctrl(w); - w.close("m:dPr"); - for e in elems { - arg(w, "m:e", e, depth); - } - w.close("m:d"); - } - MNode::Func { name, e } => { - w.open("m:func", &[]); - w.open("m:funcPr", &[]); - ctrl(w); - w.close("m:funcPr"); - arg(w, "m:fName", name, depth); - arg(w, "m:e", e, depth); - w.close("m:func"); - } - MNode::Lim { upper, e, lim } => { - let (el, pr) = if *upper { ("m:limUpp", "m:limUppPr") } else { ("m:limLow", "m:limLowPr") }; - w.open(el, &[]); - w.open(pr, &[]); - ctrl(w); - w.close(pr); - arg(w, "m:e", e, depth); - arg(w, "m:lim", lim, depth); - w.close(el); - } - MNode::Acc { chr, e } => { - w.open("m:acc", &[]); - w.open("m:accPr", &[]); - char_prop(w, "m:chr", Some(*chr)); - ctrl(w); - w.close("m:accPr"); - arg(w, "m:e", e, depth); - w.close("m:acc"); - } - MNode::Bar { top, e } => { - w.open("m:bar", &[]); - w.open("m:barPr", &[]); - val(w, "m:pos", if *top { "top" } else { "bot" }); - ctrl(w); - w.close("m:barPr"); - arg(w, "m:e", e, depth); - w.close("m:bar"); - } - MNode::BorderBox { hide, strike, e } => { - w.open("m:borderBox", &[]); - w.open("m:borderBoxPr", &[]); - for (on, name) in hide.iter().zip(["m:hideTop", "m:hideBot", "m:hideLeft", "m:hideRight"]) { - if *on { - val(w, name, "1"); - } - } - for (on, name) in strike.iter().zip(["m:strikeH", "m:strikeV", "m:strikeBLTR", "m:strikeTLBR"]) { - if *on { - val(w, name, "1"); - } - } - ctrl(w); - w.close("m:borderBoxPr"); - arg(w, "m:e", e, depth); - w.close("m:borderBox"); - } - MNode::Boxed { e } => { - w.open("m:box", &[]); - w.open("m:boxPr", &[]); - ctrl(w); - w.close("m:boxPr"); - arg(w, "m:e", e, depth); - w.close("m:box"); - } - MNode::GroupChr { chr, top, e } => { - w.open("m:groupChr", &[]); - w.open("m:groupChrPr", &[]); - char_prop(w, "m:chr", Some(*chr)); - val(w, "m:pos", if *top { "top" } else { "bot" }); - val(w, "m:vertJc", if *top { "bot" } else { "top" }); - ctrl(w); - w.close("m:groupChrPr"); - arg(w, "m:e", e, depth); - w.close("m:groupChr"); - } - MNode::EqArr { rows } => { - w.open("m:eqArr", &[]); - w.open("m:eqArrPr", &[]); - // Numbered rows (`#`) stretch across the line so the number reaches the margin. - if rows.iter().any(|r| craft_wordmodel::math::has_number_mark(r)) { - val(w, "m:maxDist", "1"); - } - ctrl(w); - w.close("m:eqArrPr"); - for r in rows { - arg(w, "m:e", r, depth); - } - w.close("m:eqArr"); - } - MNode::Matrix { .. } => write_matrix(w, n, depth), - MNode::Phant { show, zero_wid, zero_asc, zero_desc, e } => { - w.open("m:phant", &[]); - w.open("m:phantPr", &[]); - val(w, "m:show", if *show { "1" } else { "0" }); - for (on, name) in [(*zero_wid, "m:zeroWid"), (*zero_asc, "m:zeroAsc"), (*zero_desc, "m:zeroDesc")] { - if on { - val(w, name, "1"); - } - } - ctrl(w); - w.close("m:phantPr"); - arg(w, "m:e", e, depth); - w.close("m:phant"); - } - } -} - -fn run(w: &mut W, r: &MRun) { - w.open("m:r", &[]); - let sty = r.sty.map(|s| match s { - MSty::Plain => "p", - MSty::Bold => "b", - MSty::Italic => "i", - MSty::BoldItalic => "bi", - }); - let scr = match r.scr { - MScr::Roman => None, - MScr::Script => Some("script"), - MScr::Fraktur => Some("fraktur"), - MScr::DoubleStruck => Some("double-struck"), - MScr::SansSerif => Some("sans-serif"), - MScr::Monospace => Some("monospace"), - }; - if sty.is_some() || scr.is_some() || r.nor || r.lit || r.brk.is_some() { - w.open("m:rPr", &[]); - if r.lit { - w.empty("m:lit", &[]); - } - if r.nor { - w.empty("m:nor", &[]); - } - if let Some(s) = scr { - val(w, "m:scr", s); - } - if let Some(s) = sty { - val(w, "m:sty", s); - } - match r.brk { - Some(0) => w.empty("m:brk", &[]), - Some(n) => w.empty("m:brk", &[("m:alnAt", &n.to_string())]), - None => {} - } - w.close("m:rPr"); - } - w.open("w:rPr", &[]); - let font = r.font.as_deref().filter(|_| r.nor).unwrap_or("Cambria Math"); - w.empty("w:rFonts", &[("w:ascii", font), ("w:hAnsi", font)]); - if let Some(c) = r.color { - w.val("w:color", &c.hex()); - } - if let Some(sz) = r.size { - let hp = (sz * 2.0).round().clamp(2.0, 3276.0) as i64; - w.val("w:sz", &hp.to_string()); - w.val("w:szCs", &hp.to_string()); - } - w.close("w:rPr"); - w.leaf("m:t", &[("xml:space", "preserve")], &r.text); - w.close("m:r"); + let mut o = craft_math::omml::W::default(); + craft_math::omml::math_pr(&mut o, m); + w.raw(&o.s); } diff --git a/crates/wordmodel/Cargo.toml b/crates/wordmodel/Cargo.toml index 7426a738d..200cd1653 100644 --- a/crates/wordmodel/Cargo.toml +++ b/crates/wordmodel/Cargo.toml @@ -15,6 +15,9 @@ workspace = true test-support = [] [dependencies] +# The equation tree and its forms (math, math_latex, math_linear, math_symbols, math_tree), with +# symbol_fonts and Rgb, re-exported here under the paths they had. +craft-math = { workspace = true } base64 = "0.23" log = "0.4" serde = { version = "1", features = ["derive", "rc"] } diff --git a/crates/wordmodel/src/lib.rs b/crates/wordmodel/src/lib.rs index 9034bd8af..5fb2d14d8 100644 --- a/crates/wordmodel/src/lib.rs +++ b/crates/wordmodel/src/lib.rs @@ -37,11 +37,7 @@ pub mod legacy_form; pub mod line_dash; pub mod link; pub mod locale_number; -pub mod math; -pub mod math_latex; -pub mod math_linear; -pub mod math_symbols; -pub mod math_tree; +pub use craft_math::{math, math_latex, math_linear, math_symbols, math_tree, symbol_fonts}; pub mod numbering; pub mod para; pub mod picfx; @@ -50,7 +46,6 @@ pub mod resolve; pub mod section; pub mod smart_art; pub mod styles; -pub mod symbol_fonts; pub mod table; pub mod walk; pub mod wordart; diff --git a/crates/wordmodel/src/math.rs b/crates/wordmodel/src/math.rs deleted file mode 100644 index db9875fdb..000000000 --- a/crates/wordmodel/src/math.rs +++ /dev/null @@ -1,1138 +0,0 @@ -//! Equations: the Office Math object model (OMML, ECMA-376 Part 1 §22.1) as a tree, its linear -//! format (UnicodeMath, the plain text Word shows for an equation) and a parser for that format. -//! -//! An equation is a list of [`MNode`]s (an *argument*); structures hold arguments of their own -//! (a fraction's numerator and denominator, a radical's degree and base…). Layout reads the tree; -//! DOCX import builds it from OMML and keeps the source XML so a saved file carries the equation -//! exactly as Word wrote it. - -use serde::{Deserialize, Serialize}; - -use crate::props::Rgb; - -/// A sequence of math nodes (OMML's `m:e`, `m:num`, `m:sub`…). -pub type Arg = Vec; - -/// Deepest structure nesting kept (parsers stop here; layout draws nothing deeper). -pub const MAX_DEPTH: usize = 64; - -/// Most matrix columns kept (writers pad every row to the widest, so this bounds a row's cost). -pub const MAX_MATRIX_COLS: usize = 64; - -/// Most matrix rows kept. -pub const MAX_MATRIX_ROWS: usize = 1000; - -/// An equation's structure, plus what's needed to write it back. -#[derive(Clone, Debug, PartialEq, Default, Serialize, Deserialize)] -pub struct Math { - #[serde(default, skip_serializing_if = "Vec::is_empty")] - pub nodes: Arg, - /// The source `m:oMath` element (DOCX import), written back unchanged on save. Empty for - /// equations made here. - #[serde(default, skip_serializing_if = "String::is_empty")] - pub omml: String, - /// Justification of a display equation. - #[serde(default, skip_serializing_if = "is_default")] - pub jc: MathJc, - /// Shown as its linear-format text (Equation › Convert › Linear) instead of built up. - #[serde(default, skip_serializing_if = "is_default")] - pub linear: bool, -} - -fn is_default(v: &T) -> bool { - *v == T::default() -} - -impl Math { - pub fn is_empty(&self) -> bool { - self.nodes.is_empty() - } - - /// Rough heap bytes a clone copies: every node, its argument lists and its run text. - /// Iterative, so it costs at most the size of the structure. - pub fn heap_bytes(&self) -> usize { - let mut n = self.omml.len(); - let mut todo: Vec<&MNode> = self.nodes.iter().collect(); - while let Some(node) = todo.pop() { - n = n.saturating_add(std::mem::size_of::()); - if let MNode::Run(r) = node { - n = n.saturating_add(r.text.len()).saturating_add(r.font.as_ref().map_or(0, String::len)); - } - each_arg(node, &mut |a| { - n = n.saturating_add(std::mem::size_of::()); - todo.extend(a); - }); - } - n - } -} - -/// Display equation justification (`m:oMathParaPr/m:jc`). -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum MathJc { - #[default] - CenterGroup, - Center, - Left, - Right, -} - -/// Where a display equation too wide for its line breaks at a binary operator (`m:brkBin`). -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum BrkBin { - /// The operator starts the next line. - #[default] - Before, - /// The operator ends the line. - After, - /// The operator ends the line and starts the next one again. - Repeat, -} - -/// How a minus repeats across a break when [`BrkBin::Repeat`] (`m:brkBinSub`): the operator -/// ending the line, then the one starting the next. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum BrkBinSub { - /// `--` - #[default] - MinusMinus, - /// `-+` - MinusPlus, - /// `+-` - PlusMinus, -} - -/// Document-wide equation options (`m:mathPr` in the settings part) that layout uses: how long -/// display equations wrap. -#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -#[serde(default, rename_all = "camelCase")] -pub struct MathProps { - pub brk_bin: BrkBin, - pub brk_bin_sub: BrkBinSub, - /// Indent of a wrapped equation's continuation lines, points (`m:wrapIndent`). - pub wrap_indent: f32, - /// Continuation lines align right instead (`m:wrapRight`). - pub wrap_right: bool, -} - -/// `m:wrapIndent`'s default: 1440 twips, one inch. -pub const DEFAULT_WRAP_INDENT: f32 = 72.0; - -impl Default for MathProps { - fn default() -> Self { - MathProps { brk_bin: BrkBin::Before, brk_bin_sub: BrkBinSub::MinusMinus, wrap_indent: DEFAULT_WRAP_INDENT, wrap_right: false } - } -} - -/// Math run style (`m:sty`): plain, bold, italic, bold italic. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum MSty { - Plain, - Bold, - Italic, - BoldItalic, -} - -/// Math alphabet (`m:scr`). -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum MScr { - #[default] - Roman, - Script, - Fraktur, - DoubleStruck, - SansSerif, - Monospace, -} - -/// A run of math text (`m:r`). -#[derive(Clone, Debug, PartialEq, Default, Serialize, Deserialize)] -pub struct MRun { - pub text: String, - /// `None`: math default (letters italic, the rest upright). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub sty: Option, - #[serde(default, skip_serializing_if = "is_default")] - pub scr: MScr, - /// Normal (non-math) text. - #[serde(default, skip_serializing_if = "is_default")] - pub nor: bool, - /// Literal: no automatic operator handling. - #[serde(default, skip_serializing_if = "is_default")] - pub lit: bool, - /// Font size, points. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub size: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] - pub color: Option, - /// Font for normal text. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub font: Option, - /// A manual line break before this run in a display equation (`m:brk`): `Some(0)` wraps as - /// usual, `Some(n)` aligns the new line at the `n`th operator of the first line (`m:alnAt`). - #[serde(default, skip_serializing_if = "Option::is_none")] - pub brk: Option, -} - -impl MRun { - pub fn new(text: impl Into) -> MRun { - MRun { text: text.into(), ..Default::default() } - } - pub fn plain(text: impl Into) -> MRun { - MRun { text: text.into(), sty: Some(MSty::Plain), ..Default::default() } - } -} - -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum FracKind { - /// Stacked with a rule. - #[default] - Bar, - /// Skewed (num ⁄ den). - Skewed, - /// Linear (num/den). - Linear, - /// Stacked without a rule. - NoBar, -} - -#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum ScriptKind { - Sup, - Sub, - SubSup, - /// Pre-scripts (`m:sPre`). - Pre, -} - -/// Where an n-ary operator's limits go. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum LimLoc { - /// Under and over the operator. - UndOvr, - /// As sub- and superscripts. - SubSup, -} - -/// Column justification in a matrix. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default, Serialize, Deserialize)] -#[serde(rename_all = "camelCase")] -pub enum ColJc { - #[default] - Center, - Left, - Right, -} - -/// A math node. -#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -#[serde(tag = "t", rename_all = "camelCase")] -pub enum MNode { - Run(MRun), - /// `m:f`. - Frac { - #[serde(default)] - kind: FracKind, - num: Arg, - den: Arg, - }, - /// `m:sSup`, `m:sSub`, `m:sSubSup`, `m:sPre`. - Script { - kind: ScriptKind, - base: Arg, - #[serde(default)] - sub: Arg, - #[serde(default)] - sup: Arg, - }, - /// `m:rad`. - Rad { - #[serde(default)] - deg: Arg, - #[serde(default)] - deg_hide: bool, - e: Arg, - }, - /// `m:nary`: ∑ ∫ ∏ … with limits and an operand. - Nary { - chr: char, - /// `None`: Word's default (integrals take sub/sup limits, the rest under/over). - #[serde(default)] - lim_loc: Option, - #[serde(default)] - grow: bool, - #[serde(default)] - sub_hide: bool, - #[serde(default)] - sup_hide: bool, - #[serde(default)] - sub: Arg, - #[serde(default)] - sup: Arg, - e: Arg, - }, - /// `m:d`: delimiters around elements separated by `sep`. - Delim { - /// `None` = no delimiter. - beg: Option, - end: Option, - #[serde(default)] - sep: Option, - /// Delimiters grow to the content (Word's default). - #[serde(default = "yes")] - grow: bool, - /// Grow to match the content exactly rather than centred on the math axis. - #[serde(default)] - shp_match: bool, - elems: Vec, - }, - /// `m:func`: function name applied to an argument (sin x). - Func { - name: Arg, - e: Arg, - }, - /// `m:limLow` / `m:limUpp`. - Lim { - upper: bool, - e: Arg, - lim: Arg, - }, - /// `m:acc`: accent over the base. - Acc { - chr: char, - e: Arg, - }, - /// `m:bar`: overbar or underbar. - Bar { - top: bool, - e: Arg, - }, - /// `m:borderBox`. `hide`: top, bottom, left, right. `strike`: horizontal, vertical, - /// bottom-left→top-right, top-left→bottom-right. - BorderBox { - #[serde(default)] - hide: [bool; 4], - #[serde(default)] - strike: [bool; 4], - e: Arg, - }, - /// `m:box`. - Boxed { - e: Arg, - }, - /// `m:groupChr`: a stretched character (brace, arrow) under or over the base. - GroupChr { - chr: char, - top: bool, - e: Arg, - }, - /// `m:eqArr`: rows aligned at `&`. - EqArr { - rows: Vec, - }, - /// `m:m`. - Matrix { - rows: Vec>, - #[serde(default)] - col_jc: Vec, - }, - /// `m:phant`. - Phant { - #[serde(default = "yes")] - show: bool, - #[serde(default)] - zero_wid: bool, - #[serde(default)] - zero_asc: bool, - #[serde(default)] - zero_desc: bool, - e: Arg, - }, -} - -fn yes() -> bool { - true -} - -/// A row holds an equation-number mark (`#`) in a top-level run. -pub fn has_number_mark(row: &[MNode]) -> bool { - row.iter().any(|n| matches!(n, MNode::Run(r) if !r.lit && !r.nor && r.text.contains('#'))) -} - -/// Integral signs (their limits default to sub/sup positions). -pub fn is_integral(c: char) -> bool { - matches!(c, '∫' | '∬' | '∭' | '∮' | '∯' | '∰' | '∱' | '∲' | '∳' | '⨌') -} - -/// Characters that take n-ary operator form in the linear format. -pub fn is_nary_char(c: char) -> bool { - is_integral(c) || matches!(c, '∑' | '∏' | '∐' | '⋃' | '⋂' | '⋁' | '⋀' | '⨁' | '⨂' | '⨀' | '⨄' | '⨆') -} - -/// Mathematical alphanumeric for `c` in a style and alphabet (Unicode block U+1D400). -pub fn math_alnum(c: char, bold: bool, italic: bool, scr: MScr) -> char { - let off = |base: u32, k: u32| char::from_u32(base + k).unwrap_or(c); - let upper = c.is_ascii_uppercase(); - let lower = c.is_ascii_lowercase(); - if upper || lower { - let k = if upper { c as u32 - 'A' as u32 } else { c as u32 - 'a' as u32 }; - // Letters that live in the Letterlike Symbols block. - let hole = |table: &[(char, char)]| table.iter().find(|(f, _)| *f == c).map(|(_, t)| *t); - let (base_u, base_l) = match (scr, bold, italic) { - (MScr::Roman, false, false) => return c, - (MScr::Roman, true, false) => (0x1D400, 0x1D41A), - (MScr::Roman, false, true) => { - if c == 'h' { - return '\u{210E}'; - } - (0x1D434, 0x1D44E) - } - (MScr::Roman, true, true) => (0x1D468, 0x1D482), - (MScr::Script, false, _) => { - if let Some(t) = hole(&[ - ('B', 'ℬ'), - ('E', 'ℰ'), - ('F', 'ℱ'), - ('H', 'ℋ'), - ('I', 'ℐ'), - ('L', 'ℒ'), - ('M', 'ℳ'), - ('R', 'ℛ'), - ('e', 'ℯ'), - ('g', 'ℊ'), - ('o', 'ℴ'), - ]) { - return t; - } - (0x1D49C, 0x1D4B6) - } - (MScr::Script, true, _) => (0x1D4D0, 0x1D4EA), - (MScr::Fraktur, false, _) => { - if let Some(t) = hole(&[('C', 'ℭ'), ('H', 'ℌ'), ('I', 'ℑ'), ('R', 'ℜ'), ('Z', 'ℨ')]) { - return t; - } - (0x1D504, 0x1D51E) - } - (MScr::Fraktur, true, _) => (0x1D56C, 0x1D586), - (MScr::DoubleStruck, _, _) => { - if let Some(t) = hole(&[('C', 'ℂ'), ('H', 'ℍ'), ('N', 'ℕ'), ('P', 'ℙ'), ('Q', 'ℚ'), ('R', 'ℝ'), ('Z', 'ℤ')]) { - return t; - } - (0x1D538, 0x1D552) - } - (MScr::SansSerif, false, false) => (0x1D5A0, 0x1D5BA), - (MScr::SansSerif, true, false) => (0x1D5D4, 0x1D5EE), - (MScr::SansSerif, false, true) => (0x1D608, 0x1D622), - (MScr::SansSerif, true, true) => (0x1D63C, 0x1D656), - (MScr::Monospace, _, _) => (0x1D670, 0x1D68A), - }; - return off(if upper { base_u } else { base_l }, k); - } - if c.is_ascii_digit() { - let k = c as u32 - '0' as u32; - return match (scr, bold) { - (MScr::Roman, true) => off(0x1D7CE, k), - (MScr::DoubleStruck, _) => off(0x1D7D8, k), - (MScr::SansSerif, false) => off(0x1D7E2, k), - (MScr::SansSerif, true) => off(0x1D7EC, k), - (MScr::Monospace, _) => off(0x1D7F6, k), - _ => c, - }; - } - // Greek (roman alphabet only). - if scr == MScr::Roman && (bold || italic) { - let (cap, small) = match (bold, italic) { - (true, false) => (0x1D6A8, 0x1D6C2), - (false, true) => (0x1D6E2, 0x1D6FC), - _ => (0x1D71C, 0x1D736), - }; - let u = c as u32; - if (0x391..=0x3A9).contains(&u) && u != 0x3A2 { - return off(cap, u - 0x391); - } - if (0x3B1..=0x3C9).contains(&u) { - return off(small, u - 0x3B1); - } - let extra = match c { - '∂' => Some(25), - 'ϵ' => Some(26), - 'ϑ' => Some(27), - 'ϰ' => Some(28), - 'ϕ' => Some(29), - 'ϱ' => Some(30), - 'ϖ' => Some(31), - _ => None, - }; - if let Some(e) = extra - && italic - { - return off(small, e); - } - } - c -} - -// --------------------------------------------------------------------------------------------- -// Linear format out. - -/// The equation as linear-format text (`x=(-b±√(b^2-4ac))/2a`). -pub fn to_linear(nodes: &[MNode]) -> String { - let mut s = String::new(); - lin_arg(nodes, &mut s, 0); - s -} - -fn lin_arg(a: &[MNode], s: &mut String, depth: usize) { - if depth > MAX_DEPTH { - return; - } - for (i, n) in a.iter().enumerate() { - lin_node(n, s, depth + 1); - // A bare script or limit ends at a space: a letter, digit or period after it is its own item. - if matches!(n, MNode::Script { kind, .. } if *kind != ScriptKind::Pre) || matches!(n, MNode::Lim { .. }) { - let ends_bare = s.chars().next_back().is_some_and(|c| c.is_alphanumeric() || c == '.'); - if ends_bare && a.get(i + 1).and_then(first_lin_char).is_some_and(|c| c.is_alphanumeric() || c == '.') { - s.push(' '); - } - } - } -} - -/// The first character `n` is written with, when that is cheap to tell (letters and digits). -fn first_lin_char(n: &MNode) -> Option { - match n { - MNode::Run(r) => r.text.chars().next(), - MNode::Func { name, .. } => name.first().and_then(first_lin_char), - MNode::Script { kind, base, .. } if *kind != ScriptKind::Pre => base.first().and_then(first_lin_char), - MNode::Lim { e, .. } => e.first().and_then(first_lin_char), - MNode::Frac { num, .. } => num.first().and_then(first_lin_char), - _ => None, - } -} - -/// An argument as an operand: bare when it's a single simple atom, parenthesised otherwise. -fn lin_operand(a: &[MNode], s: &mut String, depth: usize) { - let mut inner = String::new(); - lin_arg(a, &mut inner, depth); - let simple = match a { - [MNode::Run(r)] => { - let mut cs = r.text.chars(); - match (cs.next(), cs.next()) { - (Some(c), None) => !"+-−=/^_ ".contains(c), - _ => r.text.chars().all(|c| c.is_alphanumeric() || c == '.'), - } - } - [MNode::Delim { .. }] => true, - _ => false, - }; - if simple && !inner.is_empty() { - s.push_str(&inner); - } else { - s.push('('); - s.push_str(&inner); - s.push(')'); - } -} - -/// A script or limit operand. The parser reads a run of letters and digits up to the next space or -/// operator, so `V_dd` is written bare as Word's Linear format shows it (`V_(dd)` reads back the same). -fn lin_script(a: &[MNode], s: &mut String, depth: usize) { - // Bare only when the parser reads the whole run back, in the same style. - if let [MNode::Run(r)] = a - && r.text.chars().nth(1).is_some() - { - let upright_name = r.sty == Some(MSty::Plain) && crate::math_linear::FUNCTION_NAMES.contains(&r.text.as_str()); - let number = r.text.starts_with(|c: char| c.is_ascii_digit()) && r.text.chars().all(|c| c.is_alphanumeric() || c == '.'); - let word = r.text.starts_with(char::is_alphabetic) && r.text.chars().all(char::is_alphanumeric); - let upright_other = r.sty == Some(MSty::Plain) && !upright_name; - if upright_other || (!number && !word) { - s.push('('); - s.push_str(&r.text); - s.push(')'); - return; - } - } - lin_operand(a, s, depth); -} - -fn lin_node(n: &MNode, s: &mut String, depth: usize) { - match n { - MNode::Run(r) => s.push_str(&r.text), - MNode::Frac { kind, num, den } => { - lin_operand(num, s, depth); - s.push(match kind { - FracKind::Skewed => '⁄', - FracKind::Linear => '∕', - FracKind::NoBar => '¦', - FracKind::Bar => '/', - }); - lin_operand(den, s, depth); - } - MNode::Script { kind, base, sub, sup } => { - if *kind == ScriptKind::Pre { - s.push('_'); - lin_script(sub, s, depth); - s.push('^'); - lin_script(sup, s, depth); - s.push('▒'); - lin_operand(base, s, depth); - return; - } - lin_operand(base, s, depth); - if matches!(kind, ScriptKind::Sub | ScriptKind::SubSup) { - s.push('_'); - lin_script(sub, s, depth); - } - if matches!(kind, ScriptKind::Sup | ScriptKind::SubSup) { - s.push('^'); - lin_script(sup, s, depth); - } - } - MNode::Rad { deg, deg_hide, e } => { - if *deg_hide || deg.is_empty() { - s.push('√'); - lin_operand(e, s, depth); - } else { - s.push_str("√("); - lin_arg(deg, s, depth); - s.push('&'); - lin_arg(e, s, depth); - s.push(')'); - } - } - MNode::Nary { chr, sub_hide, sup_hide, sub, sup, e, .. } => { - s.push(*chr); - if !sub_hide && !sub.is_empty() { - s.push('_'); - lin_script(sub, s, depth); - } - if !sup_hide && !sup.is_empty() { - s.push('^'); - lin_script(sup, s, depth); - } - s.push('▒'); - lin_operand(e, s, depth); - } - MNode::Delim { beg, end, elems, .. } => { - // A standard pair writes as itself; anything else uses explicit `├X … Y┤`. - let pair = matches!((beg, end), (Some(b), Some(e)) if crate::math_linear::closer(*b) == Some(*e)); - let invisible = beg.is_none() && end.is_none(); - if pair { - s.extend(*beg); - } else if invisible { - s.push('〖'); - } else { - s.push('├'); - s.extend(*beg); - } - for (i, e) in elems.iter().enumerate() { - if i > 0 { - s.push('│'); - } - lin_arg(e, s, depth); - } - if pair { - s.extend(*end); - } else if invisible { - s.push('〗'); - } else { - s.extend(*end); - s.push('┤'); - } - } - MNode::Func { name, e } => { - lin_arg(name, s, depth); - s.push('\u{2061}'); - lin_operand(e, s, depth); - } - MNode::Lim { upper, e, lim } => { - lin_operand(e, s, depth); - s.push(if *upper { '┴' } else { '┬' }); - lin_script(lim, s, depth); - } - MNode::Acc { chr, e } => { - lin_operand(e, s, depth); - s.push(*chr); - } - MNode::Bar { top, e } => { - s.push(if *top { '¯' } else { '▁' }); - lin_operand(e, s, depth); - } - MNode::BorderBox { e, .. } => { - s.push('▭'); - lin_operand(e, s, depth); - } - MNode::Boxed { e } => { - s.push('□'); - lin_operand(e, s, depth); - } - MNode::GroupChr { chr, e, .. } => { - s.push(*chr); - lin_operand(e, s, depth); - } - // A numbered equation (`E=mc^2#(1)`) is written as its one row. - MNode::EqArr { rows } if rows.len() == 1 && rows.first().is_some_and(|r| has_number_mark(r)) => { - if let Some(r) = rows.first() { - lin_arg(r, s, depth); - } - } - MNode::EqArr { rows } => { - s.push_str("█("); - for (i, r) in rows.iter().enumerate() { - if i > 0 { - s.push('@'); - } - lin_arg(r, s, depth); - } - s.push(')'); - } - MNode::Matrix { rows, .. } => { - s.push_str("■("); - for (i, r) in rows.iter().enumerate() { - if i > 0 { - s.push('@'); - } - for (j, c) in r.iter().enumerate() { - if j > 0 { - s.push('&'); - } - lin_arg(c, s, depth); - } - } - s.push(')'); - } - MNode::Phant { e, .. } => { - s.push('⟡'); - lin_operand(e, s, depth); - } - } -} - -// --------------------------------------------------------------------------------------------- -// Linear format in. - -/// Parse linear-format text into nodes ([`crate::math_linear`]). Never fails: what isn't -/// understood stays as text. -pub fn parse_linear(s: &str) -> Arg { - crate::math_linear::parse(s) -} - -pub use crate::math_linear::FUNCTION_NAMES; - -/// Spacing class of a math character (TeX's atom classes). -#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)] -pub enum MClass { - #[default] - Ord, - Op, - Bin, - Rel, - Open, - Close, - Punct, - Inner, -} - -/// The spacing class of a character in an equation. -pub fn math_class(c: char) -> MClass { - match c { - '+' | '−' | '-' | '±' | '∓' | '×' | '÷' | '·' | '∙' | '∗' | '*' | '∘' | '⊕' | '⊖' | '⊗' | '⊘' | '⊙' | '⊚' | '⊛' | '∪' | '∩' | '∧' | '∨' - | '∖' | '⋅' | '⋆' | '⊎' | '⊓' | '⊔' | '⋄' | '⊞' | '⊟' | '⊠' | '⊡' | '⋉' | '⋊' | '⋋' | '⋌' | '⋎' | '⋏' | '⋒' | '⋓' | '∔' | '∸' | '≀' | '⨿' - | '⊼' | '⊻' | '⊽' => MClass::Bin, - '=' | '<' | '>' | '≤' | '≥' | '≠' | '≈' | '≡' | '≢' | '∼' | '≃' | '≅' | '≐' | '∝' | '≪' | '≫' | '≺' | '≻' | '≼' | '≽' | '⊂' | '⊃' | '⊆' - | '⊇' | '⊄' | '⊅' | '⊈' | '⊉' | '⊊' | '⊋' | '∈' | '∉' | '∋' | '∌' | '⊥' | '∥' | '∦' | '∣' | '∤' | '≔' | '≕' | '≝' | '≜' | '≞' | '≟' | '→' - | '←' | '↔' | '⇒' | '⇐' | '⇔' | '↦' | '⟼' | '⟶' | '⟵' | '⟷' | '⟹' | '⟸' | '⟺' | '↑' | '↓' | '↕' | '⇑' | '⇓' | '⇕' | '↗' | '↘' | '↙' | '↖' - | '⊢' | '⊣' | '⊨' | '⊩' | '⊪' | '⊬' | '⊭' | '⊮' | '⊯' | '≲' | '≳' | '≍' | '⊏' | '⊐' | '⊑' | '⊒' | '⋢' | '⋣' | '≶' | '≷' | '≮' | '≯' | '≰' - | '≱' | '≦' | '≧' | '⩽' | '⩾' | ':' | '∶' | '∷' | '∺' | '∻' | '≗' | '≙' | '≚' | '≑' | '≒' | '≓' | '≖' | '⇌' | '⇋' | '⇄' | '⇆' | '⇇' | '⇉' - | '⇈' | '⇊' | '↪' | '↩' | '↼' | '↽' | '⇀' | '⇁' | '↿' | '↾' | '⇃' | '⇂' | '↶' | '↷' | '↺' | '↻' | '⊸' | '↜' | '↝' | '↞' | '↠' | '↢' | '↣' - | '↫' | '↬' | '↭' | '⇚' | '⇛' | '↰' | '↱' | '⇝' | '≁' | '≄' | '≉' | '≇' | '≭' | '≨' | '≩' | '⊀' | '⊁' | '⋠' | '⋡' | '⋦' | '⋧' | '⋨' | '⋩' - | '⋪' | '⋫' | '⋬' | '⋭' | '⋈' | '∴' | '∵' | '≏' | '≎' | '⋘' | '⋙' | '⊲' | '⊳' | '⊴' | '⊵' | '⋐' | '⋑' => { - MClass::Rel - } - '(' | '[' | '{' | '⟨' | '⌈' | '⌊' | '⟦' | '〈' => MClass::Open, - ')' | ']' | '}' | '⟩' | '⌉' | '⌋' | '⟧' | '〉' | '!' => MClass::Close, - ',' | ';' => MClass::Punct, - c if is_nary_char(c) => MClass::Op, - _ => MClass::Ord, - } -} - -/// Merge adjacent runs with the same formatting; recurse into every structure. -pub fn merge_runs(a: Arg) -> Arg { - let mut out: Arg = Vec::with_capacity(a.len()); - for n in a { - let n = match n { - MNode::Run(r) => { - if let Some(MNode::Run(prev)) = out.last_mut() - && prev.sty == r.sty - && prev.scr == r.scr - && prev.nor == r.nor - && prev.lit == r.lit - && prev.size == r.size - && prev.color == r.color - && prev.font == r.font - && r.brk.is_none() - { - prev.text.push_str(&r.text); - continue; - } - MNode::Run(r) - } - other => crate::math_linear::map_args(other, &merge_runs), - }; - out.push(n); - } - out -} - -/// Deepest node (a leaf counts 1) the parsers wrap further. Half of [`MAX_DEPTH`]: serde_json -/// stops reading at 128 nested containers and a node costs two (object and array). -pub const WRAP_DEPTH: usize = MAX_DEPTH / 2; - -/// Calls `f` with every argument of `n`. -fn each_arg<'a>(n: &'a MNode, f: &mut impl FnMut(&'a [MNode])) { - match n { - MNode::Run(_) => {} - MNode::Frac { num, den, .. } => { - f(num); - f(den); - } - MNode::Script { base, sub, sup, .. } => { - f(base); - f(sub); - f(sup); - } - MNode::Rad { deg, e, .. } => { - f(deg); - f(e); - } - MNode::Nary { sub, sup, e, .. } => { - f(sub); - f(sup); - f(e); - } - MNode::Delim { elems, .. } => elems.iter().for_each(|a| f(a)), - MNode::Func { name, e } => { - f(name); - f(e); - } - MNode::Lim { e, lim, .. } => { - f(e); - f(lim); - } - MNode::Acc { e, .. } - | MNode::Bar { e, .. } - | MNode::BorderBox { e, .. } - | MNode::Boxed { e } - | MNode::GroupChr { e, .. } - | MNode::Phant { e, .. } => f(e), - MNode::EqArr { rows } => rows.iter().for_each(|a| f(a)), - MNode::Matrix { rows, .. } => rows.iter().flatten().for_each(|a| f(a)), - } -} - -/// `n` is already nested too deeply to be wrapped in another structure. The parsers use this so -/// `x^1^1^1…` stays shallow (a left-nested tree 10 000 deep overflows the stack in `Clone`, -/// `Drop`, `PartialEq` and serde). Iterative, stops at the first too-deep path; costs at most the -/// size of `n`, which the parsers' input caps bound. -pub fn too_deep(n: &MNode) -> bool { - let mut stack: Vec<(&MNode, usize)> = vec![(n, 1)]; - while let Some((n, d)) = stack.pop() { - if d >= WRAP_DEPTH { - return true; - } - each_arg(n, &mut |a| stack.extend(a.iter().map(|c| (c, d + 1)))); - } - false -} - -/// Any of `a` is too deep to be wrapped ([`too_deep`]). -pub fn any_too_deep(a: &[MNode]) -> bool { - a.iter().any(too_deep) -} - -#[cfg(test)] -mod tests { - use super::*; - - fn run(t: &str) -> MNode { - MNode::Run(MRun::new(t)) - } - - #[test] - fn quadratic_formula_parses_into_structures() { - let n = parse_linear("x=(-b±√(b^2-4ac))/2a"); - assert_eq!(n.len(), 2, "{n:#?}"); - assert_eq!(n[0], run("x=")); - let MNode::Frac { num, den, .. } = &n[1] else { panic!("{n:#?}") }; - assert_eq!(den, &vec![run("2a")]); - assert!(num.iter().any(|m| matches!(m, MNode::Rad { .. })), "{num:#?}"); - let MNode::Rad { e, .. } = num.iter().find(|m| matches!(m, MNode::Rad { .. })).unwrap() else { panic!() }; - assert!(matches!(e.first(), Some(MNode::Script { kind: ScriptKind::Sup, .. })), "{e:#?}"); - } - - #[test] - fn scripts_and_nary() { - let n = parse_linear("∑_(i=1)^n▒i^2"); - let MNode::Nary { chr, sub, sup, e, .. } = &n[0] else { panic!("{n:#?}") }; - assert_eq!(*chr, '∑'); - assert_eq!(sub, &vec![run("i=1")]); - assert_eq!(sup, &vec![run("n")]); - assert!(matches!(e.first(), Some(MNode::Script { .. }))); - let n = parse_linear("x_i^2"); - assert!(matches!(&n[0], MNode::Script { kind: ScriptKind::SubSup, .. }), "{n:#?}"); - } - - #[test] - fn functions_and_matrices() { - let n = parse_linear("sin x"); - assert!(matches!(&n[0], MNode::Func { .. }), "{n:#?}"); - let n = parse_linear("■(a&b@c&d)"); - let MNode::Matrix { rows, .. } = &n[0] else { panic!("{n:#?}") }; - assert_eq!(rows.len(), 2); - assert_eq!(rows[0].len(), 2); - } - - #[test] - fn linear_round_trip_keeps_structure() { - for s in [ - "x=(-b±√(b^2-4ac))/2a", - "∑_(i=1)^n▒i", - "a/b+c", - "E=mc^2", - "√(3&x)", - "├]a[┤", - "{█(x@y)┤", - "|x|+‖v‖", - "⏞(a+b)┴k", - "_1^n▒Y", - "sin^(−1)⁡x", - "lim┬(n→∞)⁡a_n", - "A⃗+x̂", - "▭(a)+¯(b)+▁(c)", - "(a│b)", - "E=mc^2#(1)", - ] { - let n = parse_linear(s); - let back = parse_linear(&to_linear(&n)); - assert_eq!(n, back, "{s} → {}", to_linear(&n)); - } - } - - //T1 - /// Deepest nesting of `a` (a leaf counts 1), without recursion. - fn depth_of(a: &[MNode]) -> usize { - let mut best = 0; - let mut stack: Vec<(&MNode, usize)> = a.iter().map(|n| (n, 1)).collect(); - while let Some((n, d)) = stack.pop() { - best = best.max(d); - each_arg(n, &mut |a| stack.extend(a.iter().map(|c| (c, d + 1)))); - } - best - } - - /// Run `f` on a thread with a stack far below the 8 MB of a main thread (#1456). - fn on_small_stack(f: impl FnOnce() + Send + 'static) { - std::thread::Builder::new().stack_size(512 * 1024).spawn(f).unwrap().join().unwrap(); - } - - #[test] - fn deeply_chained_input_stays_shallow() { - on_small_stack(|| { - let cases: Vec<(&str, Arg)> = vec![ - ("x^1^1…", parse_linear(&format!("x{}", "^1".repeat(9999)))), - ("a/a/a…", parse_linear(&format!("a{}", "/a".repeat(9999)))), - ("accents", parse_linear(&format!("x{}", "\u{302}".repeat(9999)))), - ("limits", parse_linear(&format!("x{}", "┬a".repeat(9000)))), - ("(a b)^1^1…", parse_linear(&format!("(a b){}", "^1".repeat(9000)))), - ("latex x^1^1…", crate::math_latex::parse_latex(&format!("x{}", "^1".repeat(9999)))), - ("latex \\sum…", crate::math_latex::parse_latex(&"\\sum".repeat(5000))), - ("latex \\sin x^1…", crate::math_latex::parse_latex(&format!("\\sin x{}", "^1".repeat(9000)))), - ]; - for (name, n) in cases { - assert!(!n.is_empty(), "{name}"); - assert!(depth_of(&n) <= 70, "{name}: depth {}", depth_of(&n)); - let copy = n.clone(); - assert_eq!(copy, n, "{name}"); - let _ = to_linear(&n); - let _ = crate::math_latex::to_latex(&n); - let json = serde_json::to_string(&n).unwrap(); - let back: Arg = serde_json::from_str(&json).unwrap(); - assert_eq!(back, n, "{name}"); - } - }); - } - - #[test] - fn ordinary_nesting_is_still_built() { - // Well below the cap: scripts, fractions and operators still wrap. - let n = parse_linear(&format!("x{}", "^1".repeat(10))); - assert_eq!(depth_of(&n), 11, "{n:#?}"); - let n = crate::math_latex::parse_latex(r"\sum_{i}\sum_{j}a_{ij}"); - assert!(matches!(&n[0], MNode::Nary { e, .. } if matches!(e.first(), Some(MNode::Nary { .. }))), "{n:#?}"); - } - - //T2 - #[test] - fn latex_text_escapes_what_latex_reads_as_markup() { - for text in ["a}b\\c", "{x}", "50% of #1", "a_b&c$d~e^f", "\\", "\\alpha", "\\{"] { - let n = vec![MNode::Run(MRun { text: text.into(), nor: true, sty: Some(MSty::Plain), ..Default::default() })]; - let tex = crate::math_latex::to_latex(&n); - assert_eq!(crate::math_latex::parse_latex(&tex), n, "{text:?} → {tex}"); - } - let n = vec![MNode::Run(MRun::new("a\\b"))]; - assert_eq!(crate::math_latex::parse_latex(&crate::math_latex::to_latex(&n)), n); - } - - //T3 - #[test] - fn latex_runs_merge_inside_every_structure() { - use crate::math_latex::parse_latex; - for (tex, lin) in [(r"\overline{ab}", "¯(ab)"), (r"\underline{ab}", "▁(ab)"), (r"\boxed{ab}", "▭(ab)"), (r"\phantom{ab}", "⟡(ab)")] { - assert_eq!(parse_latex(tex), parse_linear(lin), "{tex}"); - } - let n = parse_latex(r"\begin{matrix}a b&c d\\e f&g\end{matrix}"); - let MNode::Matrix { rows, .. } = &n[0] else { panic!("{n:#?}") }; - assert!(rows.iter().flatten().all(|c| c.len() == 1), "{rows:#?}"); - let n = parse_latex(r"\hat{ab}\underbrace{ab}\underset{ab}{cd}"); - let runs = |a: &Arg| a.iter().filter(|n| matches!(n, MNode::Run(_))).count(); - assert!(matches!(&n[0], MNode::Acc { e, .. } if runs(e) == 1), "{n:#?}"); - assert!(matches!(&n[1], MNode::GroupChr { e, .. } if runs(e) == 1), "{n:#?}"); - assert!(matches!(&n[2], MNode::Lim { e, lim, .. } if runs(e) == 1 && runs(lim) == 1), "{n:#?}"); - } - - #[test] - fn multi_letter_scripts_round_trip() { - // Word for Mac 16.113.4 (#1609): a script is the whole run up to a space or operator, and Word's - // Linear format shows it bare (`V_dd`); the parenthesised form reads back identically. - let script = |kind, sub: &str, sup: &str| MNode::Script { kind, base: vec![run("V")], sub: parse_linear(sub), sup: parse_linear(sup) }; - for n in [ - vec![script(ScriptKind::Sub, "dd", "")], - vec![script(ScriptKind::Sup, "", "ab")], - vec![script(ScriptKind::SubSup, "in", "2")], - vec![MNode::Nary { - chr: '∑', - lim_loc: None, - grow: false, - sub_hide: false, - sup_hide: false, - sub: vec![run("ij")], - sup: vec![run("n")], - e: vec![run("x")], - }], - vec![MNode::Lim { upper: false, e: vec![run("⟶")], lim: vec![run("yields")] }], - ] { - assert_eq!(parse_linear(&to_linear(&n)), n, "{}", to_linear(&n)); - } - // Numbers and single letters stay bare. - assert_eq!(to_linear(&[script(ScriptKind::SubSup, "10", "2.5")]), "V_10^2.5"); - assert_eq!(to_linear(&[script(ScriptKind::Sub, "d", "")]), "V_d"); - assert_eq!(to_linear(&[script(ScriptKind::Sub, "dd", "")]), "V_dd"); - assert_eq!(to_linear(&[script(ScriptKind::Sup, "", "ab")]), "V^ab"); - } - - #[test] - fn multi_letter_script_operands_follow_word() { - // #1609, Word for Mac 16.113.4: `V_dd` and `V_(dd)` build up the same; `dd` is all subscript. - let sub_dd = MNode::Script { kind: ScriptKind::Sub, base: vec![run("V")], sub: vec![run("dd")], sup: vec![] }; - assert_eq!(parse_linear("V_dd"), vec![sub_dd.clone()]); - assert_eq!(parse_linear("V_(dd)"), vec![sub_dd]); - let sup_ab = MNode::Script { kind: ScriptKind::Sup, base: vec![run("x")], sub: vec![], sup: vec![run("ab")] }; - assert_eq!(parse_linear("x^ab"), vec![sup_ab.clone()]); - assert_eq!(parse_linear("x^(ab)"), vec![sup_ab]); - // Delimiters still end the operand. - let both = MNode::Script { kind: ScriptKind::SubSup, base: vec![run("x")], sub: vec![run("i")], sup: vec![run("2")] }; - assert_eq!(parse_linear("x_i^2"), vec![both]); - let a_i = MNode::Script { kind: ScriptKind::Sub, base: vec![run("a")], sub: vec![run("i")], sup: vec![] }; - assert_eq!(parse_linear("a_i b"), vec![a_i, run("b")]); - let both2 = MNode::Script { kind: ScriptKind::SubSup, base: vec![run("x")], sub: vec![run("ij")], sup: vec![run("kl")] }; - assert_eq!(parse_linear("x_ij^kl"), vec![both2]); - // Round trip is Word's bare form. - assert_eq!(to_linear(&parse_linear("V_(dd)")), "V_dd"); - } - - #[test] - fn a_bare_script_does_not_absorb_what_follows() { - // #1609 review: `x_i` then `y` must not read back as x with subscript `iy`. - let sc = - |kind, base: &str, sub: &str, sup: &str| MNode::Script { kind, base: vec![run(base)], sub: parse_linear(sub), sup: parse_linear(sup) }; - let sin_x = MNode::Func { name: vec![MNode::Run(MRun::plain("sin"))], e: vec![run("x")] }; - let lim = MNode::Lim { upper: false, e: vec![run("a")], lim: vec![run("n")] }; - for n in [ - vec![sc(ScriptKind::Sub, "x", "i", ""), run("y")], - vec![sc(ScriptKind::Sub, "a", "i", ""), sc(ScriptKind::Sub, "b", "j", "")], - vec![sc(ScriptKind::Sup, "x", "", "2"), run("y")], - vec![sc(ScriptKind::Sup, "x", "", "2"), run("3")], - vec![sc(ScriptKind::Sub, "x", "i", ""), sin_x], - vec![lim.clone(), run("y")], - vec![sc(ScriptKind::Sub, "x", "ab", ""), run("y")], - vec![sc(ScriptKind::Sub, "x", "i", ""), run(".5")], - ] { - let lin = to_linear(&n); - assert_eq!(parse_linear(&lin), n, "{lin}"); - } - assert_eq!(to_linear(&[sc(ScriptKind::Sub, "x", "i", ""), run("y")]), "x_i y"); - // Nothing extra when the script already ends in a bracket or a symbol follows. - assert_eq!(to_linear(&[sc(ScriptKind::Sub, "x", "i", ""), run("+")]), "x_i+"); - } - - #[test] - fn a_function_name_in_a_script_stays_upright() { - // Word for Mac (#1609): `x_max` and `x_(max)` both give an upright function name. - let up = vec![MNode::Script { kind: ScriptKind::Sub, base: vec![run("x")], sub: vec![MNode::Run(MRun::plain("max"))], sup: vec![] }]; - assert_eq!(parse_linear("x_max"), up); - assert_eq!(parse_linear("x_(max)"), up); - assert_eq!(parse_linear(&to_linear(&up)), up, "{}", to_linear(&up)); - // Other upright text is not a function name: it keeps its parentheses. - let other = vec![MNode::Script { kind: ScriptKind::Sub, base: vec![run("x")], sub: vec![MNode::Run(MRun::plain("ab"))], sup: vec![] }]; - assert_eq!(to_linear(&other), "x_(ab)"); - } - - #[test] - fn digits_then_letters_continue_a_script() { - // Word for Mac (#1609): a script runs to a space or operator, so `x_2a` is subscript "2a". - let sub = |t: &str| vec![MNode::Script { kind: ScriptKind::Sub, base: vec![run("x")], sub: vec![run(t)], sup: vec![] }]; - assert_eq!(parse_linear("x_2a"), sub("2a")); - assert_eq!(parse_linear("x_a2"), sub("a2")); - assert_eq!(parse_linear("x_αβ"), sub("αβ")); - assert_eq!(parse_linear("x_10"), sub("10")); - assert_eq!(parse_linear("x_2.5"), sub("2.5")); - assert_eq!(to_linear(&sub("2a")), "x_2a"); - assert_eq!(parse_linear(&to_linear(&sub("2a"))), sub("2a")); - } - - #[test] - fn hostile_linear_never_panics() { - for s in ["", "/", "^", "_", "√", "∑", "((((", "))))", "■(", "█(@@&&", "a/", "/b", "^^^__", "√(&)", "∫_^▒", "lim_", "sin"] { - let _ = to_linear(&parse_linear(s)); - } - let deep = "(".repeat(5000) + &")".repeat(5000); - let _ = parse_linear(&deep); - let deep = "√".repeat(3000); - let _ = parse_linear(&deep); - } - - #[test] - fn a_long_run_of_letters_parses_in_linear_time() { - // Each letter that doesn't start a function name used to rescan (and copy) the rest of - // the run, so a 20,000-letter equation did 200 million steps. (No wall-clock assert: a - // loaded runner would fail it; a regression shows as a slow test.) - for _ in 0..5 { - let e = parse_linear(&"x".repeat(30_000)); - assert!(matches!(e.as_slice(), [MNode::Run(r)] if r.text.len() == 20_000), "{:?}", e.len()); - } - // Function names are still found at the start of a run, and only as whole runs. - assert!(matches!(parse_linear("sin x").as_slice(), [MNode::Func { .. }])); - assert!(matches!(parse_linear("arcsin x").as_slice(), [MNode::Func { .. }])); - assert!(matches!(parse_linear("sinx").as_slice(), [MNode::Run(_)])); - assert!(matches!(parse_linear("arcsinhx").as_slice(), [MNode::Run(_)])); - } -} diff --git a/crates/wordmodel/src/math_latex.rs b/crates/wordmodel/src/math_latex.rs deleted file mode 100644 index 92979534c..000000000 --- a/crates/wordmodel/src/math_latex.rs +++ /dev/null @@ -1,1242 +0,0 @@ -//! LaTeX ↔ math tree (Word's LaTeX equation input and conversion). -//! -//! Input covers the commands people type in equations: `\frac`, `\sqrt[n]{}`, scripts, big -//! operators with limits, `\left…\middle…\right`, matrix and `cases`/`aligned` environments, -//! accents, `\overbrace`/`\underbrace`, `\text`/`\mathrm`/`\mathbf`/`\mathbb`…, function names, -//! spacing and every named symbol in [`crate::math_symbols`]. Unknown commands stay as text. - -use crate::math::{ - Arg, ColJc, FracKind, MAX_DEPTH, MNode, MRun, MScr, MSty, ScriptKind, WRAP_DEPTH, any_too_deep, is_nary_char, math_class, merge_runs, too_deep, -}; -use crate::math_linear::{FUNCTION_NAMES, map_args}; -use crate::math_symbols::{name_of, symbol}; - -#[derive(Clone, Debug, PartialEq)] -enum Tok { - Cmd(String), - Open, - Close, - Sup, - Sub, - Amp, - Row, - Ch(char), -} - -/// Commands whose `{…}` argument is text, where spaces are part of the content. -const TEXT_COMMANDS: &[&str] = &["text", "textrm", "mbox", "textnormal", "textbf", "textit"]; - -fn tokenize(s: &str) -> Vec { - let cs: Vec = s.chars().take(20_000).collect(); - let mut out = Vec::new(); - let mut i = 0; - // A text command was just read: its `{` starts text, where spaces count (#448). - let mut text_next = false; - // Brace depth inside a text argument. - let mut text_depth: Option = None; - while let Some(&c) = cs.get(i) { - i += 1; - if text_depth.is_some() && matches!(c, ' ' | '\t' | '\n' | '\r') { - // A run of white space is one space, as in LaTeX. - if out.last() != Some(&Tok::Ch(' ')) { - out.push(Tok::Ch(' ')); - } - continue; - } - let was_text_next = std::mem::take(&mut text_next); - match c { - '\\' => { - let start = i; - while cs.get(i).is_some_and(char::is_ascii_alphabetic) { - i += 1; - } - if i == start { - match cs.get(i) { - Some('\\') => { - i += 1; - out.push(Tok::Row); - } - Some(&c) => { - i += 1; - out.push(Tok::Cmd(c.to_string())); - } - None => {} - } - } else { - let name: String = cs.get(start..i).unwrap_or(&[]).iter().collect(); - text_next = text_depth.is_none() && TEXT_COMMANDS.contains(&name.as_str()); - out.push(Tok::Cmd(name)); - } - } - '{' => { - text_depth = match text_depth { - Some(d) => Some(d.saturating_add(1)), - None if was_text_next => Some(0), - None => None, - }; - out.push(Tok::Open); - } - '}' => { - text_depth = match text_depth { - Some(0) | None => None, - Some(d) => Some(d.saturating_sub(1)), - }; - out.push(Tok::Close); - } - '^' => out.push(Tok::Sup), - '_' => out.push(Tok::Sub), - '&' => out.push(Tok::Amp), - // White space between a text command and its `{` doesn't end the wait. - ' ' | '\t' | '\n' | '\r' => text_next = was_text_next, - '~' => out.push(Tok::Ch('\u{A0}')), - c => out.push(Tok::Ch(c)), - } - } - out -} - -/// Parse LaTeX math into nodes. Never fails. -pub fn parse_latex(s: &str) -> Arg { - let mut p = Lp { t: tokenize(s), i: 0, depth: 0 }; - let mut out = Vec::new(); - while p.i < p.t.len() { - out.extend(p.seq(false)); - // A stray `}` or `&`/`\\` at the top level. - match p.t.get(p.i) { - Some(Tok::Amp) => out.push(run("&")), - Some(Tok::Row) => {} - Some(Tok::Close) => {} - _ => {} - } - p.i += 1; - } - let out = merge_runs(out); - if out.iter().any(|n| matches!(n, MNode::Run(r) if !r.nor && r.text.contains('#'))) { - return vec![MNode::EqArr { rows: vec![out] }]; - } - out -} - -fn run(t: &str) -> MNode { - MNode::Run(MRun::new(t)) -} - -struct Lp { - t: Vec, - i: usize, - depth: usize, -} - -impl Lp { - fn peek(&self) -> Option<&Tok> { - self.t.get(self.i) - } - - /// A sequence up to `}` (in a group), `&`, `\\`, `\right`, `\middle` or `\end`. - fn seq(&mut self, in_group: bool) -> Arg { - self.depth += 1; - let mut out: Arg = Vec::new(); - if self.depth > MAX_DEPTH { - self.i = self.t.len(); - self.depth -= 1; - return out; - } - while let Some(tok) = self.peek().cloned() { - match tok { - Tok::Close if in_group => break, - Tok::Close | Tok::Amp | Tok::Row => break, - Tok::Cmd(ref c) if matches!(c.as_str(), "right" | "middle" | "end") => break, - Tok::Sup | Tok::Sub => { - self.i += 1; - let s = self.arg(); - if out.last().is_some_and(too_deep) { - // Too deep to wrap (`x^1^1^1…`): the operator is dropped, the script kept. - out.extend(s); - } else { - let base = out.pop(); - out.push(script(base, tok == Tok::Sup, s)); - } - } - Tok::Cmd(ref c) if c == "limits" || c == "nolimits" || c == "displaystyle" || c == "textstyle" => { - self.i += 1; - if c == "nolimits" - && let Some(MNode::Nary { lim_loc, .. }) = out.last_mut() - { - *lim_loc = Some(crate::math::LimLoc::SubSup); - } - if c == "limits" - && let Some(MNode::Nary { lim_loc, .. }) = out.last_mut() - { - *lim_loc = Some(crate::math::LimLoc::UndOvr); - } - } - _ => { - let a = self.atom(); - out.extend(a); - } - } - } - self.depth -= 1; - // Big operators take the following term as their operand. - nary_operands(out, self.depth) - } - - /// A command argument: a `{group}` or one token. - fn arg(&mut self) -> Arg { - match self.peek() { - Some(Tok::Open) => { - self.i += 1; - let g = self.seq(true); - if self.peek() == Some(&Tok::Close) { - self.i += 1; - } - g - } - Some(_) => self.atom(), - None => Vec::new(), - } - } - - /// `[optional]` argument. - fn opt(&mut self) -> Option { - if self.peek() != Some(&Tok::Ch('[')) { - return None; - } - self.i += 1; - let mut toks = Vec::new(); - let mut level = 0i32; - while let Some(t) = self.peek().cloned() { - self.i += 1; - match t { - Tok::Ch(']') if level == 0 => break, - Tok::Open => level += 1, - Tok::Close => level -= 1, - _ => {} - } - toks.push(t); - } - let mut p = Lp { t: toks, i: 0, depth: self.depth }; - Some(p.seq(false)) - } - - /// The text of a `{…}` argument, verbatim. - fn raw_arg(&mut self) -> String { - let mut s = String::new(); - if self.peek() != Some(&Tok::Open) { - if let Some(Tok::Ch(c)) = self.peek().cloned() { - self.i += 1; - s.push(c); - } - return s; - } - self.i += 1; - let mut level = 0i32; - // The previous token was a control word (`\backslash{}`: the empty braces end its name). - let mut after_word = false; - while let Some(t) = self.peek().cloned() { - self.i += 1; - let was_word = std::mem::replace(&mut after_word, matches!(&t, Tok::Cmd(c) if c.chars().all(|c| c.is_ascii_alphabetic()))); - match t { - Tok::Close if level == 0 => break, - Tok::Open if was_word && self.peek() == Some(&Tok::Close) => self.i += 1, - Tok::Open => { - level += 1; - s.push('{'); - } - Tok::Close => { - level -= 1; - s.push('}'); - } - Tok::Ch(c) => s.push(c), - Tok::Cmd(c) => { - if c.chars().all(|c| c.is_ascii_alphabetic()) { - match symbol(&c) { - Some(sym) => s.push(sym), - None => s.push_str(&c), - } - } else { - s.push_str(&c); - } - if c == " " { - s.push(' '); - } - } - Tok::Sup => s.push('^'), - Tok::Sub => s.push('_'), - Tok::Amp => s.push('&'), - Tok::Row => s.push('\n'), - } - } - s - } - - /// A delimiter after `\left`, `\right`, `\middle`, `\big…`. - fn delim(&mut self) -> Option { - let t = self.peek().cloned()?; - self.i += 1; - match t { - Tok::Ch('.') => None, - Tok::Ch(c) => Some(c), - Tok::Cmd(c) => match c.as_str() { - "{" => Some('{'), - "}" => Some('}'), - "|" => Some('‖'), - other => symbol(other), - }, - _ => None, - } - } - - fn atom(&mut self) -> Arg { - if self.depth > MAX_DEPTH { - self.i = self.t.len(); - return Vec::new(); - } - self.depth += 1; - let out = self.atom_inner(); - self.depth -= 1; - out - } - - fn atom_inner(&mut self) -> Arg { - let Some(t) = self.peek().cloned() else { return Vec::new() }; - self.i += 1; - match t { - Tok::Open => { - let g = self.seq(true); - if self.peek() == Some(&Tok::Close) { - self.i += 1; - } - g - } - Tok::Ch(c) => vec![run(&c.to_string())], - Tok::Close | Tok::Amp | Tok::Row | Tok::Sup | Tok::Sub => Vec::new(), - Tok::Cmd(c) => self.command(&c), - } - } - - fn command(&mut self, c: &str) -> Arg { - let styled = |a: Arg, sty: Option, scr: MScr, nor: bool| -> Arg { restyle(a, sty, scr, nor) }; - let one = |n: MNode| vec![n]; - match c { - "frac" | "dfrac" | "tfrac" | "cfrac" => { - let num = self.arg(); - let den = self.arg(); - one(MNode::Frac { kind: FracKind::Bar, num, den }) - } - "genfrac" => { - let beg = delim_arg(&self.arg()); - let end = delim_arg(&self.arg()); - let thickness = self.arg(); - let _style = self.arg(); - let num = self.arg(); - let den = self.arg(); - let kind = if is_zero_length(&thickness) { FracKind::NoBar } else { FracKind::Bar }; - let frac = MNode::Frac { kind, num, den }; - if beg.is_none() && end.is_none() { - one(frac) - } else { - one(MNode::Delim { beg, end, sep: None, grow: true, shp_match: false, elems: vec![vec![frac]] }) - } - } - "binom" | "dbinom" | "tbinom" => { - let n = self.arg(); - let k = self.arg(); - one(MNode::Delim { - beg: Some('('), - end: Some(')'), - sep: None, - grow: true, - shp_match: false, - elems: vec![vec![MNode::Frac { kind: FracKind::NoBar, num: n, den: k }]], - }) - } - "sqrt" => { - let deg = self.opt().unwrap_or_default(); - let e = self.arg(); - one(MNode::Rad { deg_hide: deg.is_empty(), deg, e }) - } - "left" => { - let beg = self.delim(); - let mut elems = vec![self.seq(false)]; - let mut sep = None; - let mut end = None; - while let Some(Tok::Cmd(k)) = self.peek().cloned() { - self.i += 1; - if k == "middle" { - sep = self.delim().or(Some('|')); - elems.push(self.seq(false)); - } else { - if k == "right" { - end = self.delim(); - } - break; - } - } - one(MNode::Delim { beg, end, sep, grow: true, shp_match: false, elems }) - } - "bigl" | "Bigl" | "biggl" | "Biggl" | "bigr" | "Bigr" | "biggr" | "Biggr" | "big" | "Big" | "bigg" | "Bigg" => { - self.delim().map(|d| vec![run(&d.to_string())]).unwrap_or_default() - } - "begin" => { - let env = self.raw_arg(); - self.environment(env.trim()) - } - "text" | "textrm" | "mbox" | "textnormal" => { - let t = self.raw_arg(); - one(MNode::Run(MRun { text: t, nor: true, sty: Some(MSty::Plain), ..Default::default() })) - } - "textbf" => { - let t = self.raw_arg(); - one(MNode::Run(MRun { text: t, nor: true, sty: Some(MSty::Bold), ..Default::default() })) - } - "textit" => { - let t = self.raw_arg(); - one(MNode::Run(MRun { text: t, nor: true, sty: Some(MSty::Italic), ..Default::default() })) - } - "operatorname" => { - let starred = self.peek() == Some(&Tok::Ch('*')); - if starred { - self.i += 1; - } - let t = self.raw_arg(); - let name = vec![MNode::Run(MRun::plain(t))]; - if starred && self.peek() == Some(&Tok::Sub) { - self.i += 1; - let lim = self.arg(); - vec![MNode::Lim { upper: false, e: name, lim }] - } else { - self.apply(name) - } - } - "tag" => { - let starred = self.peek() == Some(&Tok::Ch('*')); - if starred { - self.i += 1; - } - let t = self.raw_arg(); - let num_str = if starred { format!("#{t}") } else { format!("# ({t})") }; - let mut r = MRun::new(num_str); - r.lit = false; - vec![MNode::Run(r)] - } - "mathrm" | "rm" => { - let a = self.arg(); - styled(a, Some(MSty::Plain), MScr::Roman, false) - } - "mathbf" | "bf" => { - let a = self.arg(); - styled(a, Some(MSty::Bold), MScr::Roman, false) - } - "mathit" => { - let a = self.arg(); - styled(a, Some(MSty::Italic), MScr::Roman, false) - } - "boldsymbol" | "bm" => { - let a = self.arg(); - styled(a, Some(MSty::BoldItalic), MScr::Roman, false) - } - "mathbb" => { - let a = self.arg(); - styled(a, None, MScr::DoubleStruck, false) - } - "mathcal" | "mathscr" => { - let a = self.arg(); - styled(a, None, MScr::Script, false) - } - "mathfrak" => { - let a = self.arg(); - styled(a, None, MScr::Fraktur, false) - } - "mathsf" => { - let a = self.arg(); - styled(a, None, MScr::SansSerif, false) - } - "mathtt" => { - let a = self.arg(); - styled(a, None, MScr::Monospace, false) - } - "overline" => one(MNode::Bar { top: true, e: self.arg() }), - "underline" => one(MNode::Bar { top: false, e: self.arg() }), - "boxed" | "fbox" => one(MNode::BorderBox { hide: [false; 4], strike: [false; 4], e: self.arg() }), - "phantom" => one(MNode::Phant { show: false, zero_wid: false, zero_asc: false, zero_desc: false, e: self.arg() }), - "overbrace" | "underbrace" | "overparen" | "underparen" | "overbracket" | "underbracket" => { - let top = c.starts_with("over"); - let chr = match c { - "overbrace" => '⏞', - "underbrace" => '⏟', - "overparen" => '⏜', - "underparen" => '⏝', - "overbracket" => '⎴', - _ => '⎵', - }; - let e = self.arg(); - let g = MNode::GroupChr { chr, top, e }; - let want = if top { Tok::Sup } else { Tok::Sub }; - if self.peek() == Some(&want) { - self.i += 1; - let lim = self.arg(); - one(MNode::Lim { upper: top, e: vec![g], lim }) - } else { - one(g) - } - } - "stackrel" | "overset" => { - let over = self.arg(); - let base = self.arg(); - one(MNode::Lim { upper: true, e: base, lim: over }) - } - "underset" => { - let under = self.arg(); - let base = self.arg(); - one(MNode::Lim { upper: false, e: base, lim: under }) - } - "hat" | "widehat" | "tilde" | "widetilde" | "dot" | "ddot" | "dddot" | "check" | "breve" | "acute" | "grave" | "vec" | "bar" - | "overrightarrow" | "overleftarrow" | "overleftrightarrow" | "mathring" => { - let chr = match c { - "hat" | "widehat" => '\u{302}', - "tilde" | "widetilde" => '\u{303}', - "dot" => '\u{307}', - "ddot" => '\u{308}', - "dddot" => '\u{20DB}', - "check" => '\u{30C}', - "breve" => '\u{306}', - "acute" => '\u{301}', - "grave" => '\u{300}', - "bar" => '\u{305}', - "overleftarrow" => '\u{20D6}', - "overleftrightarrow" => '\u{20E1}', - "mathring" => '\u{30A}', - _ => '\u{20D7}', - }; - one(MNode::Acc { chr, e: self.arg() }) - } - "," | "thinspace" => vec![run("\u{2009}")], - ":" | ">" | "medspace" => vec![run("\u{205F}")], - ";" | "thickspace" => vec![run("\u{2005}")], - "!" | "negthinspace" => Vec::new(), - " " => vec![run(" ")], - "qquad" => vec![run("\u{2003}\u{2003}")], - "{" | "}" | "%" | "$" | "#" | "&" | "_" => { - // Escaped characters (a literal `#` must not number the equation). - let mut r = MRun::new(c); - r.lit = c == "#"; - vec![MNode::Run(r)] - } - "|" => vec![run("‖")], - name if FUNCTION_NAMES.contains(&name) => { - let plain = vec![MNode::Run(MRun::plain(name))]; - let limits = matches!(name, "lim" | "liminf" | "limsup" | "max" | "min" | "sup" | "inf" | "det" | "gcd" | "Pr"); - let named = if limits && self.peek() == Some(&Tok::Sub) { - self.i += 1; - let lim = self.arg(); - vec![MNode::Lim { upper: false, e: plain, lim }] - } else if matches!(self.peek(), Some(Tok::Sup) | Some(Tok::Sub)) { - let sup = self.peek() == Some(&Tok::Sup); - self.i += 1; - let s = self.arg(); - vec![script(Some(MNode::Run(MRun::plain(name))), sup, s)] - } else { - plain - }; - self.apply(named) - } - name => match symbol(name) { - Some(ch) if is_nary_char(ch) => vec![MNode::Nary { - chr: ch, - lim_loc: None, - grow: false, - sub_hide: true, - sup_hide: true, - sub: Vec::new(), - sup: Vec::new(), - e: Vec::new(), - }], - Some(ch) => vec![run(&ch.to_string())], - None => vec![run(&format!("\\{name}"))], - }, - } - } - - /// A function name applied to the next atom (with its scripts). - fn apply(&mut self, name: Arg) -> Arg { - let mut e = self.atom(); - while let Some(t @ (Tok::Sup | Tok::Sub)) = self.peek().cloned() { - self.i += 1; - let s = self.arg(); - if any_too_deep(&e) { - e.extend(s); - continue; - } - let base = if e.len() == 1 { e.pop() } else { Some(group(std::mem::take(&mut e))) }; - e.push(script(base, t == Tok::Sup, s)); - } - if e.is_empty() { - return name; - } - vec![MNode::Func { name, e }] - } - - fn environment(&mut self, env: &str) -> Arg { - let base = env.trim_end_matches('*'); - let mut col_jc = Vec::new(); - if base == "array" || base == "tabular" { - let _ = self.opt(); - let spec = self.raw_arg(); - for ch in spec.chars() { - match ch { - 'l' | 'L' => col_jc.push(ColJc::Left), - 'r' | 'R' => col_jc.push(ColJc::Right), - 'c' | 'C' => col_jc.push(ColJc::Center), - _ => {} - } - } - } else if base == "aligned" { - let _ = self.opt(); - } - // Rows of `&`-separated cells up to `\end{…}`. - let mut rows: Vec> = vec![vec![]]; - loop { - let cell = self.seq(false); - if let Some(r) = rows.last_mut() { - r.push(cell); - } - match self.peek().cloned() { - Some(Tok::Amp) => self.i += 1, - Some(Tok::Row) => { - self.i += 1; - rows.push(vec![]); - } - Some(Tok::Cmd(k)) if k == "end" => { - self.i += 1; - let _ = self.raw_arg(); - break; - } - Some(Tok::Close) => self.i += 1, - Some(_) => self.i += 1, - None => break, - } - } - // A trailing `\\` leaves an empty last row. - if rows.len() > 1 && rows.last().is_some_and(|r| r.iter().all(|c| c.is_empty())) { - rows.pop(); - } - let delims = match base { - "pmatrix" => Some((Some('('), Some(')'))), - "bmatrix" => Some((Some('['), Some(']'))), - "Bmatrix" => Some((Some('{'), Some('}'))), - "vmatrix" => Some((Some('|'), Some('|'))), - "Vmatrix" => Some((Some('‖'), Some('‖'))), - _ => None, - }; - let wrap = |inner: MNode, beg: Option, end: Option| MNode::Delim { - beg, - end, - sep: None, - grow: true, - shp_match: false, - elems: vec![vec![inner]], - }; - match base { - "matrix" | "pmatrix" | "bmatrix" | "Bmatrix" | "vmatrix" | "Vmatrix" | "smallmatrix" | "array" | "tabular" => { - let m = MNode::Matrix { rows, col_jc }; - match delims { - Some((b, e)) => vec![wrap(m, b, e)], - None => vec![m], - } - } - "cases" | "dcases" => { - let eq = MNode::EqArr { rows: rows.into_iter().map(join_cells).collect() }; - vec![wrap(eq, Some('{'), None)] - } - "rcases" => { - let eq = MNode::EqArr { rows: rows.into_iter().map(join_cells).collect() }; - vec![wrap(eq, None, Some('}'))] - } - _ => vec![MNode::EqArr { rows: rows.into_iter().map(join_cells).collect() }], - } - } -} - -/// Cells of an aligned row joined with `&` alignment marks. -fn join_cells(cells: Vec) -> Arg { - let mut out = Vec::new(); - for (i, c) in cells.into_iter().enumerate() { - if i > 0 { - out.push(run("&")); - } - out.extend(c); - } - merge_runs(out) -} - -fn group(a: Arg) -> MNode { - MNode::Delim { beg: None, end: None, sep: None, grow: true, shp_match: false, elems: vec![a] } -} - -fn script(base: Option, sup: bool, s: Arg) -> MNode { - match base { - Some(MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub, sup: sp, e }) if e.is_empty() => { - if sup { - MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide: false, sub, sup: s, e } - } else { - MNode::Nary { chr, lim_loc, grow, sub_hide: false, sup_hide, sub: s, sup: sp, e } - } - } - Some(MNode::Script { kind: ScriptKind::Sub, base, sub, .. }) if sup => MNode::Script { kind: ScriptKind::SubSup, base, sub, sup: s }, - Some(MNode::Script { kind: ScriptKind::Sup, base, sup: sp, .. }) if !sup => MNode::Script { kind: ScriptKind::SubSup, base, sub: s, sup: sp }, - b => { - let base = b.map(|b| vec![b]).unwrap_or_default(); - if sup { - MNode::Script { kind: ScriptKind::Sup, base, sub: Vec::new(), sup: s } - } else { - MNode::Script { kind: ScriptKind::Sub, base, sub: s, sup: Vec::new() } - } - } - } -} - -/// Big operators without an operand take the atoms after them up to a relation or binary operator. -/// `depth` is how deep the operators already sit; past [`WRAP_DEPTH`] they stay without operand -/// (`\sum\sum\sum…` must not nest once per operator). -fn nary_operands(a: Arg, depth: usize) -> Arg { - let mut out: Arg = Vec::with_capacity(a.len()); - let mut it = a.into_iter().peekable(); - while let Some(n) = it.next() { - match n { - MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub, sup, e } if e.is_empty() && depth < WRAP_DEPTH => { - let mut body = Vec::new(); - while let Some(next) = it.peek() { - let stop = match next { - MNode::Run(r) => r.text.chars().next().is_some_and(|c| { - matches!(math_class(c), crate::math::MClass::Rel | crate::math::MClass::Bin | crate::math::MClass::Punct) || c == '&' - }), - _ => false, - }; - if stop { - break; - } - if let Some(x) = it.next() { - body.push(x); - } - } - let body = merge_runs(nary_operands(body, depth + 1)); - if any_too_deep(&body) { - // No room to wrap the operand: it follows the operator. - out.push(MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub, sup, e }); - out.extend(body); - } else { - out.push(MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub, sup, e: body }); - } - } - other => out.push(other), - } - } - out -} - -/// Set a style on every run inside. -fn restyle(a: Arg, sty: Option, scr: MScr, nor: bool) -> Arg { - a.into_iter() - .map(|n| match n { - MNode::Run(mut r) => { - if sty.is_some() { - r.sty = sty; - } - if scr != MScr::Roman { - r.scr = scr; - } - r.nor |= nor; - MNode::Run(r) - } - other => map_args(other, &|a| restyle(a, sty, scr, nor)), - }) - .collect() -} - -// --------------------------------------------------------------------------------------------- -// Out. - -/// The equation as LaTeX. -pub fn to_latex(nodes: &[MNode]) -> String { - let mut s = String::new(); - arg(nodes, &mut s, 0); - s.trim().to_string() -} - -fn arg(a: &[MNode], s: &mut String, depth: usize) { - if depth > MAX_DEPTH { - return; - } - for n in a { - node(n, s, depth + 1); - } -} - -fn braced(a: &[MNode], s: &mut String, depth: usize) { - s.push('{'); - arg(a, s, depth); - s.push('}'); -} - -fn chars(text: &str, s: &mut String) { - for c in text.chars() { - match c { - '{' | '}' | '%' | '$' | '#' | '&' | '_' => { - s.push('\\'); - s.push(c); - } - '\\' => s.push_str("\\backslash "), - '−' => s.push('-'), - c if c.is_ascii() => s.push(c), - c => match name_of(c) { - Some(n) if !matches!(n, "sqrt" | "matrix" | "of" | "above" | "below" | "box" | "rect") => { - s.push('\\'); - s.push_str(n); - s.push(' '); - } - _ => s.push(c), - }, - } - } -} - -/// Text inside `\text{…}`: only what LaTeX would read as markup is escaped. -fn text_chars(text: &str, s: &mut String) { - for c in text.chars() { - match c { - '{' | '}' | '%' | '$' | '#' | '&' | '_' | '~' => { - s.push('\\'); - s.push(c); - } - // `{}` ends the command name without adding a space (spaces count in text). - '\\' => s.push_str("\\backslash{}"), - c => s.push(c), - } - } -} - -/// The delimiter of a `\genfrac` argument: one character, or none for `{}` and anything else. -fn delim_arg(a: &[MNode]) -> Option { - let [MNode::Run(r)] = a else { return None }; - let mut cs = r.text.chars(); - match (cs.next(), cs.next()) { - (Some(c), None) if c != '.' => Some(c), - _ => None, - } -} - -/// Whether a `\genfrac` thickness is a zero length such as `0pt`, `0.0pt` or `0 em`. An empty -/// thickness means the default rule and anything that is not a plain number and unit is not zero. -fn is_zero_length(a: &[MNode]) -> bool { - let mut text = String::new(); - for n in a { - let MNode::Run(r) = n else { return false }; - text.extend(r.text.chars().filter(|c| !c.is_whitespace() && !matches!(c, '\u{2009}' | '\u{205F}' | '\u{2005}'))); - } - let number = text.trim_end_matches(|c: char| c.is_ascii_alphabetic()); - let unit = &text[number.len()..]; - !number.is_empty() - && matches!(unit, "pt" | "em" | "ex" | "mm" | "cm" | "in" | "bp" | "pc" | "sp" | "mu") - && number.parse::().is_ok_and(|v| v == 0.0) -} - -fn delim_tex(c: Option) -> String { - match c { - None => ".".into(), - Some('{') => "\\{".into(), - Some('}') => "\\}".into(), - Some('‖') => "\\|".into(), - Some(c) if c.is_ascii() => c.to_string(), - Some(c) => name_of(c).map(|n| format!("\\{n} ")).unwrap_or_else(|| c.to_string()), - } -} - -fn group_chr_tex(chr: char, top: bool) -> &'static str { - match chr { - '⏜' => "\\overparen", - '⏝' => "\\underparen", - '⎴' => "\\overbracket", - '⎵' => "\\underbracket", - _ if top => "\\overbrace", - _ => "\\underbrace", - } -} - -fn node(node: &MNode, output: &mut String, depth: usize) { - match node { - MNode::Run(run) => run_node(run, output), - MNode::Frac { kind, num, den } => fraction_node(*kind, num, den, output, depth), - MNode::Script { kind, base, sub, sup } => script_node(*kind, base, sub, sup, output, depth), - MNode::Rad { deg, deg_hide, e } => radical_node(deg, *deg_hide, e, output, depth), - MNode::Nary { .. } => nary_node(node, output, depth), - MNode::Delim { beg, end, sep, elems, .. } => delimiter_node(*beg, *end, *sep, elems, output, depth), - MNode::Func { name, e } => { - arg(name, output, depth); - braced(e, output, depth); - } - MNode::Lim { upper, e, lim } => limit_node(*upper, e, lim, output, depth), - MNode::Acc { chr, e } => accent_node(*chr, e, output, depth), - MNode::Bar { top, e } => { - output.push_str(if *top { "\\overline" } else { "\\underline" }); - braced(e, output, depth); - } - MNode::BorderBox { e, .. } => { - output.push_str("\\boxed"); - braced(e, output, depth); - } - MNode::Boxed { e } => braced(e, output, depth), - MNode::GroupChr { chr, top, e } => { - output.push_str(group_chr_tex(*chr, *top)); - braced(e, output, depth); - } - MNode::EqArr { rows } => equation_array_node(rows, output, depth), - MNode::Matrix { rows, .. } => matrix_node(rows, output, depth), - MNode::Phant { e, .. } => { - output.push_str("\\phantom"); - braced(e, output, depth); - } - } -} - -fn run_node(run: &MRun, output: &mut String) { - let (open, close) = if run.nor { - ("\\text{", "}") - } else { - match (run.sty, run.scr) { - (_, MScr::DoubleStruck) => ("\\mathbb{", "}"), - (_, MScr::Script) => ("\\mathcal{", "}"), - (_, MScr::Fraktur) => ("\\mathfrak{", "}"), - (_, MScr::SansSerif) => ("\\mathsf{", "}"), - (_, MScr::Monospace) => ("\\mathtt{", "}"), - (Some(MSty::Bold), _) => ("\\mathbf{", "}"), - (Some(MSty::BoldItalic), _) => ("\\boldsymbol{", "}"), - (Some(MSty::Plain), _) if run.text.chars().any(char::is_alphabetic) => { - if FUNCTION_NAMES.contains(&run.text.as_str()) { - output.push('\\'); - output.push_str(&run.text); - output.push(' '); - return; - } - ("\\mathrm{", "}") - } - _ => ("", ""), - } - }; - output.push_str(open); - if run.nor { - text_chars(&run.text, output) - } else { - chars(&run.text, output) - } - output.push_str(close); -} - -fn fraction_node(kind: FracKind, numerator: &[MNode], denominator: &[MNode], output: &mut String, depth: usize) { - match kind { - FracKind::NoBar => { - output.push_str("\\genfrac{}{}{0pt}{}"); - braced(numerator, output, depth); - braced(denominator, output, depth); - } - FracKind::Linear | FracKind::Skewed => { - braced(numerator, output, depth); - output.push('/'); - braced(denominator, output, depth); - } - FracKind::Bar => { - output.push_str("\\frac"); - braced(numerator, output, depth); - braced(denominator, output, depth); - } - } -} - -fn script_node(kind: ScriptKind, base: &[MNode], sub: &[MNode], sup: &[MNode], output: &mut String, depth: usize) { - if kind == ScriptKind::Pre { - output.push_str("{}_"); - braced(sub, output, depth); - output.push('^'); - braced(sup, output, depth); - braced(base, output, depth); - return; - } - braced(base, output, depth); - if matches!(kind, ScriptKind::Sub | ScriptKind::SubSup) { - output.push('_'); - braced(sub, output, depth); - } - if matches!(kind, ScriptKind::Sup | ScriptKind::SubSup) { - output.push('^'); - braced(sup, output, depth); - } -} - -fn radical_node(degree: &[MNode], degree_hidden: bool, expression: &[MNode], output: &mut String, depth: usize) { - output.push_str("\\sqrt"); - if !degree_hidden && !degree.is_empty() { - output.push('['); - arg(degree, output, depth); - output.push(']'); - } - braced(expression, output, depth); -} - -fn nary_node(node: &MNode, output: &mut String, depth: usize) { - let MNode::Nary { chr, lim_loc, sub_hide, sup_hide, sub, sup, e, .. } = node else { return }; - match name_of(*chr) { - Some(name) => { - output.push('\\'); - output.push_str(name); - } - None => output.push(*chr), - } - match lim_loc { - Some(crate::math::LimLoc::UndOvr) => output.push_str("\\limits"), - Some(crate::math::LimLoc::SubSup) if !crate::math::is_integral(*chr) => output.push_str("\\nolimits"), - _ => {} - } - if !sub_hide && !sub.is_empty() { - output.push('_'); - braced(sub, output, depth); - } - if !sup_hide && !sup.is_empty() { - output.push('^'); - braced(sup, output, depth); - } - output.push(' '); - braced(e, output, depth); -} - -fn delimiter_node(begin: Option, end: Option, separator: Option, elements: &[Arg], output: &mut String, depth: usize) { - output.push_str("\\left"); - output.push_str(&delim_tex(begin)); - let middle = separator.map(|c| format!("\\middle{}", delim_tex(Some(c)))).unwrap_or_else(|| "\\middle|".to_string()); - for (index, element) in elements.iter().enumerate() { - if index > 0 { - output.push_str(&middle); - } - arg(element, output, depth); - } - output.push_str("\\right"); - output.push_str(&delim_tex(end)); -} - -fn limit_node(upper: bool, expression: &[MNode], limit: &[MNode], output: &mut String, depth: usize) { - let simple = matches!(expression, [MNode::Run(run)] if run.sty == Some(MSty::Plain) && FUNCTION_NAMES.contains(&run.text.as_str())); - if simple && !upper { - arg(expression, output, depth); - output.push('_'); - braced(limit, output, depth); - } else if let [MNode::GroupChr { chr, top, e: inner }] = expression { - output.push_str(group_chr_tex(*chr, *top)); - braced(inner, output, depth); - output.push(if upper { '^' } else { '_' }); - braced(limit, output, depth); - } else { - output.push_str(if upper { "\\overset" } else { "\\underset" }); - braced(limit, output, depth); - braced(expression, output, depth); - } -} - -fn accent_node(character: char, expression: &[MNode], output: &mut String, depth: usize) { - let command = match character { - '\u{302}' => "hat", - '\u{303}' => "tilde", - '\u{307}' => "dot", - '\u{308}' => "ddot", - '\u{20DB}' => "dddot", - '\u{30C}' => "check", - '\u{306}' => "breve", - '\u{301}' => "acute", - '\u{300}' => "grave", - '\u{305}' | '\u{AF}' => "bar", - '\u{20D6}' => "overleftarrow", - '\u{20E1}' => "overleftrightarrow", - _ => "vec", - }; - output.push('\\'); - output.push_str(command); - braced(expression, output, depth); -} - -fn equation_array_node(rows: &[Arg], output: &mut String, depth: usize) { - if rows.len() == 1 - && let Some(row) = rows.first() - && crate::math::has_number_mark(row) - { - arg(row, output, depth); - return; - } - output.push_str("\\begin{aligned}"); - for (index, row) in rows.iter().enumerate() { - if index > 0 { - output.push_str("\\\\"); - } - for child_node in row { - match child_node { - MNode::Run(run) if !run.lit && run.text.contains('&') => { - for (index, part) in run.text.split('&').enumerate() { - if index > 0 { - output.push('&'); - } - let mut piece = run.clone(); - piece.text = part.to_string(); - run_node(&piece, output); - } - } - other => node(other, output, depth + 1), - } - } - } - output.push_str("\\end{aligned}"); -} - -fn matrix_node(rows: &[Vec], output: &mut String, depth: usize) { - output.push_str("\\begin{matrix}"); - for (row_index, row) in rows.iter().enumerate() { - if row_index > 0 { - output.push_str("\\\\"); - } - for (column, cell) in row.iter().enumerate() { - if column > 0 { - output.push('&'); - } - arg(cell, output, depth); - } - } - output.push_str("\\end{matrix}"); -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::math::{LimLoc, MathJc}; - - #[test] - fn latex_structures() { - let n = parse_latex(r"x=\frac{-b\pm\sqrt{b^2-4ac}}{2a}"); - assert_eq!(crate::math::to_linear(&n), "x=(-b±√(b^2-4ac))/2a"); - let n = parse_latex(r"\sum_{i=1}^{n} i^2 = \frac{n(n+1)(2n+1)}{6}"); - let MNode::Nary { chr, sub, sup, e, .. } = &n[0] else { panic!("{n:#?}") }; - assert_eq!((*chr, sub.len(), sup.len()), ('∑', 1, 1)); - assert!(matches!(e.first(), Some(MNode::Script { .. })), "{e:#?}"); - let n = parse_latex(r"\begin{pmatrix} a & b \\ c & d \end{pmatrix}"); - assert!( - matches!(&n[0], MNode::Delim { beg: Some('('), elems, .. } if matches!(elems[0][0], MNode::Matrix { ref rows, .. } if rows.len() == 2)) - ); - let n = parse_latex(r"f(x)=\begin{cases} x & x\ge 0\\ -x & x<0\end{cases}"); - assert!(n.iter().any(|m| matches!(m, MNode::Delim { beg: Some('{'), end: None, .. })), "{n:#?}"); - let n = parse_latex(r"\lim_{n\to\infty}\left(1+\frac{1}{n}\right)^n"); - assert!(matches!(&n[0], MNode::Func { .. }), "{n:#?}"); - let n = parse_latex(r"\mathbb{R}\hat{x}\text{ if }\overbrace{a+b}^{k}"); - assert!(matches!(&n[0], MNode::Run(r) if r.scr == MScr::DoubleStruck)); - let _ = (LimLoc::UndOvr, MathJc::Center); - } - - #[test] - fn latex_binom_round_trip() { - let a = parse_latex(r"\binom{n}{k}"); - let l = to_latex(&a); - assert_eq!(parse_latex(&l), a, "exported as {l}"); - let nobar = |n: &str, k: &str| MNode::Frac { kind: FracKind::NoBar, num: parse_latex(n), den: parse_latex(k) }; - let b = parse_latex(r"\genfrac{}{}{0pt}{}{n}{k}"); - assert_eq!(b, vec![nobar("n", "k")]); - assert_eq!(parse_latex(&to_latex(&b)), b); - // Other spellings of a zero thickness. - for z in ["0 pt", "0.0pt", r"0\,pt", "0em"] { - let c = parse_latex(&format!(r"\genfrac{{}}{{}}{{{z}}}{{}}{{n}}{{k}}")); - assert_eq!(c, vec![nobar("n", "k")], "thickness {z}"); - } - // A non-zero or default thickness keeps the bar. - for t in ["1pt", "0.4pt", ""] { - let c = parse_latex(&format!(r"\genfrac{{}}{{}}{{{t}}}{{}}{{n}}{{k}}")); - let bar = MNode::Frac { kind: FracKind::Bar, num: parse_latex("n"), den: parse_latex("k") }; - assert_eq!(c, vec![bar], "thickness {t:?}"); - } - // Delimiters are kept. - let d = parse_latex(r"\genfrac(){0pt}{}{n}{k}"); - assert_eq!(d, a); - let e = parse_latex(r"\genfrac[]{0pt}{}{n}{k}"); - assert!(matches!(&e[..], [MNode::Delim { beg: Some('['), end: Some(']'), .. }]), "{e:#?}"); - assert_eq!(parse_latex(&to_latex(&e)), e); - } - - #[test] - fn latex_round_trip() { - for lin in ["x=(-b±√(b^2-4ac))/2a", "∑_(i=1)^n▒i^2", "√(3&x)", "a⃗+x̂", "■(a&b@c&d)", "sin⁡x+cos⁡y", "e^(-x^2)"] { - let n = crate::math::parse_linear(lin); - let tex = to_latex(&n); - let back = parse_latex(&tex); - assert_eq!(crate::math::to_linear(&back), crate::math::to_linear(&n), "{lin} → {tex}"); - } - } - - #[test] - fn group_character_latex_roundtrip() { - for cmd in ["\\overparen{a+b}", "\\underparen{a+b}", "\\overbracket{x}", "\\underbracket{x}"] { - let n = parse_latex(cmd); - let tex = to_latex(&n); - assert_eq!(tex, cmd); - } - } - - #[test] - fn hostile_latex_never_panics() { - for s in ["", "\\", "{", "}", "^", "_", "\\frac", "\\sqrt[", "\\left(", "\\right)", "\\begin{matrix", "&&\\\\", "{{{{", "\\end{x}", "a^^b__c"] - { - let _ = to_latex(&parse_latex(s)); - } - let deep = "{".repeat(5000); - let _ = parse_latex(&deep); - let deep = "\\frac{".repeat(3000); - let _ = parse_latex(&deep); - let deep = "\\sqrt".repeat(5000); - let _ = parse_latex(&deep); - let deep = "\\hat".repeat(5000) + "x"; - let _ = parse_latex(&deep); - } - - #[test] - fn array_column_specs_and_aligned_optional_args() { - let n = parse_latex(r"\begin{array}{lcr} a & b & c \\ d & e & f \end{array}"); - let MNode::Matrix { rows, col_jc } = &n[0] else { panic!("{n:#?}") }; - assert_eq!(col_jc, &[ColJc::Left, ColJc::Center, ColJc::Right]); - assert_eq!(rows.len(), 2); - assert_eq!(crate::math::to_linear(&rows[0][0]), "a"); - assert_eq!(crate::math::to_linear(&rows[0][1]), "b"); - assert_eq!(crate::math::to_linear(&rows[0][2]), "c"); - - let n = parse_latex(r"\begin{aligned}[t] x &= 1 \\ y &= 2 \end{aligned}"); - assert!(matches!(&n[0], MNode::EqArr { rows } if rows.len() == 2)); - } - - #[test] - fn operatorname_starred_and_tag() { - let n = parse_latex(r"\operatorname*{argmin}_{x} f(x)"); - assert!(matches!(&n[0], MNode::Lim { upper: false, .. })); - - let n = parse_latex(r"E=mc^2 \tag{1}"); - let MNode::EqArr { rows } = &n[0] else { panic!("{n:#?}") }; - assert!(crate::math::has_number_mark(&rows[0])); - - let n = parse_latex(r"E=mc^2 \tag*{(A)}"); - let MNode::EqArr { rows } = &n[0] else { panic!("{n:#?}") }; - assert!(crate::math::has_number_mark(&rows[0])); - } - - #[test] - fn delim_preserves_custom_separator() { - let delim = MNode::Delim { - beg: Some('{'), - end: Some('}'), - sep: Some('‖'), - grow: true, - shp_match: false, - elems: vec![vec![MNode::Run(MRun::new("x"))], vec![MNode::Run(MRun::new("y"))]], - }; - let tex = to_latex(&[delim]); - assert!(tex.contains(r"\middle\|"), "Expected \\middle\\|, got: {tex}"); - } -} diff --git a/crates/wordmodel/src/math_linear.rs b/crates/wordmodel/src/math_linear.rs deleted file mode 100644 index 39a25ab5d..000000000 --- a/crates/wordmodel/src/math_linear.rs +++ /dev/null @@ -1,762 +0,0 @@ -//! The linear format (UnicodeMath, the plain text Word builds equations from) → math tree. -//! -//! Grammar, informally: an expression is a sequence of atoms and operators. Atoms are characters, -//! bracket groups (`(…)`, `[…]`, `{…}`, `|…|`, `〖…〗` invisible, `├…┤` one-sided, `│` -//! separators), prefix structures (`√x`, `∑_a^b▒x`, `█(…)`, `■(…)`, `▭x`, `□x`, `¯x`, `▁x`, -//! `⏞x`, `⏟x`, `⟡x`), function names (`sin x`, `log_2 x`, `lim┬(n→∞) x`) and placeholders (`⬚`). -//! Postfix operators apply to the atom before them: `^` `_` scripts, combining accents, `┬` `┴` -//! limits. `/` `⁄` `∕` `¦` take the operands on either side (an operand is a run of atoms up to -//! an operator or space; a parenthesised operand loses its parentheses). `^`/`_` with nothing -//! before them are pre-scripts. A top-level `#` numbers the equation (`E=mc^2#(1)`). - -use crate::math::{Arg, FracKind, MAX_DEPTH, MClass, MNode, MRun, ScriptKind, any_too_deep, is_nary_char, math_class, merge_runs, too_deep}; - -/// Function names set upright and applied to the following operand. -pub const FUNCTION_NAMES: &[&str] = &[ - "arccos", "arccot", "arccsc", "arcsec", "arcsin", "arctan", "arg", "cos", "cosh", "cot", "coth", "csc", "csch", "deg", "det", "dim", "exp", - "gcd", "hom", "inf", "ker", "lg", "lim", "liminf", "limsup", "ln", "log", "max", "min", "mod", "Pr", "sec", "sech", "sin", "sinh", "sup", "tan", - "tanh", -]; - -/// Length of the longest of [`FUNCTION_NAMES`]. -const MAX_NAME: usize = { - let (mut max, mut i) = (0, 0); - while i < FUNCTION_NAMES.len() { - if FUNCTION_NAMES[i].len() > max { - max = FUNCTION_NAMES[i].len(); - } - i += 1; - } - max -}; - -/// Function names whose subscripts go underneath (`lim_(n→∞)`). -const LIMIT_NAMES: &[&str] = &["lim", "liminf", "limsup", "max", "min", "sup", "inf", "det", "gcd", "Pr"]; - -/// The empty-argument placeholder. -pub const PLACEHOLDER: char = '⬚'; - -/// Parse linear-format text. Never fails: what isn't understood stays as text. -pub fn parse(s: &str) -> Arg { - let mut p = P { c: s.chars().take(20_000).collect(), i: 0, depth: 0, no_limits: 0, steps: 0 }; - let mut out = Vec::new(); - // Stray closers at the top level are text. - while p.i < p.c.len() { - out.extend(p.expr(&[])); - if let Some(c) = p.peek() { - p.i += 1; - out.push(text(c)); - } - } - let mut out = clean(out); - // `expr#(n)`: a numbered equation. - if out.iter().any(|n| matches!(n, MNode::Run(r) if r.text.contains('#'))) { - out = vec![MNode::EqArr { rows: vec![out] }]; - } - out -} - -fn text(c: char) -> MNode { - MNode::Run(MRun::new(c.to_string())) -} - -/// Closing bracket for an opening one. -pub fn closer(c: char) -> Option { - Some(match c { - '(' => ')', - '[' => ']', - '{' => '}', - '⟨' => '⟩', - '〈' => '〉', - '⌈' => '⌉', - '⌊' => '⌋', - '⟦' => '⟧', - '〖' => '〗', - _ => return None, - }) -} - -fn is_closer(c: char) -> bool { - matches!(c, ')' | ']' | '}' | '⟩' | '〉' | '⌉' | '⌋' | '⟧' | '〗' | '┤') -} - -/// Prefix characters that make a structure from the operand after them. -fn is_prefix(c: char) -> bool { - matches!(c, '√' | '∛' | '∜' | '█' | '■' | '▭' | '□' | '¯' | '▁' | '⟡' | '⏞' | '⏟' | '⏜' | '⏝' | '⎴' | '⎵') || is_nary_char(c) -} - -/// Combining marks used as accents. -pub fn is_combining(c: char) -> bool { - matches!(c as u32, 0x300..=0x36F | 0x20D0..=0x20FF) -} - -/// Characters that end an operand. -fn is_boundary(c: char) -> bool { - matches!(c, ' ' | '&' | '@' | '#' | '▒' | '│' | '/' | '⁄' | '∕' | '¦' | '\u{2061}') - || is_closer(c) - || matches!(math_class(c), MClass::Bin | MClass::Rel | MClass::Punct) -} - -struct P { - c: Vec, - i: usize, - depth: usize, - /// Inside a brace's operand: limits are left for the brace. - no_limits: usize, - /// Work done (bounds hostile input). - steps: usize, -} - -const MAX_STEPS: usize = 400_000; - -impl P { - fn peek(&self) -> Option { - self.c.get(self.i).copied() - } - fn peek_at(&self, k: usize) -> Option { - self.c.get(self.i + k).copied() - } - fn tired(&mut self) -> bool { - self.steps += 1; - self.steps > MAX_STEPS || self.depth > MAX_DEPTH - } - /// The rest of the input as text (when too deep or too long). - fn rest(&mut self) -> Arg { - let t: String = self.c.get(self.i..).unwrap_or(&[]).iter().collect(); - self.i = self.c.len(); - if t.is_empty() { Vec::new() } else { vec![MNode::Run(MRun::new(t))] } - } - - /// A sequence up to one of `stops` (not consumed) or the end. - fn expr(&mut self, stops: &[char]) -> Arg { - self.depth += 1; - let mut out: Arg = Vec::new(); - if self.tired() { - self.depth -= 1; - return self.rest(); - } - while let Some(c) = self.peek() { - if stops.contains(&c) || (is_closer(c) && !stops.is_empty()) { - break; - } - if self.tired() { - out.extend(self.rest()); - break; - } - match c { - '/' | '⁄' | '∕' | '¦' => { - self.i += 1; - let start = operand_start(&out); - // A numerator that is already deep stays as it is: `/` is text then. - if out.get(start..).is_some_and(any_too_deep) { - out.push(text(c)); - continue; - } - let num = strip_parens(out.drain(start..).collect()); - let den = self.operand(stops); - let kind = match c { - '⁄' => FracKind::Skewed, - '∕' => FracKind::Linear, - '¦' => FracKind::NoBar, - _ => FracKind::Bar, - }; - out.push(MNode::Frac { kind, num, den }); - } - '^' | '_' => { - let pre = match out.last() { - None => true, - Some(MNode::Run(r)) => r.text == " " || r.text.chars().all(|c| is_boundary(c) && c != '#'), - _ => false, - }; - if pre { - out.push(self.prescript(stops)); - } else { - self.i += 1; - let s = self.script_operand(stops); - if out.last().is_some_and(too_deep) { - out.push(text(c)); - out.extend(s); - } else { - let base = out.pop(); - out.push(attach_script(base, c == '^', s)); - } - } - } - '┬' | '┴' => { - self.i += 1; - let lim = self.script_operand(stops); - if out.last().is_some_and(too_deep) { - out.push(text(c)); - out.extend(lim); - } else { - let e = out.pop().map(|n| vec![n]).unwrap_or_default(); - out.push(MNode::Lim { upper: c == '┴', e: strip_parens(e), lim }); - } - } - c if is_combining(c) => { - self.i += 1; - if out.last().is_some_and(too_deep) { - out.push(text(c)); - } else { - let e = out.pop().map(|n| strip_parens(vec![n])).unwrap_or_default(); - out.push(MNode::Acc { chr: c, e }); - } - } - '▒' | '\u{2061}' => self.i += 1, - ' ' => { - self.i += 1; - out.push(text(' ')); - } - _ => { - let atom = self.atom(stops); - out.extend(atom); - } - } - } - self.depth -= 1; - out - } - - /// One atom (a character, a bracket group, a structure or a function application). - fn atom(&mut self, stops: &[char]) -> Arg { - let Some(c) = self.peek() else { return Vec::new() }; - if let Some(close) = closer(c) { - self.i += 1; - return vec![self.group(Some(c), &[close])]; - } - if c == '├' { - self.i += 1; - if let Some(d) = self.explicit_group() { - return vec![d]; - } - return vec![self.group(None, &[')', ']', '}', '⟩', '⌉', '⌋', '|', '‖'])]; - } - if (c == '|' || c == '‖') && self.c.get(self.i + 1..).is_some_and(|r| r.contains(&c)) { - self.i += 1; - return vec![self.group(Some(c), &[c])]; - } - if is_prefix(c) { - self.i += 1; - return vec![self.prefix(c, stops)]; - } - if c.is_ascii_alphabetic() { - return self.word(stops); - } - self.i += 1; - vec![text(c)] - } - - /// A bracket group after its opening character; `closers` end it. - fn group(&mut self, open: Option, closers: &[char]) -> MNode { - let mut stops: Vec = closers.to_vec(); - stops.extend(['┤', '│']); - let mut elems = Vec::new(); - let mut end = None; - loop { - let e = self.expr(&stops); - elems.push(e); - match self.peek() { - Some('│') => self.i += 1, - Some('┤') => { - self.i += 1; - break; - } - Some(c) if closers.contains(&c) => { - self.i += 1; - end = Some(c); - break; - } - // Another closer: the group ends unclosed there. - _ => break, - } - if self.tired() { - break; - } - } - let beg = open.filter(|c| *c != '〖'); - let end = end.filter(|c| *c != '〗'); - let sep = if elems.len() > 1 { Some('|') } else { None }; - MNode::Delim { beg, end, sep, grow: true, shp_match: false, elems } - } - - /// `├X … Y┤`: brackets given explicitly (any bracket characters, either side optional), the - /// cursor just after `├`. `None` (cursor unchanged) when there's no matching `┤`. - fn explicit_group(&mut self) -> Option { - let mut level = 1usize; - let mut k = self.i; - while let Some(c) = self.c.get(k) { - match c { - '├' => level += 1, - '┤' => { - level -= 1; - if level == 0 { - break; - } - } - _ => {} - } - k += 1; - } - if level != 0 { - return None; - } - let bracket = |c: &char| matches!(c, '(' | ')' | '[' | ']' | '{' | '}' | '⟨' | '⟩' | '⌈' | '⌉' | '⌊' | '⌋' | '|' | '‖' | '⟦' | '⟧'); - let mut inner: Vec = self.c.get(self.i..k).unwrap_or(&[]).to_vec(); - self.i = k + 1; - let beg = if inner.first().is_some_and(bracket) { Some(inner.remove(0)) } else { None }; - // A closing bracket at the end that pairs with an opening one inside is content. - let paired = |inner: &[char]| -> bool { - let Some((last, body)) = inner.split_last() else { return false }; - let opener = match last { - ')' => '(', - ']' => '[', - '}' => '{', - '⟩' => '⟨', - '⌉' => '⌈', - '⌋' => '⌊', - '⟧' => '⟦', - _ => return false, - }; - body.iter().filter(|c| **c == opener).count() > body.iter().filter(|c| *c == last).count() - }; - let end = if !inner.is_empty() && inner.last().is_some_and(bracket) && !paired(&inner) { inner.pop() } else { None }; - let mut sub = P { c: inner, i: 0, depth: self.depth + 1, no_limits: 0, steps: self.steps }; - let mut elems = vec![Vec::new()]; - while sub.i < sub.c.len() { - if let Some(e) = elems.last_mut() { - e.extend(sub.expr(&['│'])); - } - match sub.peek() { - Some('│') => { - sub.i += 1; - elems.push(Vec::new()); - } - Some(c) => { - sub.i += 1; - if let Some(e) = elems.last_mut() { - e.push(text(c)); - } - } - None => {} - } - if sub.tired() { - break; - } - } - self.steps = sub.steps; - let sep = if elems.len() > 1 { Some('|') } else { None }; - Some(MNode::Delim { beg, end, sep, grow: true, shp_match: false, elems }) - } - - /// A prefix structure after its character. - fn prefix(&mut self, c: char, stops: &[char]) -> MNode { - match c { - '√'..='∜' => { - let mut deg = match c { - '∛' => vec![text('3')], - '∜' => vec![text('4')], - _ => Vec::new(), - }; - let mut e = self.operand(stops); - if let Some(k) = e.iter().position(|n| matches!(n, MNode::Run(r) if r.text == "&")) { - let rest = e.split_off(k + 1); - e.pop(); - deg = e; - e = rest; - } - MNode::Rad { deg_hide: deg.is_empty(), deg, e } - } - '█' | '■' => { - if self.peek() == Some('(') { - self.i += 1; - let inner = self.expr(&[')']); - if self.peek() == Some(')') { - self.i += 1; - } - grid(c == '■', inner) - } else { - text(c) - } - } - '▭' => MNode::BorderBox { hide: [false; 4], strike: [false; 4], e: self.operand(stops) }, - '□' => MNode::Boxed { e: self.operand(stops) }, - '¯' => MNode::Bar { top: true, e: self.operand(stops) }, - '▁' => MNode::Bar { top: false, e: self.operand(stops) }, - '⟡' => MNode::Phant { show: false, zero_wid: false, zero_asc: false, zero_desc: false, e: self.operand(stops) }, - // A label (`┴`/`┬`) after the operand belongs to the whole brace. - '⏞' | '⏜' | '⎴' | '⏟' | '⏝' | '⎵' => { - self.no_limits += 1; - let e = self.operand(stops); - self.no_limits -= 1; - MNode::GroupChr { chr: c, top: matches!(c, '⏞' | '⏜' | '⎴'), e } - } - _ => { - // N-ary operator: limits, then the operand. - let (mut sub, mut sup) = (Vec::new(), Vec::new()); - let (mut has_sub, mut has_sup) = (false, false); - loop { - match self.peek() { - Some('_') => { - self.i += 1; - sub = self.script_operand(stops); - has_sub = true; - } - Some('^') => { - self.i += 1; - sup = self.script_operand(stops); - has_sup = true; - } - _ => break, - } - } - if self.peek() == Some('▒') { - self.i += 1; - } - let e = if self.peek().is_some_and(|c| c == ' ') { Vec::new() } else { self.operand(stops) }; - MNode::Nary { chr: c, lim_loc: None, grow: false, sub_hide: !has_sub, sup_hide: !has_sup, sub, sup, e } - } - } - } - - /// Letters: a function application when they spell a function name, else one letter. - fn word(&mut self, stops: &[char]) -> Arg { - let start = self.i; - let mut end = start; - // Look no further than the longest name (and one past it, to see the run is longer): a - // scan to the end of every run made a long run of letters quadratic (CWE-407). - while end - start <= MAX_NAME && self.c.get(end).is_some_and(char::is_ascii_alphabetic) { - end += 1; - } - let word: String = if end - start > MAX_NAME { String::new() } else { self.c.get(start..end).unwrap_or(&[]).iter().collect() }; - if !FUNCTION_NAMES.contains(&word.as_str()) { - self.i = start + 1; - return vec![text(self.c.get(start).copied().unwrap_or(' '))]; - } - self.i = end; - let plain = vec![MNode::Run(MRun::plain(word.clone()))]; - // Limits and scripts on the name. - let name = match self.peek() { - Some('┬') | Some('┴') => { - let upper = self.peek() == Some('┴'); - self.i += 1; - let lim = self.script_operand(stops); - vec![MNode::Lim { upper, e: plain, lim }] - } - Some('_') if LIMIT_NAMES.contains(&word.as_str()) => { - self.i += 1; - let lim = self.script_operand(stops); - vec![MNode::Lim { upper: false, e: plain, lim }] - } - Some(c @ ('^' | '_')) => { - self.i += 1; - let s = self.script_operand(stops); - let mut n = attach_script(Some(MNode::Run(MRun::plain(word.clone()))), c == '^', s); - if let Some(c2 @ ('^' | '_')) = self.peek() { - self.i += 1; - let s2 = self.script_operand(stops); - n = attach_script(Some(n), c2 == '^', s2); - } - vec![n] - } - _ => plain, - }; - // The operand, after an optional space or function-application mark. - if matches!(self.peek(), Some(' ') | Some('\u{2061}') | Some('▒')) { - self.i += 1; - } - let next = self.peek(); - let applies = next.is_some_and(|c| !stops.contains(&c) && !is_closer(c) && (!is_boundary(c) || c == '(' || matches!(c, '−' | '-' | '+'))); - if !applies { - return name; - } - let e = self.operand(stops); - if e.is_empty() { - return name; - } - vec![MNode::Func { name, e }] - } - - /// Pre-scripts (`_a^b x`): the scripts, then the base. - fn prescript(&mut self, stops: &[char]) -> MNode { - let (mut sub, mut sup) = (Vec::new(), Vec::new()); - while let Some(c @ ('^' | '_')) = self.peek() { - self.i += 1; - let s = self.script_operand(stops); - if c == '^' { - sup = s; - } else { - sub = s; - } - } - if self.peek() == Some('▒') { - self.i += 1; - } - let base = self.operand(stops); - MNode::Script { kind: ScriptKind::Pre, base, sub, sup } - } - - /// An operand: a parenthesised group (parentheses dropped) or a run of atoms with their - /// postfix scripts, accents and limits, up to an operator or space. - fn operand(&mut self, stops: &[char]) -> Arg { - if self.tired() { - return self.rest(); - } - self.depth += 1; - let mut out: Arg = Vec::new(); - if self.peek() == Some('(') { - self.i += 1; - let g = self.group(Some('('), &[')']); - let inner = match g { - MNode::Delim { elems, end: Some(')'), .. } if elems.len() == 1 => elems.into_iter().next().unwrap_or_default(), - other => vec![other], - }; - out = inner; - self.postfix(&mut out, stops, true); - } else if self.peek() == Some('〖') { - self.i += 1; - let g = self.group(Some('〖'), &['〗']); - out = match g { - MNode::Delim { elems, .. } if elems.len() == 1 => elems.into_iter().next().unwrap_or_default(), - other => vec![other], - }; - self.postfix(&mut out, stops, true); - } else { - // A signed operand (`e^-x`, `−b`). - if let Some(c @ ('−' | '-' | '+' | '±' | '∓')) = self.peek() - && self.peek_at(1).is_some_and(|n| !is_boundary(n)) - { - self.i += 1; - out.push(text(c)); - } - while let Some(c) = self.peek() { - if stops.contains(&c) || is_boundary(c) || self.tired() { - break; - } - let a = self.atom(stops); - if a.is_empty() { - break; - } - out.extend(a); - self.postfix(&mut out, stops, false); - } - if out.is_empty() - && let Some(c) = self.peek() - && c != ' ' - && !stops.contains(&c) - && !is_closer(c) - && !matches!(c, '/' | '⁄' | '∕' | '¦' | '&' | '@' | '#' | '│' | '▒') - { - // A lone operator as operand. - self.i += 1; - out.push(text(c)); - } - } - self.depth -= 1; - out - } - - /// Postfix operators on the last atom of `out` (`whole`: on all of `out` as one group). - fn postfix(&mut self, out: &mut Arg, stops: &[char], whole: bool) { - while let Some(c) = self.peek() { - if self.tired() { - break; - } - let base = |out: &mut Arg| -> Option { - if whole && out.len() != 1 { - if out.is_empty() { - return None; - } - return Some(MNode::Delim { beg: None, end: None, sep: None, grow: true, shp_match: false, elems: vec![std::mem::take(out)] }); - } - out.pop() - }; - if !(matches!(c, '^' | '_') || matches!(c, '┬' | '┴') && self.no_limits == 0 || is_combining(c)) { - break; - } - // The base is already deep: the operator is text (see `too_deep`). - let deep = if whole { any_too_deep(out) } else { out.last().is_some_and(too_deep) }; - self.i += 1; - if deep { - let s = if matches!(c, '^' | '_' | '┬' | '┴') { self.script_operand(stops) } else { Vec::new() }; - out.push(text(c)); - out.extend(s); - continue; - } - match c { - '^' | '_' => { - let s = self.script_operand(stops); - let b = base(out); - out.push(attach_script(b, c == '^', s)); - } - '┬' | '┴' => { - let lim = self.script_operand(stops); - let b = base(out); - out.push(MNode::Lim { upper: c == '┴', e: b.map(|n| vec![n]).unwrap_or_default(), lim }); - } - _ => { - let b = base(out); - out.push(MNode::Acc { chr: c, e: b.map(|n| strip_parens(vec![n])).unwrap_or_default() }); - } - } - } - } - - /// A script or limit: a parenthesised group, a structure, a number, a signed operand, a run of - /// letters (and digits) or one other character. - fn script_operand(&mut self, stops: &[char]) -> Arg { - if self.tired() { - return self.rest(); - } - self.depth += 1; - let out = match self.peek() { - Some('(') => { - self.i += 1; - match self.group(Some('('), &[')']) { - MNode::Delim { elems, end: Some(')'), .. } if elems.len() == 1 => elems.into_iter().next().unwrap_or_default(), - other => vec![other], - } - } - Some('〖') => { - self.i += 1; - match self.group(Some('〖'), &['〗']) { - MNode::Delim { elems, .. } if elems.len() == 1 => elems.into_iter().next().unwrap_or_default(), - other => vec![other], - } - } - Some(c) if is_prefix(c) || closer(c).is_some() => self.atom(stops), - Some(c) if c.is_ascii_digit() => { - let start = self.i; - while self.peek().is_some_and(|k| k.is_alphanumeric() || k == '.') { - self.i += 1; - } - let t: String = self.c.get(start..self.i).unwrap_or(&[]).iter().collect(); - vec![MNode::Run(MRun::new(t))] - } - Some(c @ ('−' | '-' | '+' | '±' | '∓')) if self.peek_at(1).is_some_and(|n| !is_boundary(n)) => { - self.i += 1; - let mut v = vec![text(c)]; - v.extend(self.script_operand(stops)); - v - } - Some(c) if c.is_alphabetic() => { - // Word (#1609): the whole run of letters and digits up to a space or operator, so - // `V_dd` is V with a subscript `dd`, exactly like `V_(dd)`. - let start = self.i; - while self.peek().is_some_and(char::is_alphanumeric) { - self.i += 1; - } - let t: String = self.c.get(start..self.i).unwrap_or(&[]).iter().collect(); - // A known function name stays upright, as Word builds it. - vec![MNode::Run(if FUNCTION_NAMES.contains(&t.as_str()) { MRun::plain(t) } else { MRun::new(t) })] - } - Some(c) if !stops.contains(&c) && c != ' ' && !is_closer(c) => { - self.i += 1; - vec![text(c)] - } - _ => Vec::new(), - }; - self.depth -= 1; - out - } -} - -/// Where the trailing operand of `out` starts (after the last operator or space). -fn operand_start(out: &[MNode]) -> usize { - let mut k = out.len(); - while k > 0 { - match out.get(k - 1) { - Some(MNode::Run(r)) if r.text.chars().all(is_boundary) => break, - _ => k -= 1, - } - } - k -} - -/// A single parenthesised group used as an operand loses its parentheses. -pub fn strip_parens(mut a: Arg) -> Arg { - if a.len() == 1 - && let Some(MNode::Delim { beg: Some('('), end: Some(')'), elems, .. } | MNode::Delim { beg: None, end: None, elems, .. }) = a.first_mut() - && elems.len() == 1 - { - return elems.pop().unwrap_or_default(); - } - a -} - -fn attach_script(base: Option, sup: bool, s: Arg) -> MNode { - match base { - Some(MNode::Script { kind: ScriptKind::Sub, base, sub, .. }) if sup => MNode::Script { kind: ScriptKind::SubSup, base, sub, sup: s }, - Some(MNode::Script { kind: ScriptKind::Sup, base, sup: sp, .. }) if !sup => MNode::Script { kind: ScriptKind::SubSup, base, sub: s, sup: sp }, - b => { - let base = b.map(|b| vec![b]).unwrap_or_default(); - if sup { - MNode::Script { kind: ScriptKind::Sup, base, sub: Vec::new(), sup: s } - } else { - MNode::Script { kind: ScriptKind::Sub, base, sub: s, sup: Vec::new() } - } - } - } -} - -/// `█(a@b)` equation array / `■(a&b@c&d)` matrix from the parsed inside. -fn grid(matrix: bool, inner: Arg) -> MNode { - let mut rows: Vec> = vec![vec![Vec::new()]]; - for n in inner { - match &n { - MNode::Run(r) if r.text == "@" => rows.push(vec![Vec::new()]), - MNode::Run(r) if r.text == "&" && matrix => { - if let Some(row) = rows.last_mut() { - row.push(Vec::new()); - } - } - _ => { - if let Some(cell) = rows.last_mut().and_then(|r| r.last_mut()) { - cell.push(n); - } - } - } - } - if matrix { - MNode::Matrix { rows, col_jc: Vec::new() } - } else { - MNode::EqArr { rows: rows.into_iter().map(|r| r.into_iter().flatten().collect()).collect() } - } -} - -/// Drop placeholders and build-up spaces, merge runs; recursively. -fn clean(a: Arg) -> Arg { - let mut out = Vec::with_capacity(a.len()); - for n in a { - match n { - MNode::Run(r) if r.text == " " || r.text.chars().all(|c| c == PLACEHOLDER) => {} - other => out.push(map_args(other, &clean)), - } - } - merge_runs(out) -} - -/// Apply `f` to every argument of a node. -pub fn map_args(n: MNode, f: &dyn Fn(Arg) -> Arg) -> MNode { - match n { - MNode::Run(r) => MNode::Run(r), - MNode::Frac { kind, num, den } => MNode::Frac { kind, num: f(num), den: f(den) }, - MNode::Script { kind, base, sub, sup } => MNode::Script { kind, base: f(base), sub: f(sub), sup: f(sup) }, - MNode::Rad { deg, deg_hide, e } => MNode::Rad { deg: f(deg), deg_hide, e: f(e) }, - MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub, sup, e } => { - MNode::Nary { chr, lim_loc, grow, sub_hide, sup_hide, sub: f(sub), sup: f(sup), e: f(e) } - } - MNode::Delim { beg, end, sep, grow, shp_match, elems } => { - MNode::Delim { beg, end, sep, grow, shp_match, elems: elems.into_iter().map(f).collect() } - } - MNode::Func { name, e } => MNode::Func { name: f(name), e: f(e) }, - MNode::Lim { upper, e, lim } => MNode::Lim { upper, e: f(e), lim: f(lim) }, - MNode::Acc { chr, e } => MNode::Acc { chr, e: f(e) }, - MNode::Bar { top, e } => MNode::Bar { top, e: f(e) }, - MNode::BorderBox { hide, strike, e } => MNode::BorderBox { hide, strike, e: f(e) }, - MNode::Boxed { e } => MNode::Boxed { e: f(e) }, - MNode::GroupChr { chr, top, e } => MNode::GroupChr { chr, top, e: f(e) }, - MNode::EqArr { rows } => MNode::EqArr { rows: rows.into_iter().map(f).collect() }, - MNode::Matrix { rows, col_jc } => MNode::Matrix { rows: rows.into_iter().map(|r| r.into_iter().map(f).collect()).collect(), col_jc }, - MNode::Phant { show, zero_wid, zero_asc, zero_desc, e } => MNode::Phant { show, zero_wid, zero_asc, zero_desc, e: f(e) }, - } -} diff --git a/crates/wordmodel/src/math_symbols.rs b/crates/wordmodel/src/math_symbols.rs deleted file mode 100644 index 28b3a7f83..000000000 --- a/crates/wordmodel/src/math_symbols.rs +++ /dev/null @@ -1,390 +0,0 @@ -//! Named math characters: Math AutoCorrect while typing in an equation (`\alpha` + space → α) -//! and LaTeX conversion. Names are the usual TeX names plus the linear-format operators -//! (`\sqrt` → √, `\above` → ┴, `\matrix` → ■…). - -/// (name without the backslash, character). When a character has several names the first one -/// listed is the one LaTeX output uses. -pub const SYMBOLS: &[(&str, char)] = &[ - // Greek. - ("alpha", 'α'), - ("beta", 'β'), - ("gamma", 'γ'), - ("delta", 'δ'), - ("epsilon", 'ϵ'), - ("varepsilon", 'ε'), - ("zeta", 'ζ'), - ("eta", 'η'), - ("theta", 'θ'), - ("vartheta", 'ϑ'), - ("iota", 'ι'), - ("kappa", 'κ'), - ("varkappa", 'ϰ'), - ("lambda", 'λ'), - ("mu", 'μ'), - ("nu", 'ν'), - ("xi", 'ξ'), - ("omicron", 'ο'), - ("pi", 'π'), - ("varpi", 'ϖ'), - ("rho", 'ρ'), - ("varrho", 'ϱ'), - ("sigma", 'σ'), - ("varsigma", 'ς'), - ("tau", 'τ'), - ("upsilon", 'υ'), - ("phi", 'ϕ'), - ("varphi", 'φ'), - ("chi", 'χ'), - ("psi", 'ψ'), - ("omega", 'ω'), - ("Alpha", 'Α'), - ("Beta", 'Β'), - ("Gamma", 'Γ'), - ("Delta", 'Δ'), - ("Epsilon", 'Ε'), - ("Zeta", 'Ζ'), - ("Eta", 'Η'), - ("Theta", 'Θ'), - ("Iota", 'Ι'), - ("Kappa", 'Κ'), - ("Lambda", 'Λ'), - ("Mu", 'Μ'), - ("Nu", 'Ν'), - ("Xi", 'Ξ'), - ("Omicron", 'Ο'), - ("Pi", 'Π'), - ("Rho", 'Ρ'), - ("Sigma", 'Σ'), - ("Tau", 'Τ'), - ("Upsilon", 'Υ'), - ("Phi", 'Φ'), - ("Chi", 'Χ'), - ("Psi", 'Ψ'), - ("Omega", 'Ω'), - ("digamma", 'ϝ'), - // Binary operators. - ("pm", '±'), - ("mp", '∓'), - ("times", '×'), - ("div", '÷'), - ("cdot", '⋅'), - ("bullet", '∙'), - ("ast", '∗'), - ("star", '⋆'), - ("circ", '∘'), - ("oplus", '⊕'), - ("ominus", '⊖'), - ("otimes", '⊗'), - ("oslash", '⊘'), - ("odot", '⊙'), - ("cup", '∪'), - ("cap", '∩'), - ("uplus", '⊎'), - ("sqcup", '⊔'), - ("sqcap", '⊓'), - ("vee", '∨'), - ("wedge", '∧'), - ("setminus", '∖'), - ("backslash", '\\'), - ("wr", '≀'), - ("diamond", '⋄'), - ("dagger", '†'), - ("ddagger", '‡'), - ("amalg", '⨿'), - ("dotplus", '∔'), - ("ltimes", '⋉'), - ("rtimes", '⋊'), - // Relations. - ("le", '≤'), - ("leq", '≤'), - ("ge", '≥'), - ("geq", '≥'), - ("ne", '≠'), - ("neq", '≠'), - ("approx", '≈'), - ("equiv", '≡'), - ("sim", '∼'), - ("simeq", '≃'), - ("cong", '≅'), - ("propto", '∝'), - ("ll", '≪'), - ("gg", '≫'), - ("prec", '≺'), - ("succ", '≻'), - ("preceq", '≼'), - ("succeq", '≽'), - ("subset", '⊂'), - ("supset", '⊃'), - ("subseteq", '⊆'), - ("supseteq", '⊇'), - ("sqsubseteq", '⊑'), - ("sqsupseteq", '⊒'), - ("in", '∈'), - ("ni", '∋'), - ("notin", '∉'), - ("perp", '⊥'), - ("parallel", '∥'), - ("mid", '∣'), - ("nmid", '∤'), - ("asymp", '≍'), - ("doteq", '≐'), - ("models", '⊨'), - ("vdash", '⊢'), - ("dashv", '⊣'), - ("bowtie", '⋈'), - ("coloneq", '≔'), - ("defeq", '≝'), - ("triangleq", '≜'), - ("nless", '≮'), - ("ngtr", '≯'), - ("nleq", '≰'), - ("ngeq", '≱'), - ("nsim", '≁'), - ("ncong", '≇'), - ("nequiv", '≢'), - ("nsubset", '⊄'), - ("nsupset", '⊅'), - ("nsubseteq", '⊈'), - ("nsupseteq", '⊉'), - ("subsetneq", '⊊'), - ("supsetneq", '⊋'), - ("lesssim", '≲'), - ("gtrsim", '≳'), - ("therefore", '∴'), - ("because", '∵'), - // Arrows. - ("rightarrow", '→'), - ("to", '→'), - ("leftarrow", '←'), - ("gets", '←'), - ("leftrightarrow", '↔'), - ("uparrow", '↑'), - ("downarrow", '↓'), - ("updownarrow", '↕'), - ("Rightarrow", '⇒'), - ("Leftarrow", '⇐'), - ("Leftrightarrow", '⇔'), - ("Uparrow", '⇑'), - ("Downarrow", '⇓'), - ("Updownarrow", '⇕'), - ("longrightarrow", '⟶'), - ("longleftarrow", '⟵'), - ("longleftrightarrow", '⟷'), - ("Longrightarrow", '⟹'), - ("Longleftarrow", '⟸'), - ("Longleftrightarrow", '⟺'), - ("implies", '⟹'), - ("iff", '⟺'), - ("mapsto", '↦'), - ("longmapsto", '⟼'), - ("nearrow", '↗'), - ("searrow", '↘'), - ("swarrow", '↙'), - ("nwarrow", '↖'), - ("hookleftarrow", '↩'), - ("hookrightarrow", '↪'), - ("leftharpoonup", '↼'), - ("leftharpoondown", '↽'), - ("rightharpoonup", '⇀'), - ("rightharpoondown", '⇁'), - ("rightleftharpoons", '⇌'), - ("leftrightharpoons", '⇋'), - ("circlearrowleft", '↺'), - ("circlearrowright", '↻'), - // Big operators. - ("sum", '∑'), - ("prod", '∏'), - ("coprod", '∐'), - ("int", '∫'), - ("iint", '∬'), - ("iiint", '∭'), - ("oint", '∮'), - ("oiint", '∯'), - ("oiiint", '∰'), - ("bigcup", '⋃'), - ("bigcap", '⋂'), - ("bigvee", '⋁'), - ("bigwedge", '⋀'), - ("bigoplus", '⨁'), - ("bigotimes", '⨂'), - ("bigodot", '⨀'), - ("biguplus", '⨄'), - ("bigsqcup", '⨆'), - // Brackets. - ("langle", '⟨'), - ("rangle", '⟩'), - ("lceil", '⌈'), - ("rceil", '⌉'), - ("lfloor", '⌊'), - ("rfloor", '⌋'), - ("lbrack", '['), - ("rbrack", ']'), - ("lbrace", '{'), - ("rbrace", '}'), - ("Vert", '‖'), - ("norm", '‖'), - ("vert", '|'), - ("lbbrack", '⟦'), - ("rbbrack", '⟧'), - // Miscellaneous. - ("infty", '∞'), - ("partial", '∂'), - ("nabla", '∇'), - ("forall", '∀'), - ("exists", '∃'), - ("nexists", '∄'), - ("neg", '¬'), - ("lnot", '¬'), - ("emptyset", '∅'), - ("varnothing", '∅'), - ("hbar", 'ℏ'), - ("ell", 'ℓ'), - ("Re", 'ℜ'), - ("Im", 'ℑ'), - ("wp", '℘'), - ("aleph", 'ℵ'), - ("beth", 'ℶ'), - ("gimel", 'ℷ'), - ("daleth", 'ℸ'), - ("degree", '°'), - ("angle", '∠'), - ("measuredangle", '∡'), - ("sphericalangle", '∢'), - ("rightangle", '∟'), - ("triangle", '△'), - ("square", '□'), - ("blacksquare", '■'), - ("qed", '∎'), - ("top", '⊤'), - ("bot", '⊥'), - ("cdots", '⋯'), - ("ldots", '…'), - ("dots", '…'), - ("vdots", '⋮'), - ("ddots", '⋱'), - ("adots", '⋰'), - ("prime", '′'), - ("dprime", '″'), - ("trprime", '‴'), - ("dd", 'ⅆ'), - ("Dd", 'ⅅ'), - ("ee", 'ⅇ'), - ("ii", 'ⅈ'), - ("jj", 'ⅉ'), - ("imath", 'ı'), - ("jmath", 'ȷ'), - ("complement", '∁'), - ("Angstrom", 'Å'), - ("mho", '℧'), - ("eth", 'ð'), - ("flat", '♭'), - ("natural", '♮'), - ("sharp", '♯'), - ("clubsuit", '♣'), - ("diamondsuit", '♢'), - ("heartsuit", '♡'), - ("spadesuit", '♠'), - // Spaces. - ("quad", '\u{2003}'), - ("emsp", '\u{2003}'), - ("ensp", '\u{2002}'), - ("thicksp", '\u{2005}'), - ("medsp", '\u{205F}'), - ("thinsp", '\u{2009}'), - ("hairsp", '\u{200A}'), - ("nbsp", '\u{A0}'), - ("zwsp", '\u{200B}'), - // Linear-format operators (build-up). - ("sqrt", '√'), - ("cbrt", '∛'), - ("qdrt", '∜'), - ("matrix", '■'), - ("eqarray", '█'), - ("above", '┴'), - ("below", '┬'), - ("of", '▒'), - ("naryand", '▒'), - ("funcapply", '\u{2061}'), - ("box", '□'), - ("rect", '▭'), - ("overbar", '¯'), - ("underbar", '▁'), - ("phantom", '⟡'), - ("overbrace", '⏞'), - ("underbrace", '⏟'), - ("overparen", '⏜'), - ("underparen", '⏝'), - ("atop", '¦'), - ("over", '/'), - ("ldiv", '∕'), - ("sdiv", '⁄'), - ("begin", '〖'), - ("end", '〗'), - ("left", '├'), - ("right", '┤'), - ("eqno", '#'), - ("placeholder", '⬚'), - // Accents (combining, typed after the base). - ("hat", '\u{302}'), - ("check", '\u{30C}'), - ("tilde", '\u{303}'), - ("acute", '\u{301}'), - ("grave", '\u{300}'), - ("dot", '\u{307}'), - ("ddot", '\u{308}'), - ("dddot", '\u{20DB}'), - ("breve", '\u{306}'), - ("bar", '\u{305}'), - ("Bar", '\u{33F}'), - ("vec", '\u{20D7}'), - ("tvec", '\u{20E1}'), - ("lhvec", '\u{20D0}'), - ("hvec", '\u{20D1}'), -]; - -/// The character for a name (`alpha` → α), including `doubleR`, `scriptL`, `frakturg`. -pub fn symbol(name: &str) -> Option { - if let Some((_, c)) = SYMBOLS.iter().find(|(n, _)| *n == name) { - return Some(*c); - } - for (prefix, scr) in [("double", crate::math::MScr::DoubleStruck), ("script", crate::math::MScr::Script), ("fraktur", crate::math::MScr::Fraktur)] - { - if let Some(rest) = name.strip_prefix(prefix) { - let mut cs = rest.chars(); - if let (Some(c), None) = (cs.next(), cs.next()) - && c.is_ascii_alphanumeric() - { - let m = crate::math::math_alnum(c, false, false, scr); - if m != c { - return Some(m); - } - } - } - } - None -} - -/// The (first) name of a character. -pub fn name_of(c: char) -> Option<&'static str> { - SYMBOLS.iter().find(|(_, s)| *s == c).map(|(n, _)| *n) -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn names_resolve() { - assert_eq!(symbol("alpha"), Some('α')); - assert_eq!(symbol("doubleR"), Some('ℝ')); - assert_eq!(symbol("scriptL"), Some('ℒ')); - assert_eq!(symbol("nonesuch"), None); - assert_eq!(name_of('≤'), Some("le")); - // No name is listed twice. - let mut names: Vec<&str> = SYMBOLS.iter().map(|(n, _)| *n).collect(); - names.sort_unstable(); - let before = names.len(); - names.dedup(); - assert_eq!(before, names.len()); - } -} diff --git a/crates/wordmodel/src/math_tree.rs b/crates/wordmodel/src/math_tree.rs deleted file mode 100644 index 45d2c603a..000000000 --- a/crates/wordmodel/src/math_tree.rs +++ /dev/null @@ -1,104 +0,0 @@ -//! Structural access to an equation tree: the arguments of a node, and every run in a tree. - -use crate::math::{Arg, MAX_DEPTH, MNode, MRun}; - -/// Number of child arguments of a node. -pub fn child_count(n: &MNode) -> usize { - match n { - MNode::Run(_) => 0, - MNode::Frac { .. } | MNode::Func { .. } | MNode::Lim { .. } => 2, - MNode::Rad { .. } => 2, - MNode::Script { .. } | MNode::Nary { .. } => 3, - MNode::Delim { elems, .. } => elems.len(), - MNode::EqArr { rows } => rows.len(), - MNode::Matrix { rows, .. } => rows.iter().map(|r| r.len()).sum(), - MNode::Acc { .. } | MNode::Bar { .. } | MNode::BorderBox { .. } | MNode::Boxed { .. } | MNode::GroupChr { .. } | MNode::Phant { .. } => 1, - } -} - -/// Child argument `i` of a node (see [`child_count`] for the numbering: fraction num 0 / den 1; -/// script base 0 / sub 1 / sup 2; radical degree 0 / base 1; n-ary sub 0 / sup 1 / operand 2; -/// function name 0 / argument 1; limit base 0 / limit 1; matrices row by row). -pub fn child(n: &MNode, i: usize) -> Option<&Arg> { - match (n, i) { - (MNode::Frac { num, .. }, 0) => Some(num), - (MNode::Frac { den, .. }, 1) => Some(den), - (MNode::Script { base, .. }, 0) => Some(base), - (MNode::Script { sub, .. }, 1) => Some(sub), - (MNode::Script { sup, .. }, 2) => Some(sup), - (MNode::Rad { deg, .. }, 0) => Some(deg), - (MNode::Rad { e, .. }, 1) => Some(e), - (MNode::Nary { sub, .. }, 0) => Some(sub), - (MNode::Nary { sup, .. }, 1) => Some(sup), - (MNode::Nary { e, .. }, 2) => Some(e), - (MNode::Delim { elems, .. }, i) => elems.get(i), - (MNode::Func { name, .. }, 0) => Some(name), - (MNode::Func { e, .. }, 1) => Some(e), - (MNode::Lim { e, .. }, 0) => Some(e), - (MNode::Lim { lim, .. }, 1) => Some(lim), - ( - MNode::Acc { e, .. } - | MNode::Bar { e, .. } - | MNode::BorderBox { e, .. } - | MNode::Boxed { e } - | MNode::GroupChr { e, .. } - | MNode::Phant { e, .. }, - 0, - ) => Some(e), - (MNode::EqArr { rows }, i) => rows.get(i), - (MNode::Matrix { rows, .. }, i) => rows.iter().flatten().nth(i), - _ => None, - } -} - -pub fn child_mut(n: &mut MNode, i: usize) -> Option<&mut Arg> { - match (n, i) { - (MNode::Frac { num, .. }, 0) => Some(num), - (MNode::Frac { den, .. }, 1) => Some(den), - (MNode::Script { base, .. }, 0) => Some(base), - (MNode::Script { sub, .. }, 1) => Some(sub), - (MNode::Script { sup, .. }, 2) => Some(sup), - (MNode::Rad { deg, .. }, 0) => Some(deg), - (MNode::Rad { e, .. }, 1) => Some(e), - (MNode::Nary { sub, .. }, 0) => Some(sub), - (MNode::Nary { sup, .. }, 1) => Some(sup), - (MNode::Nary { e, .. }, 2) => Some(e), - (MNode::Delim { elems, .. }, i) => elems.get_mut(i), - (MNode::Func { name, .. }, 0) => Some(name), - (MNode::Func { e, .. }, 1) => Some(e), - (MNode::Lim { e, .. }, 0) => Some(e), - (MNode::Lim { lim, .. }, 1) => Some(lim), - ( - MNode::Acc { e, .. } - | MNode::Bar { e, .. } - | MNode::BorderBox { e, .. } - | MNode::Boxed { e } - | MNode::GroupChr { e, .. } - | MNode::Phant { e, .. }, - 0, - ) => Some(e), - (MNode::EqArr { rows }, i) => rows.get_mut(i), - (MNode::Matrix { rows, .. }, i) => rows.iter_mut().flatten().nth(i), - _ => None, - } -} - -/// Call `f` on every run in the tree. -pub fn each_run_mut(root: &mut Arg, f: &mut dyn FnMut(&mut MRun)) { - fn go(a: &mut Arg, f: &mut dyn FnMut(&mut MRun), depth: usize) { - if depth > MAX_DEPTH { - return; - } - for n in a.iter_mut() { - if let MNode::Run(r) = n { - f(r); - } - for c in 0..child_count(n) { - if let Some(ch) = child_mut(n, c) { - go(ch, f, depth + 1); - } - } - } - } - go(root, f, 0); -} diff --git a/crates/wordmodel/src/props/color.rs b/crates/wordmodel/src/props/color.rs index baf2d948d..9fce90039 100644 --- a/crates/wordmodel/src/props/color.rs +++ b/crates/wordmodel/src/props/color.rs @@ -1,83 +1,7 @@ use super::{Deserialize, Serialize}; -/// An sRGB colour. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Default)] -pub struct Rgb(pub u8, pub u8, pub u8); - -impl<'de> Deserialize<'de> for Rgb { - /// Reads what it writes (`[r, g, b]`) and also a string [`Rgb::parse`] reads, so the colours in - /// `CharProps` / `ParaProps` take the `RRGGBB` that `para.borders` and `table.shading` take (#865). - fn deserialize>(d: D) -> Result { - use serde::de::{Error, SeqAccess, Unexpected, Visitor}; - struct V; - impl<'de> Visitor<'de> for V { - type Value = Rgb; - fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { - f.write_str("`[r, g, b]`, `RRGGBB` or `#RRGGBB`") - } - fn visit_str(self, s: &str) -> Result { - Rgb::parse(s).ok_or_else(|| E::invalid_value(Unexpected::Str(s), &self)) - } - fn visit_seq>(self, mut a: A) -> Result { - let mut c = [0u8; 3]; - for (i, x) in c.iter_mut().enumerate() { - *x = a.next_element()?.ok_or_else(|| A::Error::invalid_length(i, &self))?; - } - if a.next_element::()?.is_some() { - return Err(A::Error::invalid_length(4, &self)); - } - Ok(Rgb(c[0], c[1], c[2])) - } - } - d.deserialize_any(V) - } -} - -impl Rgb { - pub const BLACK: Rgb = Rgb(0, 0, 0); - pub const WHITE: Rgb = Rgb(255, 255, 255); - /// `RRGGBB` (no `#`). - pub fn hex(self) -> String { - format!("{:02X}{:02X}{:02X}", self.0, self.1, self.2) - } - /// Parse `RRGGBB` or `#RRGGBB`. - pub fn parse(s: &str) -> Option { - let s = s.trim(); - let s = s.strip_prefix('#').unwrap_or(s); - if s.len() != 6 || !s.bytes().all(|b| b.is_ascii_hexdigit()) { - return None; - } - let p = |i: usize| s.get(i..i + 2).and_then(|h| u8::from_str_radix(h, 16).ok()); - Some(Rgb(p(0)?, p(2)?, p(4)?)) - } - /// This colour with the DrawingML colour modifier `name` (`lumMod`, `lumOff`, `tint`, - /// `shade`, `satMod`, ECMA-376 §20.1.2.3) at `v` (1.0 = 100%); other names leave it - /// unchanged. - pub fn modified(self, name: &str, v: f64) -> Rgb { - match name { - "lumMod" => adjust_lum(self, |l| l * v), - "lumOff" => adjust_lum(self, |l| l + v), - "tint" => mix(self, 255.0, v), - "shade" => mix(self, 0.0, v), - "satMod" => { - let (h, sat, l) = to_hsl(self); - from_hsl(h, (sat * v).clamp(0.0, 1.0), l) - } - _ => self, - } - } - /// The colour with hue `h` (0..1 of a turn), saturation `s` and lightness `l` (0..1). - pub fn from_hsl(h: f64, s: f64, l: f64) -> Rgb { - from_hsl(h, s, l) - } - /// This colour with its HSL luminance changed by `f`. - pub fn with_lum(self, f: impl Fn(f64) -> f64) -> Rgb { - adjust_lum(self, f) - } - pub fn luma(self) -> f32 { - 0.299 * self.0 as f32 + 0.587 * self.1 as f32 + 0.114 * self.2 as f32 - } -} +/// An sRGB colour: craft-math's, which equations and text share. +pub use craft_math::Rgb; /// A text colour: automatic (black on light backgrounds, white on dark) or a fixed colour. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Default)] @@ -115,57 +39,3 @@ impl<'de> Deserialize<'de> for TextColor { } } } - -/// `tint` (towards `to` = 255) and `shade` (towards 0): `v` is the share of the original colour kept. -fn mix(c: Rgb, to: f64, v: f64) -> Rgb { - let f = |x: u8| (x as f64 * v + to * (1.0 - v)).round().clamp(0.0, 255.0) as u8; - Rgb(f(c.0), f(c.1), f(c.2)) -} - -/// `c` with its HSL luminance changed by `f`. -fn adjust_lum(c: Rgb, f: impl Fn(f64) -> f64) -> Rgb { - let (h, s, l) = to_hsl(c); - from_hsl(h, s, f(l).clamp(0.0, 1.0)) -} - -fn to_hsl(c: Rgb) -> (f64, f64, f64) { - let (r, g, b) = (c.0 as f64 / 255.0, c.1 as f64 / 255.0, c.2 as f64 / 255.0); - let (max, min) = (r.max(g).max(b), r.min(g).min(b)); - let l = (max + min) / 2.0; - if max == min { - return (0.0, 0.0, l); - } - let d = max - min; - let s = if l > 0.5 { d / (2.0 - max - min) } else { d / (max + min) }; - let h = if max == r { - (g - b) / d + if g < b { 6.0 } else { 0.0 } - } else if max == g { - (b - r) / d + 2.0 - } else { - (r - g) / d + 4.0 - }; - (h / 6.0, s, l) -} - -fn from_hsl(h: f64, s: f64, l: f64) -> Rgb { - let byte = |v: f64| (v * 255.0).round().clamp(0.0, 255.0) as u8; - if s == 0.0 { - return Rgb(byte(l), byte(l), byte(l)); - } - let q = if l < 0.5 { l * (1.0 + s) } else { l + s - l * s }; - let p = 2.0 * l - q; - let channel = |t: f64| { - let t = t.rem_euclid(1.0); - let v = if t < 1.0 / 6.0 { - p + (q - p) * 6.0 * t - } else if t < 0.5 { - q - } else if t < 2.0 / 3.0 { - p + (q - p) * (2.0 / 3.0 - t) * 6.0 - } else { - p - }; - byte(v) - }; - Rgb(channel(h + 1.0 / 3.0), channel(h), channel(h - 1.0 / 3.0)) -} diff --git a/crates/wordmodel/src/symbol_fonts.rs b/crates/wordmodel/src/symbol_fonts.rs deleted file mode 100644 index 1d5175540..000000000 --- a/crates/wordmodel/src/symbol_fonts.rs +++ /dev/null @@ -1,316 +0,0 @@ -//! Word's symbol fonts — Symbol, Wingdings, Wingdings 2, Wingdings 3 and Webdings — mapped to -//! standard Unicode characters, so text set in them shows the intended symbols when the font -//! isn't installed (they are proprietary and never bundled) and exports as real characters. -//! -//! Word stores a symbol-font character either as the private-use code U+F000 + the font's own -//! 8-bit code (U+F020–U+F0FF, what Insert › Symbol and `w:sym` write) or as the plain 8-bit code -//! (U+0020–U+00FF) in a run whose font is the symbol font. Both map the same way. The stored text -//! never changes: the mapping is for display, layout and plain-text output only. -//! -//! Sources (Unicode's own published data; no font file was read): -//! - Symbol: the Adobe Symbol encoding table published by Unicode, -//! (codes it maps to the -//! private use area are left out; 0x6D maps to GREEK SMALL LETTER MU). -//! - Wingdings, Wingdings 2, Wingdings 3, Webdings: the correspondence tables in "Status of -//! encoding of Wingdings and Webdings Symbols", Unicode L2/12-368 (WG2 N4384, M. Suignard, -//! 2012), the frozen repertoire of the Unicode 7.0 encoding (earlier drafts L2/11-052, -//! L2/11-344), checked against the Unicode Character Database names. The one character not -//! encoded (Wingdings 0xFF) has no entry. -//! -//! Tables are indexed by code − 0x20; 0 = no Unicode equivalent. - -/// The symbol table for a single family name, by lowercase family name. -fn single_table(family: &str) -> Option<&'static [u32; 224]> { - let f = family.trim(); - if f.eq_ignore_ascii_case("symbol") || f.eq_ignore_ascii_case("symbol (pcl6)") { - Some(&SYMBOL) - } else if f.eq_ignore_ascii_case("wingdings") { - Some(&WINGDINGS) - } else if f.eq_ignore_ascii_case("wingdings 2") { - Some(&WINGDINGS_2) - } else if f.eq_ignore_ascii_case("wingdings 3") { - Some(&WINGDINGS_3) - } else if f.eq_ignore_ascii_case("webdings") { - Some(&WEBDINGS) - } else { - None - } -} - -/// The symbol fonts this module knows, by lowercase family name. -fn table(family: &str) -> Option<&'static [u32; 224]> { - if family.contains(';') { family.split(';').map(str::trim).filter(|s| !s.is_empty()).find_map(single_table) } else { single_table(family) } -} - -/// Whether `family` is one of Word's symbol fonts (Symbol, Wingdings, Wingdings 2, Wingdings 3, -/// Webdings), whose characters [`symbol_char`] maps. -pub fn is_symbol_font(family: &str) -> bool { - table(family).is_some() -} - -/// The standard Unicode character for `c` in a run set in the symbol font `family`: `c` is the -/// font's code stored as U+F020–U+F0FF or as U+0020–U+00FF (except the soft hyphen U+00AD). -/// `None` for other fonts, other characters and codes with no Unicode equivalent. -pub fn symbol_char(family: &str, c: char) -> Option { - let t = table(family)?; - let u = u32::from(c); - let code = match u { - 0xF020..=0xF0FF => u - 0xF000, - // A soft hyphen stays a soft hyphen (Word stores `w:softHyphen` as U+00AD). - 0xAD => return None, - 0x20..=0xFF => u, - _ => return None, - }; - let i = usize::try_from(code.checked_sub(0x20)?).ok()?; - t.get(i).copied().filter(|v| *v != 0).and_then(char::from_u32) -} - -/// `text` (in a run set in `family`) with every symbol-font character replaced by its Unicode -/// equivalent; unchanged for other fonts. -pub fn map_text(family: &str, text: &str) -> String { - if !is_symbol_font(family) { - return text.to_string(); - } - text.chars().map(|c| symbol_char(family, c).unwrap_or(c)).collect() -} - -const SYMBOL: [u32; 224] = [ - 0x0020, 0x0021, 0x2200, 0x0023, 0x2203, 0x0025, 0x0026, 0x220B, // 0x20 - 0x0028, 0x0029, 0x2217, 0x002B, 0x002C, 0x2212, 0x002E, 0x002F, // 0x28 - 0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037, // 0x30 - 0x0038, 0x0039, 0x003A, 0x003B, 0x003C, 0x003D, 0x003E, 0x003F, // 0x38 - 0x2245, 0x0391, 0x0392, 0x03A7, 0x0394, 0x0395, 0x03A6, 0x0393, // 0x40 - 0x0397, 0x0399, 0x03D1, 0x039A, 0x039B, 0x039C, 0x039D, 0x039F, // 0x48 - 0x03A0, 0x0398, 0x03A1, 0x03A3, 0x03A4, 0x03A5, 0x03C2, 0x03A9, // 0x50 - 0x039E, 0x03A8, 0x0396, 0x005B, 0x2234, 0x005D, 0x22A5, 0x005F, // 0x58 - 0x0000, 0x03B1, 0x03B2, 0x03C7, 0x03B4, 0x03B5, 0x03C6, 0x03B3, // 0x60 - 0x03B7, 0x03B9, 0x03D5, 0x03BA, 0x03BB, 0x03BC, 0x03BD, 0x03BF, // 0x68 - 0x03C0, 0x03B8, 0x03C1, 0x03C3, 0x03C4, 0x03C5, 0x03D6, 0x03C9, // 0x70 - 0x03BE, 0x03C8, 0x03B6, 0x007B, 0x007C, 0x007D, 0x223C, 0x0000, // 0x78 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0x80 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0x88 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0x90 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0x98 - 0x20AC, 0x03D2, 0x2032, 0x2264, 0x2044, 0x221E, 0x0192, 0x2663, // 0xA0 - 0x2666, 0x2665, 0x2660, 0x2194, 0x2190, 0x2191, 0x2192, 0x2193, // 0xA8 - 0x00B0, 0x00B1, 0x2033, 0x2265, 0x00D7, 0x221D, 0x2202, 0x2022, // 0xB0 - 0x00F7, 0x2260, 0x2261, 0x2248, 0x2026, 0x0000, 0x0000, 0x21B5, // 0xB8 - 0x2135, 0x2111, 0x211C, 0x2118, 0x2297, 0x2295, 0x2205, 0x2229, // 0xC0 - 0x222A, 0x2283, 0x2287, 0x2284, 0x2282, 0x2286, 0x2208, 0x2209, // 0xC8 - 0x2220, 0x2207, 0x0000, 0x0000, 0x0000, 0x220F, 0x221A, 0x22C5, // 0xD0 - 0x00AC, 0x2227, 0x2228, 0x21D4, 0x21D0, 0x21D1, 0x21D2, 0x21D3, // 0xD8 - 0x25CA, 0x2329, 0x0000, 0x0000, 0x0000, 0x2211, 0x0000, 0x0000, // 0xE0 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0xE8 - 0x0000, 0x232A, 0x222B, 0x2320, 0x0000, 0x2321, 0x0000, 0x0000, // 0xF0 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0xF8 -]; - -const WEBDINGS: [u32; 224] = [ - 0x0020, 0x1F577, 0x1F578, 0x1F572, 0x1F576, 0x1F3C6, 0x1F396, 0x1F587, // 0x20 - 0x1F5E8, 0x1F5E9, 0x1F5F0, 0x1F5F1, 0x1F336, 0x1F397, 0x1F67E, 0x1F67C, // 0x28 - 0x1F5D5, 0x1F5D6, 0x1F5D7, 0x23F4, 0x23F5, 0x23F6, 0x23F7, 0x23EA, // 0x30 - 0x23E9, 0x23EE, 0x23ED, 0x23F8, 0x23F9, 0x23FA, 0x1F5DA, 0x1F5F3, // 0x38 - 0x1F6E0, 0x1F3D7, 0x1F3D8, 0x1F3D9, 0x1F3DA, 0x1F3DC, 0x1F3ED, 0x1F3DB, // 0x40 - 0x1F3E0, 0x1F3D6, 0x1F3DD, 0x1F6E3, 0x1F50D, 0x1F3D4, 0x1F441, 0x1F442, // 0x48 - 0x1F3DE, 0x1F3D5, 0x1F6E4, 0x1F3DF, 0x1F6F3, 0x1F56C, 0x1F56B, 0x1F568, // 0x50 - 0x1F508, 0x1F394, 0x1F395, 0x1F5EC, 0x1F67D, 0x1F5ED, 0x1F5EA, 0x1F5EB, // 0x58 - 0x2B94, 0x2714, 0x1F6B2, 0x2B1C, 0x1F6E1, 0x1F4E6, 0x1F6F1, 0x2B1B, // 0x60 - 0x1F691, 0x1F6C8, 0x1F6E9, 0x1F6F0, 0x1F7C8, 0x1F574, 0x2B24, 0x1F6E5, // 0x68 - 0x1F694, 0x1F5D8, 0x1F5D9, 0x2753, 0x1F6F2, 0x1F687, 0x1F68D, 0x26F3, // 0x70 - 0x29B8, 0x2296, 0x1F6AD, 0x1F5EE, 0x23D0, 0x1F5EF, 0x1F5F2, 0x0000, // 0x78 - 0x1F6B9, 0x1F6BA, 0x1F6C9, 0x1F6CA, 0x1F6BC, 0x1F47D, 0x1F3CB, 0x26F7, // 0x80 - 0x1F3C2, 0x1F3CC, 0x1F3CA, 0x1F3C4, 0x1F3CD, 0x1F3CE, 0x1F698, 0x1F5E0, // 0x88 - 0x1F6E2, 0x1F4B0, 0x1F3F7, 0x1F4B3, 0x1F46A, 0x1F5E1, 0x1F5E2, 0x1F5E3, // 0x90 - 0x272F, 0x1F584, 0x1F585, 0x1F583, 0x1F586, 0x1F5B9, 0x1F5BA, 0x1F5BB, // 0x98 - 0x1F575, 0x1F570, 0x1F5BD, 0x1F5BE, 0x1F4CB, 0x1F5D2, 0x1F5D3, 0x1F56E, // 0xA0 - 0x1F4DA, 0x1F5DE, 0x1F5DF, 0x1F5C3, 0x1F5C2, 0x1F5BC, 0x1F3AD, 0x1F39C, // 0xA8 - 0x1F398, 0x1F399, 0x1F3A7, 0x1F4BF, 0x1F39E, 0x1F4F7, 0x1F39F, 0x1F3AC, // 0xB0 - 0x1F4FD, 0x1F4F9, 0x1F4FE, 0x1F4FB, 0x1F39A, 0x1F39B, 0x1F4FA, 0x1F4BB, // 0xB8 - 0x1F5A5, 0x1F5A6, 0x1F5A7, 0x1F579, 0x1F3AE, 0x1F57B, 0x1F57C, 0x1F4DF, // 0xC0 - 0x1F581, 0x1F580, 0x1F5A8, 0x1F5A9, 0x1F5BF, 0x1F5AA, 0x1F5DC, 0x1F512, // 0xC8 - 0x1F513, 0x1F5DD, 0x1F4E5, 0x1F4E4, 0x1F573, 0x1F323, 0x1F324, 0x1F325, // 0xD0 - 0x1F326, 0x2601, 0x1F328, 0x1F327, 0x1F329, 0x1F32A, 0x1F32C, 0x1F32B, // 0xD8 - 0x1F31C, 0x1F321, 0x1F6CB, 0x1F6CF, 0x1F37D, 0x1F378, 0x1F6CE, 0x1F6CD, // 0xE0 - 0x24C5, 0x267F, 0x1F6C6, 0x1F588, 0x1F393, 0x1F5E4, 0x1F5E5, 0x1F5E6, // 0xE8 - 0x1F5E7, 0x1F6EA, 0x1F43F, 0x1F426, 0x1F41F, 0x1F415, 0x1F408, 0x1F66C, // 0xF0 - 0x1F66E, 0x1F66D, 0x1F66F, 0x1F5FA, 0x1F30D, 0x1F30F, 0x1F30E, 0x1F54A, // 0xF8 -]; - -const WINGDINGS: [u32; 224] = [ - 0x0020, 0x1F589, 0x2702, 0x2701, 0x1F453, 0x1F56D, 0x1F56E, 0x1F56F, // 0x20 - 0x1F57F, 0x2706, 0x1F582, 0x1F583, 0x1F4EA, 0x1F4EB, 0x1F4EC, 0x1F4ED, // 0x28 - 0x1F5C0, 0x1F5C1, 0x1F5CE, 0x1F5CF, 0x1F5D0, 0x1F5C4, 0x23F3, 0x1F5AE, // 0x30 - 0x1F5B0, 0x1F5B2, 0x1F5B3, 0x1F5B4, 0x1F5AB, 0x1F5AC, 0x2707, 0x270D, // 0x38 - 0x1F58E, 0x270C, 0x1F58F, 0x1F44D, 0x1F44E, 0x261C, 0x261E, 0x261D, // 0x40 - 0x1F597, 0x1F590, 0x263A, 0x1F610, 0x2639, 0x1F4A3, 0x1F571, 0x1F3F3, // 0x48 - 0x1F3F1, 0x2708, 0x263C, 0x1F322, 0x2744, 0x1F546, 0x271E, 0x1F548, // 0x50 - 0x2720, 0x2721, 0x262A, 0x262F, 0x1F549, 0x2638, 0x2648, 0x2649, // 0x58 - 0x264A, 0x264B, 0x264C, 0x264D, 0x264E, 0x264F, 0x2650, 0x2651, // 0x60 - 0x2652, 0x2653, 0x1F670, 0x1F675, 0x26AB, 0x1F53E, 0x25FC, 0x1F78F, // 0x68 - 0x1F790, 0x2751, 0x2752, 0x1F79F, 0x29EB, 0x25C6, 0x2756, 0x1F799, // 0x70 - 0x2327, 0x2BB9, 0x2318, 0x1F3F5, 0x1F3F6, 0x1F676, 0x1F677, 0x0000, // 0x78 - 0x1F10B, 0x2780, 0x2781, 0x2782, 0x2783, 0x2784, 0x2785, 0x2786, // 0x80 - 0x2787, 0x2788, 0x2789, 0x1F10C, 0x278A, 0x278B, 0x278C, 0x278D, // 0x88 - 0x278E, 0x278F, 0x2790, 0x2791, 0x2792, 0x2793, 0x1F662, 0x1F660, // 0x90 - 0x1F661, 0x1F663, 0x1F65E, 0x1F65C, 0x1F65D, 0x1F65F, 0x2219, 0x2022, // 0x98 - 0x2B1D, 0x2B58, 0x1F786, 0x1F788, 0x1F78A, 0x1F78B, 0x1F53F, 0x25AA, // 0xA0 - 0x1F78E, 0x1F7C1, 0x1F7C5, 0x2605, 0x1F7CB, 0x1F7CF, 0x1F7D3, 0x1F7D1, // 0xA8 - 0x2BD0, 0x2316, 0x2BCE, 0x2BCF, 0x2BD1, 0x272A, 0x2730, 0x1F550, // 0xB0 - 0x1F551, 0x1F552, 0x1F553, 0x1F554, 0x1F555, 0x1F556, 0x1F557, 0x1F558, // 0xB8 - 0x1F559, 0x1F55A, 0x1F55B, 0x2BB0, 0x2BB1, 0x2BB2, 0x2BB3, 0x2BB4, // 0xC0 - 0x2BB5, 0x2BB6, 0x2BB7, 0x1F66A, 0x1F66B, 0x1F655, 0x1F654, 0x1F657, // 0xC8 - 0x1F656, 0x1F650, 0x1F651, 0x1F652, 0x1F653, 0x232B, 0x2326, 0x2B98, // 0xD0 - 0x2B9A, 0x2B99, 0x2B9B, 0x2B88, 0x2B8A, 0x2B89, 0x2B8B, 0x1F868, // 0xD8 - 0x1F86A, 0x1F869, 0x1F86B, 0x1F86C, 0x1F86D, 0x1F86F, 0x1F86E, 0x1F878, // 0xE0 - 0x1F87A, 0x1F879, 0x1F87B, 0x1F87C, 0x1F87D, 0x1F87F, 0x1F87E, 0x21E6, // 0xE8 - 0x21E8, 0x21E7, 0x21E9, 0x2B04, 0x21F3, 0x2B01, 0x2B00, 0x2B03, // 0xF0 - 0x2B02, 0x1F8AC, 0x1F8AD, 0x1F5F6, 0x2713, 0x1F5F7, 0x1F5F9, 0x0000, // 0xF8 -]; - -const WINGDINGS_2: [u32; 224] = [ - 0x0020, 0x1F58A, 0x1F58B, 0x1F58C, 0x1F58D, 0x2704, 0x2700, 0x1F57E, // 0x20 - 0x1F57D, 0x1F5C5, 0x1F5C6, 0x1F5C7, 0x1F5C8, 0x1F5C9, 0x1F5CA, 0x1F5CB, // 0x28 - 0x1F5CC, 0x1F5CD, 0x1F4CB, 0x1F5D1, 0x1F5D4, 0x1F5B5, 0x1F5B6, 0x1F5B7, // 0x30 - 0x1F5B8, 0x1F5AD, 0x1F5AF, 0x1F5B1, 0x1F592, 0x1F593, 0x1F598, 0x1F599, // 0x38 - 0x1F59A, 0x1F59B, 0x1F448, 0x1F449, 0x1F59C, 0x1F59D, 0x1F59E, 0x1F59F, // 0x40 - 0x1F5A0, 0x1F5A1, 0x1F446, 0x1F447, 0x1F5A2, 0x1F5A3, 0x1F591, 0x1F5F4, // 0x48 - 0x1F5F8, 0x1F5F5, 0x2611, 0x2BBD, 0x2612, 0x2BBE, 0x2BBF, 0x1F6C7, // 0x50 - 0x29B8, 0x1F671, 0x1F674, 0x1F672, 0x1F673, 0x203D, 0x1F679, 0x1F67A, // 0x58 - 0x1F67B, 0x1F666, 0x1F664, 0x1F665, 0x1F667, 0x1F65A, 0x1F658, 0x1F659, // 0x60 - 0x1F65B, 0x24EA, 0x2460, 0x2461, 0x2462, 0x2463, 0x2464, 0x2465, // 0x68 - 0x2466, 0x2467, 0x2468, 0x2469, 0x24FF, 0x2776, 0x2777, 0x2778, // 0x70 - 0x2779, 0x277A, 0x277B, 0x277C, 0x277D, 0x277E, 0x277F, 0x0000, // 0x78 - 0x2609, 0x1F315, 0x263D, 0x263E, 0x2E3F, 0x271D, 0x1F547, 0x1F55C, // 0x80 - 0x1F55D, 0x1F55E, 0x1F55F, 0x1F560, 0x1F561, 0x1F562, 0x1F563, 0x1F564, // 0x88 - 0x1F565, 0x1F566, 0x1F567, 0x1F668, 0x1F669, 0x22C5, 0x1F784, 0x2981, // 0x90 - 0x25CF, 0x25CB, 0x1F785, 0x1F787, 0x1F789, 0x2299, 0x29BF, 0x1F78C, // 0x98 - 0x1F78D, 0x25FE, 0x25A0, 0x25A1, 0x1F791, 0x1F792, 0x1F793, 0x1F794, // 0xA0 - 0x25A3, 0x1F795, 0x1F796, 0x1F797, 0x1F798, 0x2B29, 0x2B25, 0x25C7, // 0xA8 - 0x1F79A, 0x25C8, 0x1F79B, 0x1F79C, 0x1F79D, 0x1F79E, 0x2B2A, 0x2B27, // 0xB0 - 0x25CA, 0x1F7A0, 0x25D6, 0x25D7, 0x2BCA, 0x2BCB, 0x2BC0, 0x2BC1, // 0xB8 - 0x2B1F, 0x2BC2, 0x2B23, 0x2B22, 0x2BC3, 0x2BC4, 0x1F7A1, 0x1F7A2, // 0xC0 - 0x1F7A3, 0x1F7A4, 0x1F7A5, 0x1F7A6, 0x1F7A7, 0x1F7A8, 0x1F7A9, 0x1F7AA, // 0xC8 - 0x1F7AB, 0x1F7AC, 0x1F7AD, 0x1F7AE, 0x1F7AF, 0x1F7B0, 0x1F7B1, 0x1F7B2, // 0xD0 - 0x1F7B3, 0x1F7B4, 0x1F7B5, 0x1F7B6, 0x1F7B7, 0x1F7B8, 0x1F7B9, 0x1F7BA, // 0xD8 - 0x1F7BB, 0x1F7BC, 0x1F7BD, 0x1F7BE, 0x1F7BF, 0x1F7C0, 0x1F7C2, 0x1F7C4, // 0xE0 - 0x1F7C6, 0x1F7C9, 0x1F7CA, 0x2736, 0x1F7CC, 0x1F7CE, 0x1F7D0, 0x1F7D2, // 0xE8 - 0x2739, 0x1F7C3, 0x1F7C7, 0x272F, 0x1F7CD, 0x1F7D4, 0x2BCC, 0x2BCD, // 0xF0 - 0x203B, 0x2042, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0xF8 -]; - -const WINGDINGS_3: [u32; 224] = [ - 0x0020, 0x2B60, 0x2B62, 0x2B61, 0x2B63, 0x2B66, 0x2B67, 0x2B69, // 0x20 - 0x2B68, 0x2B70, 0x2B72, 0x2B71, 0x2B73, 0x2B76, 0x2B78, 0x2B7B, // 0x28 - 0x2B7D, 0x2B64, 0x2B65, 0x2B6A, 0x2B6C, 0x2B6B, 0x2B6D, 0x2B4D, // 0x30 - 0x2BA0, 0x2BA1, 0x2BA2, 0x2BA3, 0x2BA4, 0x2BA5, 0x2BA6, 0x2BA7, // 0x38 - 0x2B90, 0x2B91, 0x2B92, 0x2B93, 0x2B80, 0x2B83, 0x2B7E, 0x2B7F, // 0x40 - 0x2B84, 0x2B86, 0x2B85, 0x2B87, 0x2B8F, 0x2B8D, 0x2B8E, 0x2B8C, // 0x48 - 0x2B6E, 0x2B6F, 0x238B, 0x2324, 0x2303, 0x2325, 0x2423, 0x237D, // 0x50 - 0x21EA, 0x2BB8, 0x1F8A0, 0x1F8A1, 0x1F8A2, 0x1F8A3, 0x1F8A4, 0x1F8A5, // 0x58 - 0x1F8A6, 0x1F8A7, 0x1F8A8, 0x1F8A9, 0x1F8AA, 0x1F8AB, 0x1F850, 0x1F852, // 0x60 - 0x1F851, 0x1F853, 0x1F854, 0x1F855, 0x1F857, 0x1F856, 0x1F858, 0x1F859, // 0x68 - 0x25B2, 0x25BC, 0x25B3, 0x25BD, 0x25C0, 0x25B6, 0x25C1, 0x25B7, // 0x70 - 0x25E3, 0x25E2, 0x25E4, 0x25E5, 0x1F780, 0x1F782, 0x1F781, 0x0000, // 0x78 - 0x1F783, 0x2BC5, 0x2BC6, 0x2BC7, 0x2BC8, 0x2B9C, 0x2B9E, 0x2B9D, // 0x80 - 0x2B9F, 0x1F810, 0x1F812, 0x1F811, 0x1F813, 0x1F814, 0x1F816, 0x1F815, // 0x88 - 0x1F817, 0x1F818, 0x1F81A, 0x1F819, 0x1F81B, 0x1F81C, 0x1F81E, 0x1F81D, // 0x90 - 0x1F81F, 0x1F800, 0x1F802, 0x1F801, 0x1F803, 0x1F804, 0x1F806, 0x1F805, // 0x98 - 0x1F807, 0x1F808, 0x1F80A, 0x1F809, 0x1F80B, 0x1F820, 0x1F822, 0x1F824, // 0xA0 - 0x1F826, 0x1F828, 0x1F82A, 0x1F82C, 0x1F89C, 0x1F89D, 0x1F89E, 0x1F89F, // 0xA8 - 0x1F82E, 0x1F830, 0x1F832, 0x1F834, 0x1F836, 0x1F838, 0x1F83A, 0x1F839, // 0xB0 - 0x1F83B, 0x1F898, 0x1F89A, 0x1F899, 0x1F89B, 0x1F83C, 0x1F83E, 0x1F83D, // 0xB8 - 0x1F83F, 0x1F840, 0x1F842, 0x1F841, 0x1F843, 0x1F844, 0x1F846, 0x1F845, // 0xC0 - 0x1F847, 0x2BA8, 0x2BA9, 0x2BAA, 0x2BAB, 0x2BAC, 0x2BAD, 0x2BAE, // 0xC8 - 0x2BAF, 0x1F860, 0x1F862, 0x1F861, 0x1F863, 0x1F864, 0x1F865, 0x1F867, // 0xD0 - 0x1F866, 0x1F870, 0x1F872, 0x1F871, 0x1F873, 0x1F874, 0x1F875, 0x1F877, // 0xD8 - 0x1F876, 0x1F880, 0x1F882, 0x1F881, 0x1F883, 0x1F884, 0x1F885, 0x1F887, // 0xE0 - 0x1F886, 0x1F890, 0x1F892, 0x1F891, 0x1F893, 0x1F894, 0x1F896, 0x1F895, // 0xE8 - 0x1F897, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0xF0 - 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, // 0xF8 -]; - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn both_storage_forms_map_alike() { - for f in ["Symbol", "Wingdings", "Wingdings 2", "Wingdings 3", "Webdings"] { - assert!(is_symbol_font(f)); - for code in 0x20u32..=0xFF { - let (Some(pua), Some(plain)) = (char::from_u32(0xF000 + code), char::from_u32(code)) else { continue }; - if code == 0xAD { - assert_eq!(symbol_char(f, plain), None); - continue; - } - assert_eq!(symbol_char(f, pua), symbol_char(f, plain), "{f} {code:#X}"); - } - } - } - - #[test] - fn pcl6_symbol_encoding() { - for family in ["Symbol (PCL6)", " symbol (pcl6) "] { - assert!(is_symbol_font(family)); - for code in 0x20..=0xFF { - let c = char::from_u32(code).unwrap(); - assert_eq!(symbol_char(family, c), symbol_char("Symbol", c)); - let pua = char::from_u32(0xF000 + code).unwrap(); - assert_eq!(symbol_char(family, pua), symbol_char("Symbol", pua)); - } - assert_eq!(symbol_char(family, '\u{F0B3}'), Some('≥')); - } - assert!(!is_symbol_font("Symbol (unknown)")); - } - - #[test] - fn symbol_uses_the_adobe_symbol_encoding() { - assert_eq!(symbol_char("Symbol", '\u{F061}'), Some('α')); - assert_eq!(symbol_char("symbol", 'a'), Some('α')); - assert_eq!(symbol_char("Symbol", 'm'), Some('μ')); - assert_eq!(symbol_char("Symbol", 'W'), Some('Ω')); - assert_eq!(symbol_char("Symbol", '\u{F0B7}'), Some('•')); - assert_eq!(symbol_char("Symbol", '\u{F0A5}'), Some('∞')); - assert_eq!(symbol_char("Symbol", '\u{F0D6}'), Some('√')); - // The radical extender is private use in the Adobe table: no equivalent. - assert_eq!(symbol_char("Symbol", '\u{F060}'), None); - } - - #[test] - fn wingdings_codes() { - assert_eq!(symbol_char("Wingdings", '\u{F04A}'), Some('☺')); - assert_eq!(symbol_char("Wingdings", 'J'), Some('☺')); - assert_eq!(symbol_char("Wingdings", 'L'), Some('☹')); - assert_eq!(symbol_char("Wingdings", '\u{F0FC}'), Some('✓')); - assert_eq!(symbol_char("Wingdings", '\u{F0FE}'), Some('\u{1F5F9}')); - assert_eq!(symbol_char("Wingdings", '\u{F06E}'), Some('◼')); - assert_eq!(symbol_char("Wingdings", '\u{F0A7}'), Some('▪')); - assert_eq!(symbol_char("Wingdings", '\u{F076}'), Some('❖')); - assert_eq!(symbol_char("Wingdings", '\u{F028}'), Some('\u{1F57F}')); - // The one character Unicode didn't encode. - assert_eq!(symbol_char("Wingdings", '\u{F0FF}'), None); - } - - #[test] - fn wingdings_2_3_and_webdings_codes() { - assert_eq!(symbol_char("Wingdings 2", '\u{F050}'), Some('\u{1F5F8}')); - assert_eq!(symbol_char("Wingdings 2", 'O'), Some('\u{1F5F4}')); - assert_eq!(symbol_char("Wingdings 2", '\u{F0FA}'), None); - assert_eq!(symbol_char("Wingdings 3", '\u{F021}'), Some('⭠')); - assert_eq!(symbol_char("Wingdings 3", '\u{F0F1}'), None); - assert_eq!(symbol_char("Webdings", '\u{F059}'), Some('\u{1F394}')); - assert_eq!(symbol_char("Webdings", 'a'), Some('\u{2714}')); - assert_eq!(symbol_char("Webdings", '\u{F0FF}'), Some('\u{1F54A}')); - } - - #[test] - fn other_fonts_and_characters_are_left_alone() { - assert!(!is_symbol_font("Arial")); - assert_eq!(symbol_char("Arial", '\u{F04A}'), None); - assert_eq!(symbol_char("Wingdings", '\u{1F600}'), None); - assert_eq!(symbol_char("Wingdings", '\u{10FFFF}'), None); - assert_eq!(symbol_char("Wingdings", '\u{1F}'), None); - assert_eq!(map_text("Wingdings", "J\u{F04C}é"), "☺☹\u{1F879}"); - assert_eq!(map_text("Arial", "J\u{F04C}"), "J\u{F04C}"); - } -}