From 7480fc2ac42f885f03539211edaebd15cd800584 Mon Sep 17 00:00:00 2001 From: Tex Jernigan Date: Wed, 11 Nov 2026 07:11:59 -0500 Subject: [PATCH] craft-media: write and read MP4/MOV chapter lists (#526) * craft-media: write and read MP4/MOV chapter lists WriterOptions::chapters (Chapter { start, title }, starts in the movie timescale) makes Mp4Writer write a chapter list in the two forms players read: - a QuickTime chapter track (a disabled text track referenced by tref/chap from the first video or audio track, one UTF-8 sample with an encd atom per chapter) for QuickTime / AVFoundation, and - a Nero udta/chpl list (version 1, 100 ns units) for VLC and ffprobe -show_chapters. Chapters are normalized (sorted, one per start, inside the movie, at most 255 chapters with 255-byte titles cut at a character boundary, control characters replaced) and a late first chapter is placed with an empty edit. FragmentedWriter writes chpl only. The demuxer reads both forms back: Mp4File::chpl_chapters and Mp4File::read_chapters(src). Ported from FilmCraft's own isobmff crate (storytold/filmcraft#2821, for storytold/filmcraft#2747), now that FilmCraft uses craft-media. Co-Authored-By: Claude Opus 5.5 * craft-media: read chapters only from enabled tracks, inside the edit, with bounded titles read_chapters took a tref/chap from any track, though QuickTime requires the referencing track to be enabled. It ignored the chapter track's edit list, so a chapter before the edit start became a second chapter at zero and samples past its end were returned. It kept titles of any length. Now a disabled host is skipped; chapters outside the first non-empty edit are dropped, and one spanning its start begins there; each sample read is capped at 4098 bytes and titles are cut to MAX_CHAPTER_TITLE_BYTES, which bounds what is kept. Later edits are still ignored. Co-authored-by: Claude Opus 5.5 * craft-media: a chapter title longer than the sample read keeps its start A sample declaring more title than the 4096 bytes read parsed to nothing, so the chapter lost its whole title rather than being cut to MAX_CHAPTER_TITLE_BYTES. A sample cut by the read now decodes what it holds; one read whole but declaring more than it has is still no title. --------- Co-authored-by: Claude Opus 5.5 --- crates/media/README.md | 23 +- crates/media/src/isobmff/chapters.rs | 239 ++++++++++++++++++++ crates/media/src/isobmff/demux.rs | 61 ++++- crates/media/src/isobmff/mod.rs | 4 +- crates/media/src/isobmff/mux.rs | 318 +++++++++++++++++++++++++-- crates/media/tests/oracle_mux.rs | 68 ++++++ 6 files changed, 694 insertions(+), 19 deletions(-) create mode 100644 crates/media/src/isobmff/chapters.rs diff --git a/crates/media/README.md b/crates/media/README.md index 91fc86f9..8683b124 100644 --- a/crates/media/README.md +++ b/crates/media/README.md @@ -151,7 +151,8 @@ let bytes = mp4.read_sample(&file, v, key)?; // raw sample (AVCC/HVCC length- The one exception is the first sample of a `tmcd` track, which it reads (4 bytes) to get the timecode start frame. - `Mp4File`: brands, `is_quicktime`, movie `timescale`/`duration`, `tracks`, `metadata`, `fragmented`, - `fragment_count`. + `fragment_count`, `chpl_chapters` (Nero `udta/chpl`). `read_chapters(src)` returns the chapter list: + the QuickTime chapter track (`tref/chap`) when there is one, else `chpl`. - `Track`: `id`, `kind` (Video/Audio/Timecode/Subtitle/Other), `handler`, `timescale`, `duration`, `language`, `tkhd` fields (size, matrix, volume, enabled), `entries: Vec`, `samples`, `edits`, `edit_offset`, `references` (`tref`), `sdtp`, `sample_groups` (`sbgp`+`sgpd`, raw), `pcm_chunked`. @@ -171,7 +172,7 @@ let bytes = mp4.read_sample(&file, v, key)?; // raw sample (AVCC/HVCC length- #### Parsed boxes `ftyp`, `moov/mvhd`, `trak/tkhd`, `edts/elst` (v0/v1), `tref`, `mdia/mdhd` (v0/v1, ISO and Mac language codes), `hdlr` (ISO and Pascal names), `minf/stbl`: `stsd`, `stts`, `ctts` (v0/v1, read as signed), `stss`, -`stsc`, `stsz`/`stz2` (4/8/16-bit), `stco`/`co64`, `sdtp`, `sbgp`/`sgpd`. Also `mvex/trex`, `moof/traf/tfhd/tfdt/trun` +`stsc`, `stsz`/`stz2` (4/8/16-bit), `stco`/`co64`, `sdtp`, `sbgp`/`sgpd`, `udta/chpl`. Also `mvex/trex`, `moof/traf/tfhd/tfdt/trun` (every `tfhd` base-offset mode and multiple `moof`s), and `udta` metadata (QuickTime `©xxx` atoms, `meta/ilst`, and `meta/keys` (mdta)). `uuid`, `free`, `skip`, `wide`, `sidx` and `mfra` are skipped. Boxes with 64-bit sizes and with size 0 (runs to the end of the file) are supported. @@ -239,6 +240,15 @@ let bytes = w.finish_faststart()?.into_inner(); // or finish() for moov-at-end - Timecode: `add_timecode_track(video, TimecodeConfig, start_frame)` adds a `tmcd` track (with `gmhd`) and a `tref/tmcd` link from the video track. - `WriterOptions::metadata` writes `©xxx` text atoms to `udta`. +- `WriterOptions::chapters` (`Chapter { start, title }`, starts in the movie timescale) writes a chapter + list in both forms players read: a QuickTime chapter track (a disabled `text` track with `gmhd`, + one length-prefixed UTF-8 sample plus `encd` atom per chapter, referenced by `tref/chap` from the first + video or audio track; Apple's QuickTime File Format Specification, 2016 edition, "Chapter lists" + and "Text media") for Apple players, and a Nero `udta/chpl` list (version 1, 100 ns units; there is + no official specification, this is the publicly documented layout) for VLC and ffprobe. Chapters are + sorted, one per start time, cut to the movie duration, at most 255 (`MAX_CHAPTERS`) with titles of at + most 255 bytes (`MAX_CHAPTER_TITLE_BYTES`, cut at a character boundary). A first chapter after the + start is placed with an empty edit. The fragmented writer writes only `chpl`. - `FragmentedWriter` writes fMP4. The init segment is `ftyp` + `moov` with `mvex/trex`. Each `flush_fragment()` then writes `moof` (`mfhd`, `traf` with default-base-is-moof `tfhd`, `tfdt` v1, `trun` with per-sample duration, size, flags and composition offset) followed by `mdat`. @@ -286,6 +296,11 @@ for b in BoxIter::new(&bytes) { - Muxer: one `mdat`, no chunk-size or duration limits outside PCM (the caller controls interleaving), no `sdtp`, `sgpd`/`sbgp` or `cslg` output, ISO-style `meta/ilst` metadata is not written, and the fragmented writer has no PCM, `sidx` or `mfra`. +- Chapters: at most 255 per file and 255 bytes per title (the `chpl` count and length fields are one + byte each); chapters past the end of the movie are dropped. Only the first `tref/chap` reference is + read, and chapter-track samples are read as titles only (`styl`/`hlit` atoms and per-chapter URLs or + images are ignored). ffprobe ends the last chapter early by the empty edit when the first chapter + starts after zero. ## `matroska` @@ -460,7 +475,9 @@ custom source (a web `Blob`, a network range reader) to use every container with fragmented) and read them back. - `tests/oracle_demux.rs` and `tests/oracle_mux.rs` compare the demuxer and muxer with ffprobe and ffmpeg on generated fixtures in `target/fixtures/isobmff/` (`$CRAFT_MEDIA_FIXTURES_DIR` - overrides it). Fixtures are never committed. + overrides it). Fixtures are never committed. `chapters_are_listed_by_ffprobe` checks that + `ffprobe -show_chapters` lists the chapters the muxer writes (count, starts, ends, UTF-8 titles) in + MP4 and MOV. - `tests/oracle_matroska.rs` compares every packet (stream, size, keyframe flag, pts, duration), codec names, time bases, dimensions and audio parameters with ffprobe on fixtures in `target/fixtures/matroska/`: H.264+AAC, VP9+Opus WebM, FLAC, Vorbis, SRT+ASS, WebVTT WebM, Cues diff --git a/crates/media/src/isobmff/chapters.rs b/crates/media/src/isobmff/chapters.rs new file mode 100644 index 00000000..dfb83197 --- /dev/null +++ b/crates/media/src/isobmff/chapters.rs @@ -0,0 +1,239 @@ +//! Chapter lists, in the two forms players read. +//! +//! - **QuickTime chapter track** (QuickTime File Format, "Chapter lists" and "Text media"): a +//! disabled text track whose samples are the chapter titles, referenced from a video or audio track +//! by a `tref/chap` track reference. QuickTime Player, Apple TV, iOS and Final Cut read this form. +//! Each sample is a 16-bit big-endian length, the title, and an `encd` atom with value `0x100` +//! marking the text as UTF-8 (the convention Apple's own chapter tracks in `.m4a`/`.m4v` files use; +//! plain QuickTime text is Mac Roman). +//! - **Nero chapter list** `moov/udta/chpl`, version 1: `u32` reserved, `u8` chapter count, then per +//! chapter a `u64` start in 100 ns units, a `u8` title length and the UTF-8 title. It has no +//! official specification; this is the layout publicly documented for Nero Digital files and +//! written by many tools. VLC and ffprobe (`-show_chapters`) read it. +//! +//! Both forms carry at most [`MAX_CHAPTERS`] chapters with titles of at most +//! [`MAX_CHAPTER_TITLE_BYTES`] bytes (the `chpl` count and length fields are one byte each). + +use crate::isobmff::bytes::{BoxBuf, Cur}; + +/// One chapter mark: `start` in the movie timescale ([`crate::isobmff::WriterOptions::movie_timescale`] +/// when writing, [`crate::isobmff::Mp4File::timescale`] when reading), `title` UTF-8. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Chapter { + pub start: u64, + pub title: String, +} + +/// Most chapters written (the `chpl` count is one byte); later ones are dropped. +pub const MAX_CHAPTERS: usize = 255; +/// Longest chapter title in bytes (the `chpl` length is one byte); longer titles are cut at a +/// character boundary. +pub const MAX_CHAPTER_TITLE_BYTES: usize = 255; + +/// `chpl` times are in 100 ns units. +const CHPL_UNITS_PER_SECOND: u64 = 10_000_000; + +/// Cut `s` to at most `max` bytes without splitting a character. +pub(crate) fn truncate_utf8(s: &str, max: usize) -> &str { + if s.len() <= max { + return s; + } + let mut end = max; + while end > 0 && !s.is_char_boundary(end) { + end -= 1; + } + s.get(..end).unwrap_or("") +} + +/// The chapters a file can carry: sorted by start, one per start time (the first title given for +/// a time wins), starts before `duration` only (when it is known), titles cut to +/// [`MAX_CHAPTER_TITLE_BYTES`] with control characters replaced by spaces, at most +/// [`MAX_CHAPTERS`]. +pub fn normalize_chapters(chapters: &[Chapter], duration: Option) -> Vec { + let mut v: Vec<&Chapter> = chapters.iter().filter(|c| duration.is_none_or(|d| c.start < d)).collect(); + v.sort_by_key(|c| c.start); + v.dedup_by_key(|c| c.start); + v.into_iter() + .take(MAX_CHAPTERS) + .map(|c| { + let clean: String = c.title.chars().map(|ch| if ch.is_control() { ' ' } else { ch }).collect(); + Chapter { start: c.start, title: truncate_utf8(clean.trim(), MAX_CHAPTER_TITLE_BYTES).to_string() } + }) + .collect() +} + +fn rescale(v: u64, from: u64, to: u64) -> u64 { + if from == 0 { + return 0; + } + (u128::from(v) * u128::from(to) / u128::from(from)).min(u128::from(u64::MAX)) as u64 +} + +/// Write `udta/chpl` content (the box itself) for already normalized chapters. +pub(crate) fn write_chpl(b: &mut BoxBuf, chapters: &[Chapter], movie_timescale: u32) { + let m = b.start_full(b"chpl", 1, 0); + b.u32(0); + let n = chapters.len().min(MAX_CHAPTERS); + b.u8(n as u8); + for c in chapters.iter().take(n) { + b.u64(rescale(c.start, u64::from(movie_timescale), CHPL_UNITS_PER_SECOND)); + let t = truncate_utf8(&c.title, MAX_CHAPTER_TITLE_BYTES); + b.u8(t.len() as u8); + b.bytes(t.as_bytes()); + } + b.end(m); +} + +/// Parse a `chpl` payload (after the box header) into chapters in `movie_timescale` units. +/// Malformed or truncated lists yield the chapters read so far. +pub(crate) fn parse_chpl(p: &[u8], movie_timescale: u32) -> Vec { + let mut out = Vec::new(); + let mut c = Cur::labeled(p, "chpl"); + let Ok((version, _)) = c.full_header() else { return out }; + if version >= 1 && c.skip(4).is_err() { + return out; + } + let Ok(n) = c.u8() else { return out }; + for _ in 0..n { + let Ok(start) = c.u64() else { break }; + let Ok(len) = c.u8() else { break }; + let Ok(title) = c.bytes(len as usize) else { break }; + out.push(Chapter { + start: rescale(start, CHPL_UNITS_PER_SECOND, u64::from(movie_timescale)), + title: String::from_utf8_lossy(title).into_owned(), + }); + } + out +} + +/// Payload of the QuickTime `text` sample description (after its 8-byte box header), QTFF "Text +/// sample description": reserved + data reference index, display flags, justification, background +/// colour, default text box, reserved, font number and face, reserved, foreground colour, an empty +/// Pascal font name, then an `ftab` font table naming one font. +pub(crate) fn text_sample_entry_payload() -> Vec { + let mut b = BoxBuf::new(); + b.zeros(6); + b.u16(1); // data reference index + b.u32(0); // display flags + b.i32(0); // justification: left + b.zeros(6); // background colour (RGB) + b.zeros(8); // default text box (top, left, bottom, right) + b.zeros(8); // reserved + b.u16(1); // font number (matches the ftab entry) + b.u16(0); // font face: plain + b.u8(0); // reserved + b.u16(0); // reserved + b.zeros(6); // foreground colour (RGB) + b.u8(0); // text name: empty Pascal string + let f = b.start(b"ftab"); + b.u16(1); + b.u16(1); + let name = b"Sans-Serif"; + b.u8(name.len() as u8); + b.bytes(name); + b.end(f); + b.buf +} + +/// One chapter-track sample: length-prefixed UTF-8 title plus an `encd` atom (UTF-8). +pub(crate) fn text_sample(title: &str) -> Vec { + let t = truncate_utf8(title, MAX_CHAPTER_TITLE_BYTES); + let mut b = BoxBuf::new(); + b.u16(t.len() as u16); + b.bytes(t.as_bytes()); + let e = b.start(b"encd"); + b.u32(0x100); + b.end(e); + b.buf +} + +/// The title in a QuickTime text sample (UTF-8, or UTF-16 when it starts with a byte order mark). +pub fn parse_text_sample(data: &[u8]) -> Option { + let mut c = Cur::labeled(data, "text sample"); + let n = c.u16().ok()? as usize; + decode_title(c.bytes(n).ok()?) +} + +/// [`parse_text_sample`] of the start of a longer sample: as much of the title as `data` holds. +pub(crate) fn parse_text_sample_prefix(data: &[u8]) -> Option { + let (declared, rest) = data.split_first_chunk::<2>()?; + decode_title(rest.get(..usize::from(u16::from_be_bytes(*declared))).unwrap_or(rest)) +} + +fn decode_title(t: &[u8]) -> Option { + if let Some(rest) = t.strip_prefix(&[0xFE, 0xFF]) { + let units: Vec = rest.as_chunks::<2>().0.iter().map(|u| u16::from_be_bytes(*u)).collect(); + return Some(String::from_utf16_lossy(&units)); + } + if let Some(rest) = t.strip_prefix(&[0xFF, 0xFE]) { + let units: Vec = rest.as_chunks::<2>().0.iter().map(|u| u16::from_le_bytes(*u)).collect(); + return Some(String::from_utf16_lossy(&units)); + } + // trailing atoms (`encd`, `styl` …) are not part of the text + Some(String::from_utf8_lossy(t).into_owned()) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn ch(start: u64, title: &str) -> Chapter { + Chapter { start, title: title.into() } + } + + #[test] + fn normalize_sorts_dedups_drops_and_caps() { + let v = normalize_chapters(&[ch(500, "b"), ch(0, "a"), ch(500, "dup"), ch(2000, "past end"), ch(1999, "\tend\n")], Some(2000)); + assert_eq!(v, vec![ch(0, "a"), ch(500, "b"), ch(1999, "end")]); + let many: Vec = (0..1000).map(|i| ch(i, "x")).collect(); + assert_eq!(normalize_chapters(&many, None).len(), MAX_CHAPTERS); + let long = "é".repeat(300); + let t = &normalize_chapters(&[ch(0, &long)], None)[0].title; + assert!(t.len() <= MAX_CHAPTER_TITLE_BYTES && t.chars().all(|c| c == 'é')); + assert!(normalize_chapters(&[ch(0, "a")], Some(0)).is_empty()); + } + + #[test] + fn chpl_roundtrip() { + let chapters = vec![ch(0, "Intro"), ch(1500, "Teil 2 — ü"), ch(u64::MAX, "far")]; + let mut b = BoxBuf::new(); + write_chpl(&mut b, &chapters, 1000); + // box header (8) + version/flags (4) + reserved (4) + count (1) + assert_eq!(&b.buf[4..8], b"chpl"); + assert_eq!(b.buf[8], 1); + assert_eq!(b.buf[16], 3); + assert_eq!(u64::from_be_bytes(b.buf[17..25].try_into().unwrap()), 0); + assert_eq!(u64::from_be_bytes(b.buf[31..39].try_into().unwrap()), 15_000_000); + let back = parse_chpl(&b.buf[8..], 1000); + assert_eq!(back[..2], chapters[..2]); + assert_eq!(back.len(), 3); + } + + #[test] + fn chpl_parse_survives_garbage() { + let mut b = BoxBuf::new(); + write_chpl(&mut b, &[ch(0, "a"), ch(10, "bb")], 1000); + let p = b.buf[8..].to_vec(); + for cut in 0..p.len() { + let _ = parse_chpl(&p[..cut], 1000); + } + let mut lying = p.clone(); + lying[8] = 255; // count says 255 + assert_eq!(parse_chpl(&lying, 1000).len(), 2); + assert!(parse_chpl(&p, 0).iter().all(|c| c.start == 0)); + } + + #[test] + fn text_sample_roundtrip() { + let s = text_sample("Kapitel “1”"); + assert_eq!(parse_text_sample(&s).as_deref(), Some("Kapitel “1”")); + assert_eq!(&s[s.len() - 12..s.len() - 4], &[0, 0, 0, 12, b'e', b'n', b'c', b'd']); + assert_eq!(parse_text_sample(&[0, 4, 0xFE, 0xFF, 0, b'A']).as_deref(), Some("A")); + assert_eq!(parse_text_sample(&[0, 9, b'a']), None); + assert_eq!(parse_text_sample(&[]), None); + // The start of a longer sample: what it holds of the title, and no further than the title. + assert_eq!(parse_text_sample_prefix(&[0, 9, b'a']).as_deref(), Some("a")); + assert_eq!(parse_text_sample_prefix(&[0, 1, b'a', b'b']).as_deref(), Some("a")); + assert_eq!(parse_text_sample_prefix(&[0]), None); + } +} diff --git a/crates/media/src/isobmff/demux.rs b/crates/media/src/isobmff/demux.rs index 3c7a1c88..b6f2e704 100644 --- a/crates/media/src/isobmff/demux.rs +++ b/crates/media/src/isobmff/demux.rs @@ -291,6 +291,9 @@ pub struct Mp4File { pub moov_offset: u64, /// Number of `moof` boxes found. pub fragment_count: usize, + /// Nero chapter list (`udta/chpl`), starts in the movie timescale. QuickTime chapter tracks + /// live in media data: read them with [`Mp4File::read_chapters`]. + pub chpl_chapters: Vec, } impl Mp4File { @@ -306,12 +309,60 @@ impl Mp4File { pub fn sync_sample_before(&self, track: usize, index: usize) -> Option { Some(self.tracks.get(track)?.sync_sample_before(index)) } + /// The chapter list: the QuickTime chapter track (the text track an enabled track's `tref/chap` + /// points at; titles read from `src`, starts in the movie timescale, edit-adjusted) when there is + /// one, else the Nero `chpl` list. Empty when the file has neither. + /// + /// Chapters outside the track's first non-empty edit are left out, and one spanning that edit's + /// start begins where the edit does; later edits are ignored. Titles are cut to + /// [`MAX_CHAPTER_TITLE_BYTES`](crate::isobmff::MAX_CHAPTER_TITLE_BYTES). + pub fn read_chapters(&self, src: &S) -> Result> { + let chap_id = self + .tracks + .iter() + .filter(|t| t.enabled) + .flat_map(|t| &t.references) + .find(|(k, _)| k == b"chap") + .and_then(|(_, ids)| ids.first().copied()); + let Some(t) = chap_id.and_then(|id| self.tracks.iter().find(|t| t.id == id)) else { return Ok(self.chpl_chapters.clone()) }; + let edit = t.edits.iter().find(|e| e.media_time != -1); + let media_start = edit.map_or(i64::MIN, |e| e.media_time); + let media_end = edit + .filter(|e| e.segment_duration > 0) + .map_or(i64::MAX, |e| media_start.saturating_add(rescale(i128::from(e.segment_duration), self.timescale, t.timescale))); + let mut out = Vec::new(); + for s in t.samples.iter().take(crate::isobmff::MAX_CHAPTERS.saturating_mul(16)) { + if s.pts >= media_end || (s.pts.saturating_add(i64::from(s.duration)) <= media_start && s.pts < media_start) { + continue; + } + let data = read_range(src, s.offset, u64::from(s.size).min(MAX_CHAPTER_SAMPLE_BYTES))?; + let title = if u64::from(s.size) > MAX_CHAPTER_SAMPLE_BYTES { + crate::isobmff::chapters::parse_text_sample_prefix(&data) + } else { + crate::isobmff::chapters::parse_text_sample(&data) + }; + let title = title.unwrap_or_default(); + let title = crate::isobmff::chapters::truncate_utf8(&title, crate::isobmff::MAX_CHAPTER_TITLE_BYTES).to_string(); + let pts = s.pts.max(media_start).saturating_add(t.edit_offset).max(0) as u64; + let start = if t.timescale == 0 { + 0 + } else { + (u128::from(pts) * u128::from(self.timescale) / u128::from(t.timescale)).min(u128::from(u64::MAX)) as u64 + }; + out.push(crate::isobmff::Chapter { start, title }); + } + Ok(out) + } /// First track of the given kind. pub fn track_of_kind(&self, kind: TrackKind) -> Option { self.tracks.iter().position(|t| t.kind == kind) } } +/// Most of a chapter sample read for its title: its 16-bit length and far more title than is kept. +/// A longer title is decoded from what this holds of it. +const MAX_CHAPTER_SAMPLE_BYTES: u64 = 4098; + fn read_range(src: &S, offset: u64, size: u64) -> Result> { let end = offset.checked_add(size).ok_or(Error::Truncated("sample"))?; if end > src.len() { @@ -368,6 +419,7 @@ pub fn open(src: S) -> Result { fragmented: false, moov_offset, fragment_count: moofs.len(), + chpl_chapters: Vec::new(), }; if let Some(f) = &ftyp { let mut c = Cur::labeled(f, "ftyp"); @@ -381,6 +433,7 @@ pub fn open(src: S) -> Result { let mut trex: HashMap = HashMap::new(); let mut trak_payloads = Vec::new(); let mut meta_left = MAX_METADATA_BYTES; + let mut chpl = None; for b in boxes(&moov) { let b = b?; match &b.kind.0 { @@ -410,11 +463,17 @@ pub fn open(src: S) -> Result { } } } - b"udta" => parse_udta(b.payload, &mut file.metadata, &mut meta_left), + b"udta" => { + parse_udta(b.payload, &mut file.metadata, &mut meta_left); + chpl = find(b.payload, b"chpl"); + } b"meta" => parse_meta(b.payload, &mut file.metadata, &mut meta_left), _ => {} } } + if let Some(p) = chpl { + file.chpl_chapters = crate::isobmff::chapters::parse_chpl(p, file.timescale); + } let mut budget = SampleBudget::new(len); for p in trak_payloads { let t = parse_trak(p, &file, len, &mut budget)?; diff --git a/crates/media/src/isobmff/mod.rs b/crates/media/src/isobmff/mod.rs index a971f144..4692ebed 100644 --- a/crates/media/src/isobmff/mod.rs +++ b/crates/media/src/isobmff/mod.rs @@ -4,7 +4,7 @@ //! [`ByteSource`] and builds per-track [`Sample`] tables; media data is read on demand with //! [`Mp4File::read_sample`]. //! - **Mux:** [`Mp4Writer`] writes progressive MP4/MOV (optionally faststart); -//! [`FragmentedWriter`] writes fragmented MP4. +//! [`FragmentedWriter`] writes fragmented MP4. Both can write a chapter list ([`chapters`]). //! - **Boxes:** [`boxes`] iterates the boxes in a byte slice and reads their payloads, for box //! formats and boxes the demuxer does not model (`uuid` boxes, HEIF items, JPEG 2000, CR3). //! @@ -12,6 +12,7 @@ pub mod boxes; mod bytes; +pub mod chapters; mod codec; mod demux; mod error; @@ -19,6 +20,7 @@ mod mux; pub use crate::source::{ByteSource, ReadSeekSource}; pub use bytes::FourCc; +pub use chapters::{Chapter, MAX_CHAPTER_TITLE_BYTES, MAX_CHAPTERS}; pub use codec::{ AacConfig, ApvConfig, AudioParams, Av1Config, AvcConfig, BitRate, CleanAperture, CodecConfig, ColorInfo, FieldInfo, FlacConfig, HevcConfig, HevcNalArray, MasteringDisplay, OpusConfig, PcmConfig, SampleEntry, TimecodeConfig, VideoParams, VpcConfig, parse_asc, diff --git a/crates/media/src/isobmff/mux.rs b/crates/media/src/isobmff/mux.rs index 836aaaed..e38186e9 100644 --- a/crates/media/src/isobmff/mux.rs +++ b/crates/media/src/isobmff/mux.rs @@ -1,6 +1,7 @@ //! Muxers: progressive MP4/MOV ([`Mp4Writer`], optional faststart) and fragmented MP4 ([`FragmentedWriter`]). use crate::isobmff::bytes::BoxBuf; +use crate::isobmff::chapters::{Chapter, normalize_chapters, text_sample, text_sample_entry_payload, write_chpl}; use crate::isobmff::codec::{CodecConfig, SampleEntry, TimecodeConfig}; use crate::isobmff::demux::Edit; use crate::isobmff::error::{Error, Result}; @@ -23,11 +24,15 @@ pub struct WriterOptions { pub movie_timescale: u32, /// Text metadata written to `moov/udta` (keys like `©nam`, `©ART`, `©cmt`, `©too`). pub metadata: Vec<(String, String)>, + /// Chapter marks (starts in `movie_timescale`). [`Mp4Writer`] writes them both as a QuickTime + /// chapter track (`tref/chap` → disabled text track) and as a Nero `udta/chpl` list; + /// [`FragmentedWriter`] writes only `chpl`. See [`crate::isobmff::chapters`] for the formats and limits. + pub chapters: Vec, } impl WriterOptions { pub fn new(brand: Brand) -> Self { - WriterOptions { brand, movie_timescale: 1000, metadata: Vec::new() } + WriterOptions { brand, movie_timescale: 1000, metadata: Vec::new(), chapters: Vec::new() } } } @@ -70,6 +75,8 @@ enum Handler { Video, Audio, Timecode, + /// QuickTime text track carrying chapter titles. + Text, Other, } @@ -178,6 +185,8 @@ struct WTrack { tmcd_ref: Option, /// For timecode tracks: (video track index, start frame). tmcd_source: Option<(usize, u32)>, + /// Track id of the chapter text track (`tref/chap`). + chap_ref: Option, } fn pack_language(lang: &str) -> u16 { @@ -268,7 +277,8 @@ fn write_trak(b: &mut BoxBuf, t: &WTrack, movie_ts: u32, qt: bool, shift: u64, f let trak = b.start(b"trak"); let dur = presentation_duration(t, movie_ts); let v1 = dur > u32::MAX as u64; - let m = b.start_full(b"tkhd", v1 as u8, 3); + // a chapter track is in the movie but disabled: players list it as chapters, never show it + let m = b.start_full(b"tkhd", v1 as u8, if t.handler == Handler::Text { 2 } else { 3 }); if v1 { b.u64(0); b.u64(0); @@ -322,11 +332,15 @@ fn write_trak(b: &mut BoxBuf, t: &WTrack, movie_ts: u32, qt: bool, shift: u64, f b.end(l); b.end(e); } - if let Some(id) = t.tmcd_ref { + if t.tmcd_ref.is_some() || t.chap_ref.is_some() { let r = b.start(b"tref"); - let m = b.start(b"tmcd"); - b.u32(id); - b.end(m); + for (kind, id) in [(b"tmcd", t.tmcd_ref), (b"chap", t.chap_ref)] { + if let Some(id) = id { + let m = b.start(kind); + b.u32(id); + b.end(m); + } + } b.end(r); } let mdia = b.start(b"mdia"); @@ -350,6 +364,7 @@ fn write_trak(b: &mut BoxBuf, t: &WTrack, movie_ts: u32, qt: bool, shift: u64, f Handler::Video => (b"vide", "VideoHandler"), Handler::Audio => (b"soun", "SoundHandler"), Handler::Timecode => (b"tmcd", "TimeCodeHandler"), + Handler::Text => (b"text", "Chapters"), Handler::Other => (b"meta", "MetaHandler"), }; let name = t.cfg.handler_name.clone().unwrap_or_else(|| default_name.into()); @@ -401,6 +416,24 @@ fn write_trak(b: &mut BoxBuf, t: &WTrack, movie_ts: u32, qt: bool, shift: u64, f b.end(tm); b.end(g); } + Handler::Text => { + // QTFF "Base media information header" with a text media information atom (a matrix) + let g = b.start(b"gmhd"); + let m = b.start_full(b"gmin", 0, 0); + b.u16(0x40); + b.u16(0x8000); + b.u16(0x8000); + b.u16(0x8000); + b.i16(0); + b.u16(0); + b.end(m); + let tx = b.start(b"text"); + for v in MATRIX { + b.i32(v); + } + b.end(tx); + b.end(g); + } Handler::Other => { let m = b.start_full(b"nmhd", 0, 0); b.end(m); @@ -514,7 +547,7 @@ fn write_stbl(b: &mut BoxBuf, t: &WTrack, qt: bool, shift: u64, force_co64: bool Ok(()) } -fn write_udta(b: &mut BoxBuf, md: &[(String, String)]) { +fn write_udta(b: &mut BoxBuf, md: &[(String, String)], chapters: &[Chapter], movie_ts: u32) { let items: Vec<([u8; 4], &str)> = md .iter() .filter_map(|(k, v)| { @@ -526,7 +559,7 @@ fn write_udta(b: &mut BoxBuf, md: &[(String, String)]) { (rest.len() == 3).then(|| ([0xA9, rest[0], rest[1], rest[2]], v.as_str())) }) .collect(); - if items.is_empty() { + if items.is_empty() && chapters.is_empty() { return; } let u = b.start(b"udta"); @@ -537,6 +570,9 @@ fn write_udta(b: &mut BoxBuf, md: &[(String, String)]) { b.bytes(v.as_bytes()); b.end(m); } + if !chapters.is_empty() { + write_chpl(b, chapters, movie_ts); + } b.end(u); } @@ -567,6 +603,8 @@ pub struct Mp4Writer { mdat_pos: u64, pos: u64, last_track: Option, + /// Chapters as written (normalized against the movie duration at finish). + chapters: Vec, } impl Mp4Writer { @@ -581,7 +619,7 @@ impl Mp4Writer { b.leaf(b"mdat", &[]); w.write_all(&b.buf)?; let pos = start + b.len() as u64; - Ok(Mp4Writer { w, opts, tracks: Vec::new(), pad_pos, mdat_pos, pos, last_track: None }) + Ok(Mp4Writer { w, opts, tracks: Vec::new(), pad_pos, mdat_pos, pos, last_track: None, chapters: Vec::new() }) } /// Add a track; returns its index (track ids are index + 1). @@ -590,7 +628,16 @@ impl Mp4Writer { let handler = handler_of(&cfg.entry); let mut t = Tables::new(); t.const_size = pcm; - self.tracks.push(WTrack { id: self.tracks.len() as u32 + 1, handler, pcm_frame_bytes: pcm, t, cfg, tmcd_ref: None, tmcd_source: None }); + self.tracks.push(WTrack { + id: self.tracks.len() as u32 + 1, + handler, + pcm_frame_bytes: pcm, + t, + cfg, + tmcd_ref: None, + tmcd_source: None, + chap_ref: None, + }); Ok(self.tracks.len() - 1) } @@ -637,8 +684,8 @@ impl Mp4Writer { /// number of audio frames. pub fn write_sample(&mut self, track: usize, s: WriteSample) -> Result<()> { let tr = self.tracks.get(track).ok_or(Error::NoSuchTrack(track))?; - if tr.tmcd_source.is_some() { - return Err(Error::Mux("timecode track samples are written automatically".into())); + if tr.tmcd_source.is_some() || tr.handler == Handler::Text { + return Err(Error::Mux("timecode and chapter track samples are written automatically".into())); } let (count, dur) = match tr.pcm_frame_bytes { Some(fb) => { @@ -713,6 +760,7 @@ impl Mp4Writer { t.media_duration = d as u64; t.chunks.push((off, 1)); } + self.write_chapter_track()?; let size = self.pos - self.mdat_pos; if size <= u32::MAX as u64 { self.w.seek(SeekFrom::Start(self.mdat_pos))?; @@ -730,6 +778,59 @@ impl Mp4Writer { Ok(()) } + /// Add the QuickTime chapter track and its samples (one per chapter, each lasting until the next + /// chapter or the end of the movie), referenced by `tref/chap` from the first video or audio + /// track. A first chapter after the start is placed with an empty edit. + fn write_chapter_track(&mut self) -> Result<()> { + let ts = self.opts.movie_timescale.max(1); + let duration = movie_duration(&self.tracks, ts); + self.chapters = normalize_chapters(&self.opts.chapters, Some(duration)); + let Some(first) = self.chapters.first().map(|c| c.start) else { return Ok(()) }; + let Some(host) = self.tracks.iter().position(|t| matches!(t.handler, Handler::Video | Handler::Audio)) else { + // nothing to hang a QuickTime chapter track on; `chpl` still carries the chapters + return Ok(()); + }; + let entry = SampleEntry { + format: crate::isobmff::bytes::FourCc(*b"text"), + data_reference_index: 1, + codec: CodecConfig::Unknown { fourcc: crate::isobmff::bytes::FourCc(*b"text"), raw: text_sample_entry_payload() }, + video: None, + audio: None, + bitrate: None, + }; + let mut cfg = TrackConfig::new(entry, ts); + if first > 0 { + cfg.edits = vec![ + Edit { segment_duration: first, media_time: -1, media_rate: 0x10000 }, + Edit { segment_duration: duration.saturating_sub(first), media_time: 0, media_rate: 0x10000 }, + ]; + } + let id = self.tracks.len() as u32 + 1; + let mut t = Tables::new(); + let off = self.pos; + for (i, c) in self.chapters.iter().enumerate() { + let end = self.chapters.get(i + 1).map_or(duration, |n| n.start); + let d = u32::try_from(end.saturating_sub(c.start)).map_err(|_| Error::Mux("chapter longer than 2^32 movie ticks".into()))?; + let data = text_sample(&c.title); + self.w.write_all(&data)?; + self.pos += data.len() as u64; + Tables::push_run(&mut t.stts, 1, d); + Tables::push_run(&mut t.ctts, 1, 0); + t.sizes.push(data.len() as u32); + t.sample_count += 1; + t.sync.push(t.sample_count as u32); + t.media_duration += u64::from(d); + } + t.comp_end = t.media_duration; + t.chunks.push((off, self.chapters.len() as u32)); + self.last_track = None; + self.tracks.push(WTrack { cfg, id, handler: Handler::Text, pcm_frame_bytes: None, t, tmcd_ref: None, tmcd_source: None, chap_ref: None }); + if let Some(h) = self.tracks.get_mut(host) { + h.chap_ref = Some(id); + } + Ok(()) + } + fn build_moov(&self, shift: u64, force_co64: bool) -> Result> { let qt = self.opts.brand == Brand::Mov; let ts = self.opts.movie_timescale.max(1); @@ -739,7 +840,7 @@ impl Mp4Writer { for t in &self.tracks { write_trak(&mut b, t, ts, qt, shift, force_co64)?; } - write_udta(&mut b, &self.opts.metadata); + write_udta(&mut b, &self.opts.metadata, &self.chapters, ts); b.end(m); Ok(b.buf) } @@ -829,6 +930,7 @@ impl FragmentedWriter { cfg, tmcd_ref: None, tmcd_source: None, + chap_ref: None, }); self.pending.push(Vec::new()); self.next_dts.push(0); @@ -856,7 +958,7 @@ impl FragmentedWriter { b.end(x); } b.end(mv); - write_udta(&mut b, &self.opts.metadata); + write_udta(&mut b, &self.opts.metadata, &normalize_chapters(&self.opts.chapters, None), ts); b.end(m); self.w.write_all(&b.buf)?; self.init_written = true; @@ -1027,6 +1129,194 @@ mod tests { assert!(matches!(w.set_sample_entry(5, avc_entry()), Err(Error::NoSuchTrack(5)))); } + fn chaptered(brand: Brand, chapters: Vec, fast: bool) -> Vec { + let mut opts = WriterOptions::new(brand); + opts.chapters = chapters; + let mut w = Mp4Writer::new(Cursor::new(Vec::new()), opts).unwrap(); + let v = w.add_track(TrackConfig::new(avc_entry(), 12800)).unwrap(); + // 100 frames of 512 / 12800 s = 4 s + for i in 0..100u32 { + w.write_sample(v, WriteSample { data: &[1, 2, 3], duration: 512, composition_offset: 0, is_sync: i % 25 == 0 }).unwrap(); + } + if fast { w.finish_faststart() } else { w.finish() }.unwrap().into_inner() + } + + fn ch(start: u64, title: &str) -> Chapter { + Chapter { start, title: title.into() } + } + + #[test] + fn chapters_are_written_as_quicktime_track_and_chpl() { + for brand in [Brand::Mp4, Brand::Mov] { + for fast in [false, true] { + let out = chaptered(brand, vec![ch(2500, "Zwei"), ch(0, "Eins"), ch(1000, "日本語")], fast); + let f = crate::isobmff::open(out.as_slice()).unwrap(); + let want = vec![ch(0, "Eins"), ch(1000, "日本語"), ch(2500, "Zwei")]; + assert_eq!(f.chpl_chapters, want, "{brand:?} chpl"); + assert_eq!(f.read_chapters(out.as_slice()).unwrap(), want, "{brand:?} chapter track"); + // the text track: disabled, referenced by the video track's tref/chap, lasting the movie + assert_eq!(f.tracks.len(), 2); + let t = &f.tracks[1]; + assert_eq!(t.handler, b"text"); + assert!(!t.enabled); + assert_eq!(f.tracks[0].references, vec![(crate::isobmff::FourCc(*b"chap"), vec![t.id])]); + let durs: Vec = t.samples.iter().map(|s| s.duration).collect(); + assert_eq!(durs, vec![1000, 1500, 1500]); + assert_eq!(f.duration, 4000); + // the video still reads back + assert_eq!(f.tracks[0].samples.len(), 100); + assert_eq!(f.read_sample(out.as_slice(), 0, 99).unwrap(), vec![1, 2, 3]); + } + } + } + + #[test] + fn a_late_first_chapter_is_placed_by_an_empty_edit() { + let out = chaptered(Brand::Mp4, vec![ch(1500, "Later"), ch(3000, "End")], false); + let f = crate::isobmff::open(out.as_slice()).unwrap(); + let t = &f.tracks[1]; + assert_eq!(t.edits.len(), 2); + assert_eq!(t.edits[0].media_time, -1); + assert_eq!(t.edits[0].segment_duration, 1500); + assert_eq!(f.read_chapters(out.as_slice()).unwrap(), vec![ch(1500, "Later"), ch(3000, "End")]); + assert_eq!(f.chpl_chapters, vec![ch(1500, "Later"), ch(3000, "End")]); + } + + /// The media time (chapter track timescale) of `movie` ticks. + fn chapter_media_time(f: &crate::isobmff::Mp4File, movie: u64) -> i64 { + (u128::from(movie) * u128::from(f.tracks[1].timescale) / u128::from(f.timescale)) as i64 + } + + #[test] + fn chapters_are_clipped_to_the_chapter_tracks_edit() { + let out = chaptered(Brand::Mp4, vec![ch(0, "A"), ch(1000, "B"), ch(2000, "C"), ch(3000, "D")], false); + let mut f = crate::isobmff::open(out.as_slice()).unwrap(); + // the edit starts inside B and ends inside C: A and D are outside, B begins at zero + let from = chapter_media_time(&f, 1500); + f.tracks[1].edits = vec![crate::isobmff::Edit { segment_duration: 1000, media_time: from, media_rate: 0x0001_0000 }]; + f.tracks[1].edit_offset = -from; + assert_eq!(f.read_chapters(out.as_slice()).unwrap(), vec![ch(0, "B"), ch(500, "C")]); + } + + #[test] + fn chapters_come_from_an_enabled_track_only() { + let out = chaptered(Brand::Mp4, vec![ch(0, "Track"), ch(1000, "Two")], false); + let mut f = crate::isobmff::open(out.as_slice()).unwrap(); + f.chpl_chapters = vec![ch(0, "Nero")]; + assert_eq!(f.read_chapters(out.as_slice()).unwrap()[0].title, "Track"); + for t in f.tracks.iter_mut().filter(|t| t.references.iter().any(|(k, _)| k == b"chap")) { + t.enabled = false; + } + assert_eq!(f.read_chapters(out.as_slice()).unwrap(), vec![ch(0, "Nero")]); + } + + #[test] + fn a_hostile_chapter_sample_is_cut_and_read_only_in_part() { + let mut out = chaptered(Brand::Mp4, vec![ch(0, "A"), ch(1000, "B")], false); + let mut f = crate::isobmff::open(out.as_slice()).unwrap(); + let offset = out.len() as u64; + out.extend_from_slice(&3000u16.to_be_bytes()); + out.extend_from_slice(&[b'x'; 3000]); + // a long title is cut, and a declared size past the end of the file reads only what is needed + f.tracks[1].samples[0].offset = offset; + f.tracks[1].samples[0].size = 3002; + f.tracks[1].samples[1].offset = offset; + f.tracks[1].samples[1].size = u32::MAX; + out.extend_from_slice(&[0; 4098]); + let v = f.read_chapters(out.as_slice()).unwrap(); + assert_eq!(v.len(), 2); + assert!(v.iter().all(|c| c.title.len() == crate::isobmff::MAX_CHAPTER_TITLE_BYTES), "{:?}", v[0].title.len()); + } + + /// A title longer than the part of the sample that is read keeps its first bytes, in either encoding. + #[test] + fn a_title_longer_than_the_sample_read_keeps_its_start() { + let declared = |title: &[u8]| { + let mut sample = (title.len() as u16).to_be_bytes().to_vec(); + sample.extend_from_slice(title); + sample + }; + let utf16 = |bom: [u8; 2], unit: fn(u16) -> [u8; 2]| { + let mut t = bom.to_vec(); + t.extend(std::iter::repeat_n(unit(0x00E9), 3000).flatten()); + t + }; + let cases = [ + (declared(&vec![b'x'; 4097]), "x".repeat(crate::isobmff::MAX_CHAPTER_TITLE_BYTES)), + // 255 bytes of two-byte characters is cut back to a whole character + (declared("é".repeat(10_000).as_bytes()), "é".repeat(127)), + (declared(&utf16([0xFE, 0xFF], u16::to_be_bytes)), "é".repeat(127)), + (declared(&utf16([0xFF, 0xFE], u16::to_le_bytes)), "é".repeat(127)), + ]; + for (sample, want) in cases { + let mut out = chaptered(Brand::Mp4, vec![ch(0, "A"), ch(1000, "B")], false); + let mut f = crate::isobmff::open(out.as_slice()).unwrap(); + f.tracks[1].samples[0].offset = out.len() as u64; + f.tracks[1].samples[0].size = sample.len() as u32; + out.extend_from_slice(&sample); + assert_eq!(f.read_chapters(out.as_slice()).unwrap()[0].title, want); + } + } + + /// A sample whose whole length was read but declares more than it holds is still no title. + #[test] + fn a_short_sample_declaring_a_longer_title_has_no_title() { + let mut out = chaptered(Brand::Mp4, vec![ch(0, "A"), ch(1000, "B")], false); + let mut f = crate::isobmff::open(out.as_slice()).unwrap(); + f.tracks[1].samples[0].offset = out.len() as u64; + f.tracks[1].samples[0].size = 3; + out.extend_from_slice(&[0, 9, b'a']); + assert_eq!(f.read_chapters(out.as_slice()).unwrap()[0].title, ""); + } + + #[test] + fn hostile_chapters_do_not_break_the_file() { + let long = "x".repeat(10_000); + let chapters = vec![ch(0, ""), ch(0, "dup"), ch(4000, "at the end"), ch(u64::MAX, "far"), ch(10, &long), ch(20, "a\0b\nc")]; + let out = chaptered(Brand::Mov, chapters, true); + let f = crate::isobmff::open(out.as_slice()).unwrap(); + let got = f.read_chapters(out.as_slice()).unwrap(); + assert_eq!(got.iter().map(|c| c.start).collect::>(), vec![0, 10, 20]); + assert_eq!(got[0].title, ""); + assert_eq!(got[1].title.len(), crate::isobmff::MAX_CHAPTER_TITLE_BYTES); + assert_eq!(got[2].title, "a b c"); + assert_eq!(f.chpl_chapters, got); + // all chapters past the end: no chapter track, no chpl + let out = chaptered(Brand::Mp4, vec![ch(9000, "gone")], false); + let f = crate::isobmff::open(out.as_slice()).unwrap(); + assert_eq!(f.tracks.len(), 1); + assert!(f.chpl_chapters.is_empty() && f.read_chapters(out.as_slice()).unwrap().is_empty()); + // over the cap + let many: Vec = (0..1000).map(|i| ch(i * 3, "c")).collect(); + let out = chaptered(Brand::Mp4, many, false); + let f = crate::isobmff::open(out.as_slice()).unwrap(); + assert_eq!(f.read_chapters(out.as_slice()).unwrap().len(), crate::isobmff::MAX_CHAPTERS); + assert_eq!(f.chpl_chapters.len(), crate::isobmff::MAX_CHAPTERS); + } + + #[test] + fn chapter_track_samples_cannot_be_written_by_hand() { + let mut opts = WriterOptions::new(Brand::Mp4); + opts.chapters = vec![ch(0, "a")]; + let mut w = Mp4Writer::new(Cursor::new(Vec::new()), opts).unwrap(); + let v = w.add_track(TrackConfig::new(avc_entry(), 1000)).unwrap(); + w.write_sample(v, WriteSample { data: &[1], duration: 40, composition_offset: 0, is_sync: true }).unwrap(); + assert!(w.write_sample(5, WriteSample { data: &[1], duration: 40, composition_offset: 0, is_sync: true }).is_err()); + let out = w.finish().unwrap().into_inner(); + assert_eq!(crate::isobmff::open(out.as_slice()).unwrap().chpl_chapters, vec![ch(0, "a")]); + } + + #[test] + fn fragmented_files_carry_chpl() { + let mut opts = WriterOptions::new(Brand::Mp4); + opts.chapters = vec![ch(40, "b"), ch(0, "a")]; + let mut w = FragmentedWriter::new(Vec::new(), opts); + let v = w.add_track(TrackConfig::new(avc_entry(), 1000)).unwrap(); + w.write_sample(v, WriteSample { data: &[1], duration: 80, composition_offset: 0, is_sync: true }).unwrap(); + let out = w.finish().unwrap(); + assert_eq!(crate::isobmff::open(out.as_slice()).unwrap().chpl_chapters, vec![ch(0, "a"), ch(40, "b")]); + } + #[test] fn fragmented_roundtrip() { let mut w = FragmentedWriter::new(Vec::new(), WriterOptions::new(Brand::Mp4)); diff --git a/crates/media/tests/oracle_mux.rs b/crates/media/tests/oracle_mux.rs index 1294965a..bcb35623 100644 --- a/crates/media/tests/oracle_mux.rs +++ b/crates/media/tests/oracle_mux.rs @@ -268,3 +268,71 @@ fn fragmented_writer_ffmpeg() { } } } + +/// Chapters written by [`Mp4Writer`] (QuickTime chapter track + Nero `chpl`) are what ffprobe +/// lists with `-show_chapters`: the right count, starts, ends and titles, in MP4 and MOV, and the +/// audio/video packets are unchanged. +#[test] +fn chapters_are_listed_by_ffprobe() { + let (Some(src), Some(_)) = (fixture("h264_bframes.mp4"), ffprobe()) else { return }; + let data = std::fs::read(&src).unwrap(); + let f = open(data.as_slice()).unwrap(); + let av: Vec = (0..f.tracks.len()).filter(|&i| matches!(f.tracks[i].kind, TrackKind::Video | TrackKind::Audio)).collect(); + for (name, brand, late_first) in + [("chapters.mp4", Brand::Mp4, false), ("chapters.mov", Brand::Mov, false), ("chapters_late.mp4", Brand::Mp4, true)] + { + let mut opts = WriterOptions::new(brand); + // the copied edit lists are in the source's movie timescale + opts.movie_timescale = f.timescale; + let ms = |v: u64| v * u64::from(f.timescale) / 1000; + let first = if late_first { 250 } else { 0 }; + opts.chapters = vec![ + Chapter { start: ms(first), title: "Opening".into() }, + Chapter { start: ms(500), title: "Mitte — ü 日本".into() }, + Chapter { start: ms(800), title: "End".into() }, + ]; + let mut w = Mp4Writer::new(Cursor::new(Vec::new()), opts).unwrap(); + let mut map = vec![usize::MAX; f.tracks.len()]; + for &i in &av { + map[i] = w.add_track(track_config(&f, &f.tracks[i], true)).unwrap(); + } + for (t, i) in interleave(&f, &av) { + let s = f.tracks[t].samples[i]; + let bytes = f.read_sample(data.as_slice(), t, i).unwrap(); + w.write_sample( + map[t], + WriteSample { data: &bytes, duration: s.duration, composition_offset: (s.pts - s.dts) as i32, is_sync: s.is_sync }, + ) + .unwrap(); + } + let out = w.finish_faststart().unwrap().into_inner(); + let path = out_dir().join(name); + std::fs::write(&path, &out).unwrap(); + let Some(v) = probe_json(&path, &["-show_chapters", "-show_format"]) else { return }; + let ch = v["chapters"].as_array().cloned().unwrap_or_default(); + let got: Vec<(f64, String)> = ch + .iter() + .map(|c| (c["start_time"].as_str().and_then(|s| s.parse().ok()).unwrap_or(-1.0), c["tags"]["title"].as_str().unwrap_or("").to_string())) + .collect(); + let want = [(first as f64 / 1000.0, "Opening"), (0.5, "Mitte — ü 日本"), (0.8, "End")]; + assert_eq!(got.len(), want.len(), "{name}: {v}"); + for ((s, t), (ws, wt)) in got.iter().zip(want) { + assert!((s - ws).abs() < 0.002, "{name}: start {s} vs {ws}"); + assert_eq!(t, wt, "{name}"); + } + // The last chapter runs to the end of the movie. With a late first chapter ffprobe ends it + // early by the empty edit (it takes the end from the chapter track's media duration, + // without the edit); the starts, which players use, are exact. + let end: f64 = ch.last().and_then(|c| c["end_time"].as_str()).and_then(|s| s.parse().ok()).unwrap_or(-1.0); + let dur: f64 = v["format"]["duration"].as_str().and_then(|s| s.parse().ok()).unwrap_or(0.0); + let early = if late_first { 0.25 } else { 0.0 }; + assert!((end + early - dur).abs() < 0.05, "{name}: last chapter ends at {end}, movie lasts {dur}"); + // Our own reader sees the same list in both forms. + let g = open(out.as_slice()).unwrap(); + let ours: Vec = g.read_chapters(out.as_slice()).unwrap().into_iter().map(|c| c.title).collect(); + assert_eq!(ours, ["Opening", "Mitte — ü 日本", "End"], "{name}"); + assert_eq!(g.chpl_chapters.len(), 3, "{name}"); + assert_same_packets(&src, &path); + decode_clean(&path); + } +}