Files
craft-libs/crates/captions/src/vtt.rs
86e59ebc08 craft-captions: ten caption fixes ported from open FilmCraft pull requests (#932)
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks

Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0,
U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a
leading or trailing <br/> was trimmed away.

* craft-captions: TTML/DFXP export drops characters XML 1.0 forbids

Ported from storytold/filmcraft#4467. A control character in cue text made the
exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the
review of that pull request asked for.

* craft-captions: WebVTT cue-text arrow escape is reversible

Ported from storytold/filmcraft#4437. A literal --&gt; in cue text came back as
--> after a write and parse; the writer now adds one amp; level and the reader
removes exactly one.

* craft-captions: a WebVTT speaker name with > no longer spills into the cue text

Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in
place of > inside the voice tag. Also from that pull request's review: a line
break in a name is written as a space, and a name ending in - is kept apart
from the tag's closing >.

* craft-captions: an SRT text line that looks like a timing line survives a round trip

Ported from storytold/filmcraft#4448. The writer marks such a line with a
leading zero-width space and the reader removes it, so the cue is not split.

* craft-captions: back-to-back SCC captions with the same text stay separate cues

Ported from storytold/filmcraft#3808. Every End of Caption now closes the open
cue. MCC's CEA-608 path uses the same decoder.

* craft-captions: an SCC pop-on load may run across the previous caption's erase

Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely
follows another popped on about a second late, because its load had to be one
unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed
between them.

* craft-captions: CEA-708 export starts each caption at its window's top left

Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now
sends DeleteWindows before DefineWindow in place of the ClearWindows after it:
a window made anew is empty with the pen at its top left in any decoder, for
the same number of bytes, so cue timing is unchanged.

* craft-captions: TTML/DFXP export keeps literal < and > text

Ported from storytold/filmcraft#2830. The writer treated every < with a later >
as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its
middle. Only a well-formed tag is markup now; a > inside a quoted attribute
value does not end the tag.

* craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written

Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could
collide with the style ID, and a generated c1 could take the ID of a later cue.
Valid explicit IDs are reserved first; missing, invalid or repeated ones get a
replacement that is in no one's way. IDs are checked as XML NCNames.

* craft-captions: compare the craft-xml and roxmltree views of TTML documents

The TTML reader moved from roxmltree to craft-xml. This dev-only test checks
that both hand it the same elements, attributes and text, by namespace, on
TTML documents and on the writer's output. roxmltree becomes a dev-dependency,
which the cue ID tests use too.

---------

Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
Co-authored-by: SeeRay11 <searay112023@gmail.com>
Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com>
Co-authored-by: JonRC <jonarc06@gmail.com>
Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com>
2026-11-11 18:44:12 -05:00

355 lines
14 KiB
Rust
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! WebVTT (`.vtt`).
//!
//! ```text
//! WEBVTT
//!
//! STYLE
//! ::cue { color: yellow }
//!
//! intro
//! 00:00:01.000 --> 00:00:03.500 line:90% align:center
//! <v Ann>Hello there
//! ```
//!
//! Kept on read: cue identifiers, cue settings (verbatim), a leading `<v Speaker>` voice span (as
//! the speaker), inline markup in the text, and STYLE / REGION / NOTE blocks. A missing `WEBVTT`
//! signature is tolerated. Timestamps may omit hours.
use crate::srt::parse_timing_line;
use crate::{Cue, Document, Error, Result, format_clock};
/// Split a leading `<v Name>` / `<v.class Name>` voice span off cue text.
pub fn split_voice(text: &str) -> (Option<String>, String) {
let t = text.trim_start();
if let Some(rest) = t.strip_prefix("<v")
&& (rest.starts_with(' ') || rest.starts_with('.') || rest.starts_with('\t'))
&& let Some(close) = rest.find('>')
{
let inner = &rest[..close];
// skip classes: `.loud.x Name`
let name = match inner.find([' ', '\t']) {
Some(sp) => inner[sp..].trim(),
None => "",
};
let body = &rest[close + 1..];
// a closing </v> at the very end goes too; one in the middle means several voices: keep all
let body = match body.find("</v>") {
Some(p) if p + 4 == body.trim_end().len() => body[..p].to_string(),
Some(_) => return (None, text.to_string()),
None => body.to_string(),
};
if body.contains("<v ") || body.contains("<v.") || body.contains("<v\t") {
return (None, text.to_string());
}
return ((!name.is_empty()).then(|| name.to_string()), body);
}
(None, text.to_string())
}
/// Undo [`write`]'s cue-text escape, so `A --> B` comes back as `A --> B` and not `A --&gt; B`.
///
/// Only the one substitution the writer makes is reversed: a literal `-->` in cue text would read
/// as a timing line to anything scanning the block, so it is written `--&gt;`. Other entities
/// (`&amp;`, `&lt;`, …) are cue markup and stay verbatim, as `parse` has always kept them.
fn unescape_text(text: &str) -> String {
rewrite_arrows(text, false)
}
/// Apply [`write`]'s cue-text escape. `-->` becomes `--&gt;`, and a literal `--&gt;` (or the same
/// with more `&amp;` in front of `gt;`) gains one `&amp;`, so the escape is reversible.
fn escape_text(text: &str) -> String {
rewrite_arrows(text, true)
}
/// Treat `--&` + `amp;`×k + `gt;` as level k, and `-->` as level -1. Escaping raises the level by
/// one, unescaping lowers it by one; anything else is copied unchanged.
fn rewrite_arrows(text: &str, escape: bool) -> String {
let mut out = String::with_capacity(text.len());
let mut rest = text;
while let Some(c) = rest.chars().next() {
if let Some(after) = rest.strip_prefix("--") {
if escape && after.starts_with('>') {
out.push_str("--&gt;");
rest = &after[1..];
continue;
}
let mut tail = after.strip_prefix('&').unwrap_or("\0");
let mut k = 0usize;
while let Some(t) = tail.strip_prefix("amp;") {
tail = t;
k += 1;
}
if let Some(t) = tail.strip_prefix("gt;") {
if escape {
out.push_str("--&");
out.push_str(&"amp;".repeat(k + 1));
out.push_str("gt;");
} else if k == 0 {
out.push_str("-->");
} else {
out.push_str("--&");
out.push_str(&"amp;".repeat(k - 1));
out.push_str("gt;");
}
rest = t;
continue;
}
}
out.push(c);
rest = &rest[c.len_utf8()..];
}
out
}
pub fn parse(text: &str) -> Result<Document> {
let mut doc = Document::default();
let text = text.strip_prefix('\u{feff}').unwrap_or(text);
let mut lines = text.lines().peekable();
// signature + header text
let has_sig = lines.peek().is_some_and(|&l| l == "WEBVTT" || l.starts_with("WEBVTT ") || l.starts_with("WEBVTT\t"));
if has_sig {
for l in lines.by_ref() {
if l.trim().is_empty() {
break;
}
}
}
// blocks
let mut block: Vec<&str> = Vec::new();
let mut blocks: Vec<Vec<&str>> = Vec::new();
for l in lines {
if l.trim().is_empty() {
if !block.is_empty() {
blocks.push(std::mem::take(&mut block));
}
} else {
block.push(l);
}
}
if !block.is_empty() {
blocks.push(block);
}
let mut seen_cue = false;
for b in blocks {
let first = b[0].trim();
let timing_idx = b.iter().position(|line| parse_timing_line(line).is_some());
if timing_idx.is_none() && (first == "NOTE" || first.starts_with("NOTE ") || first.starts_with("NOTE\t")) {
doc.blocks.push(b.join("\n"));
continue;
}
if !seen_cue && timing_idx.is_none() && (first == "STYLE" || first == "REGION") {
doc.blocks.push(b.join("\n"));
continue;
}
let Some(timing_idx) = timing_idx else {
doc.warnings.push(format!("ignored block \"{first}\""));
continue;
};
let id = (timing_idx > 0).then(|| b[..timing_idx].join("\n").trim().to_string());
let timing = &b[timing_idx];
let Some((start, end, settings)) = parse_timing_line(timing) else {
doc.warnings.push(format!("bad timing line \"{}\"", timing.trim()));
continue;
};
seen_cue = true;
if end <= start {
doc.warnings.push(format!("cue at {} has end before start; skipped", format_clock(start, '.')));
continue;
}
let raw = b[timing_idx + 1..].iter().map(|l| l.trim_end()).collect::<Vec<_>>().join("\n");
let (speaker, text) = split_voice(&unescape_text(&raw));
doc.cues.push(Cue { start, end, text, speaker, id, settings: settings.to_string() });
}
if !has_sig && doc.cues.is_empty() && !text.trim().is_empty() {
return Err(Error::NotFormat("WebVTT"));
}
Ok(doc)
}
pub fn write(doc: &Document) -> String {
let mut out = String::from("WEBVTT\n\n");
for b in &doc.blocks {
let b = b.trim_end();
if !b.is_empty() {
out.push_str(b);
out.push_str("\n\n");
}
}
for c in &doc.cues {
if let Some(id) = c.id.as_deref().map(str::trim).filter(|i| !i.is_empty() && !i.contains("-->")) {
out.push_str(&id.replace(['\r', '\n'], " "));
out.push('\n');
}
out.push_str(&format!("{} --> {}", format_clock(c.start, '.'), format_clock(c.end, '.')));
if !c.settings.trim().is_empty() {
out.push(' ');
out.push_str(c.settings.trim());
}
out.push('\n');
let mut text = String::new();
if let Some(s) = c.speaker.as_deref().filter(|s| !s.is_empty()) {
// `>` would end the voice tag early and spill the rest of the name into the cue text, so it
// is written as the lookalike `›` (U+203A), and a line break as a space. A name ending
// in `-` gets a space before the closing `>`, so the two never read as an arrow.
let name = s.replace('>', "\u{203a}").replace(['\r', '\n'], " ");
text.push_str(&format!("<v {name}{}>", if name.ends_with('-') { " " } else { "" }));
}
text.push_str(&c.text);
for l in text.lines().filter(|l| !l.trim().is_empty()) {
out.push_str(&escape_text(l));
out.push('\n');
}
out.push('\n');
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::TICKS_PER_MS;
use crate::time::Tick;
const SAMPLE: &str = "WEBVTT - with header\nKind: captions\n\nSTYLE\n::cue { color: yellow }\n\nNOTE a comment\nspanning lines\n\nintro\n00:01.000 --> 00:00:03.500 line:90% align:start\n<v.loud Ann Smith>Hello there\nsecond line\n\n00:00:04.000 --> 00:00:05.000\n<i>plain</i> &amp; tags\n";
#[test]
fn bom_without_signature_is_ignored() {
let d = parse("\u{feff}00:00:01.000 --> 00:00:02.000\nHi\n\n00:00:03.000 --> 00:00:04.000\nYo\n").unwrap();
assert_eq!(d.cues.len(), 2);
assert_eq!(d.cues[0].text, "Hi");
}
#[test]
fn style_region_identifier_cue_is_kept() {
for id in ["STYLE", "REGION"] {
let d = parse(&format!("WEBVTT\n\n{id}\n00:00:01.000 --> 00:00:02.000\nHi\n")).unwrap();
assert_eq!(d.cues.len(), 1, "{id}");
assert!(d.blocks.is_empty(), "{id}");
}
}
#[test]
fn parse_sample() {
let d = parse(SAMPLE).unwrap();
assert_eq!(d.blocks, vec!["STYLE\n::cue { color: yellow }".to_string(), "NOTE a comment\nspanning lines".to_string()]);
assert_eq!(d.cues.len(), 2);
let c = &d.cues[0];
assert_eq!(c.id.as_deref(), Some("intro"));
assert_eq!(c.start.0, 1000 * TICKS_PER_MS);
assert_eq!(c.end.0, 3500 * TICKS_PER_MS);
assert_eq!(c.settings, "line:90% align:start");
assert_eq!(c.speaker.as_deref(), Some("Ann Smith"));
assert_eq!(c.text, "Hello there\nsecond line");
assert_eq!(d.cues[1].text, "<i>plain</i> &amp; tags");
let w = write(&d);
assert!(w.starts_with("WEBVTT\n\nSTYLE\n::cue { color: yellow }\n\nNOTE a comment\nspanning lines\n\nintro\n00:00:01.000 --> 00:00:03.500 line:90% align:start\n<v Ann Smith>Hello there\n"));
assert_eq!(parse(&w).unwrap().cues, d.cues);
}
#[test]
fn speaker_with_gt_keeps_cue_text_intact() {
for (name, back_name) in [("A > B", "A \u{203a} B"), ("Tom &amp; Jerry", "Tom &amp; Jerry"), ("A > --", "A \u{203a} --"), ("A\nB", "A B")] {
let mut doc = Document::default();
doc.cues.push(Cue { start: Tick(0), end: Tick(TICKS_PER_MS), text: "Hi".into(), speaker: Some(name.into()), ..Default::default() });
let written = write(&doc);
assert!(!written.contains("--&gt;"), "the voice tag keeps its own `>`: {written}");
let back = parse(&written).unwrap();
assert_eq!(back.cues[0].speaker.as_deref(), Some(back_name));
assert_eq!(back.cues[0].text, "Hi");
}
}
#[test]
fn newline_in_cue_id_round_trips() {
let mut doc = Document::default();
doc.cues.push(Cue {
start: crate::Tick(0),
end: crate::Tick(254_016_000),
text: "x".into(),
speaker: None,
id: Some("first\nsecond".into()),
settings: String::new(),
});
let output = write(&doc);
assert!(output.contains("first second\n00:00:00.000 --> 00:00:00.001\n"), "{output:?}");
assert!(!output.contains("first\nsecond"));
assert_eq!(parse(&output).unwrap().cues[0].id.as_deref(), Some("first second"));
}
#[test]
fn voices() {
assert_eq!(split_voice("<v Bob>hi</v>"), (Some("Bob".into()), "hi".into()));
assert_eq!(split_voice("<v Bob>hi</v> <v Ann>yo</v>"), (None, "<v Bob>hi</v> <v Ann>yo</v>".into()));
assert_eq!(split_voice("<vx>hi"), (None, "<vx>hi".into()));
}
#[test]
fn multiple_voices_with_tab_separator() {
let text = "<v Ann>Hello <v\tBob>there";
assert_eq!(split_voice(text), (None, text.to_string()));
let source = format!("WEBVTT\n\n00:00:01.000 --> 00:00:02.000\n{text}\n");
let doc = parse(&source).unwrap();
assert_eq!(doc.cues[0].speaker, None);
assert_eq!(doc.cues[0].text, text);
assert_eq!(parse(&write(&doc)).unwrap().cues, doc.cues);
let closed = "<v Ann>Hi <v\tBob>x</v>";
assert_eq!(split_voice(closed), (None, closed.to_string()));
}
#[test]
fn note_prefixed_cue_id_is_a_cue() {
let d = parse("WEBVTT\n\nNOTE 1\n00:00:01.000 --> 00:00:02.000\nhello\n").unwrap();
assert_eq!(d.cues.len(), 1);
assert_eq!(d.cues[0].id.as_deref(), Some("NOTE 1"));
assert!(d.blocks.is_empty());
}
#[test]
fn without_signature() {
let d = parse("00:00:01.000 --> 00:00:02.000\nx\n").unwrap();
assert_eq!(d.cues.len(), 1);
assert!(parse("just text").is_err());
}
/// The writer escapes a cue's `-->` as `--&gt;` (it would read as a timing line
/// otherwise), but the parser never decoded it, so `A --> B` came back as `A --&gt; B` on
/// every VTT round trip.
#[test]
fn an_arrow_in_cue_text_survives_a_round_trip() {
let mut d = Document::default();
// A second of ticks: `format_clock` writes milliseconds, so a sub-millisecond end would
// re-parse as `end == start` and the cue would be dropped as backwards.
let ms = TICKS_PER_MS;
d.cues.push(Cue { start: Tick(0), end: Tick(1000 * ms), text: "A --> B".into(), ..Cue::default() });
let w = write(&d);
assert!(w.contains("A --&gt; B"), "the writer still escapes it: {w}");
let back = parse(&w).unwrap();
assert_eq!(back.cues.len(), 1);
assert_eq!(back.cues[0].text, "A --> B", "the parser decodes the writer's escape");
assert_eq!(write(&back), w, "stable from there on");
}
/// A literal `--&gt;` in cue text must not turn into `-->` on a round trip.
#[test]
fn a_literal_escaped_arrow_survives_a_round_trip() {
for text in ["a --&gt; b", "a --> b", "--&amp;gt;", "---&gt;-->", "x --&amp;amp;gt; y"] {
let mut d = Document::default();
d.cues.push(Cue { start: Tick(0), end: Tick(1000 * TICKS_PER_MS), text: text.into(), ..Cue::default() });
let back = parse(&write(&d)).unwrap();
assert_eq!(back.cues[0].text, text);
}
}
/// Only the writer's own `-->` escape is reversed: cue markup keeps its entities verbatim,
/// as `parse` has always done.
#[test]
fn other_entities_stay_verbatim_in_cue_text() {
let d = parse(SAMPLE).unwrap();
assert_eq!(d.cues[1].text, "<i>plain</i> &amp; tags");
let w = write(&d);
assert!(w.contains("&amp; tags"), "not re-escaped on the way out: {w}");
assert_eq!(parse(&w).unwrap().cues[1].text, "<i>plain</i> &amp; tags");
}
}