mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-17 19:14:26 +00:00
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0, U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a leading or trailing <br/> was trimmed away. * craft-captions: TTML/DFXP export drops characters XML 1.0 forbids Ported from storytold/filmcraft#4467. A control character in cue text made the exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the review of that pull request asked for. * craft-captions: WebVTT cue-text arrow escape is reversible Ported from storytold/filmcraft#4437. A literal --> in cue text came back as --> after a write and parse; the writer now adds one amp; level and the reader removes exactly one. * craft-captions: a WebVTT speaker name with > no longer spills into the cue text Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in place of > inside the voice tag. Also from that pull request's review: a line break in a name is written as a space, and a name ending in - is kept apart from the tag's closing >. * craft-captions: an SRT text line that looks like a timing line survives a round trip Ported from storytold/filmcraft#4448. The writer marks such a line with a leading zero-width space and the reader removes it, so the cue is not split. * craft-captions: back-to-back SCC captions with the same text stay separate cues Ported from storytold/filmcraft#3808. Every End of Caption now closes the open cue. MCC's CEA-608 path uses the same decoder. * craft-captions: an SCC pop-on load may run across the previous caption's erase Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely follows another popped on about a second late, because its load had to be one unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed between them. * craft-captions: CEA-708 export starts each caption at its window's top left Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now sends DeleteWindows before DefineWindow in place of the ClearWindows after it: a window made anew is empty with the pen at its top left in any decoder, for the same number of bytes, so cue timing is unchanged. * craft-captions: TTML/DFXP export keeps literal < and > text Ported from storytold/filmcraft#2830. The writer treated every < with a later > as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its middle. Only a well-formed tag is markup now; a > inside a quoted attribute value does not end the tag. * craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could collide with the style ID, and a generated c1 could take the ID of a later cue. Valid explicit IDs are reserved first; missing, invalid or repeated ones get a replacement that is in no one's way. IDs are checked as XML NCNames. * craft-captions: compare the craft-xml and roxmltree views of TTML documents The TTML reader moved from roxmltree to craft-xml. This dev-only test checks that both hand it the same elements, attributes and text, by namespace, on TTML documents and on the writer's output. roxmltree becomes a dev-dependency, which the cue ID tests use too. --------- Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com> Co-authored-by: SeeRay11 <searay112023@gmail.com> Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com> Co-authored-by: JonRC <jonarc06@gmail.com> Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com>
355 lines
14 KiB
Rust
355 lines
14 KiB
Rust
//! WebVTT (`.vtt`).
|
||
//!
|
||
//! ```text
|
||
//! WEBVTT
|
||
//!
|
||
//! STYLE
|
||
//! ::cue { color: yellow }
|
||
//!
|
||
//! intro
|
||
//! 00:00:01.000 --> 00:00:03.500 line:90% align:center
|
||
//! <v Ann>Hello there
|
||
//! ```
|
||
//!
|
||
//! Kept on read: cue identifiers, cue settings (verbatim), a leading `<v Speaker>` voice span (as
|
||
//! the speaker), inline markup in the text, and STYLE / REGION / NOTE blocks. A missing `WEBVTT`
|
||
//! signature is tolerated. Timestamps may omit hours.
|
||
|
||
use crate::srt::parse_timing_line;
|
||
use crate::{Cue, Document, Error, Result, format_clock};
|
||
|
||
/// Split a leading `<v Name>` / `<v.class Name>` voice span off cue text.
|
||
pub fn split_voice(text: &str) -> (Option<String>, String) {
|
||
let t = text.trim_start();
|
||
if let Some(rest) = t.strip_prefix("<v")
|
||
&& (rest.starts_with(' ') || rest.starts_with('.') || rest.starts_with('\t'))
|
||
&& let Some(close) = rest.find('>')
|
||
{
|
||
let inner = &rest[..close];
|
||
// skip classes: `.loud.x Name`
|
||
let name = match inner.find([' ', '\t']) {
|
||
Some(sp) => inner[sp..].trim(),
|
||
None => "",
|
||
};
|
||
let body = &rest[close + 1..];
|
||
// a closing </v> at the very end goes too; one in the middle means several voices: keep all
|
||
let body = match body.find("</v>") {
|
||
Some(p) if p + 4 == body.trim_end().len() => body[..p].to_string(),
|
||
Some(_) => return (None, text.to_string()),
|
||
None => body.to_string(),
|
||
};
|
||
if body.contains("<v ") || body.contains("<v.") || body.contains("<v\t") {
|
||
return (None, text.to_string());
|
||
}
|
||
return ((!name.is_empty()).then(|| name.to_string()), body);
|
||
}
|
||
(None, text.to_string())
|
||
}
|
||
|
||
/// Undo [`write`]'s cue-text escape, so `A --> B` comes back as `A --> B` and not `A --> B`.
|
||
///
|
||
/// Only the one substitution the writer makes is reversed: a literal `-->` in cue text would read
|
||
/// as a timing line to anything scanning the block, so it is written `-->`. Other entities
|
||
/// (`&`, `<`, …) are cue markup and stay verbatim, as `parse` has always kept them.
|
||
fn unescape_text(text: &str) -> String {
|
||
rewrite_arrows(text, false)
|
||
}
|
||
|
||
/// Apply [`write`]'s cue-text escape. `-->` becomes `-->`, and a literal `-->` (or the same
|
||
/// with more `&` in front of `gt;`) gains one `&`, so the escape is reversible.
|
||
fn escape_text(text: &str) -> String {
|
||
rewrite_arrows(text, true)
|
||
}
|
||
|
||
/// Treat `--&` + `amp;`×k + `gt;` as level k, and `-->` as level -1. Escaping raises the level by
|
||
/// one, unescaping lowers it by one; anything else is copied unchanged.
|
||
fn rewrite_arrows(text: &str, escape: bool) -> String {
|
||
let mut out = String::with_capacity(text.len());
|
||
let mut rest = text;
|
||
while let Some(c) = rest.chars().next() {
|
||
if let Some(after) = rest.strip_prefix("--") {
|
||
if escape && after.starts_with('>') {
|
||
out.push_str("-->");
|
||
rest = &after[1..];
|
||
continue;
|
||
}
|
||
let mut tail = after.strip_prefix('&').unwrap_or("\0");
|
||
let mut k = 0usize;
|
||
while let Some(t) = tail.strip_prefix("amp;") {
|
||
tail = t;
|
||
k += 1;
|
||
}
|
||
if let Some(t) = tail.strip_prefix("gt;") {
|
||
if escape {
|
||
out.push_str("--&");
|
||
out.push_str(&"amp;".repeat(k + 1));
|
||
out.push_str("gt;");
|
||
} else if k == 0 {
|
||
out.push_str("-->");
|
||
} else {
|
||
out.push_str("--&");
|
||
out.push_str(&"amp;".repeat(k - 1));
|
||
out.push_str("gt;");
|
||
}
|
||
rest = t;
|
||
continue;
|
||
}
|
||
}
|
||
out.push(c);
|
||
rest = &rest[c.len_utf8()..];
|
||
}
|
||
out
|
||
}
|
||
|
||
pub fn parse(text: &str) -> Result<Document> {
|
||
let mut doc = Document::default();
|
||
let text = text.strip_prefix('\u{feff}').unwrap_or(text);
|
||
let mut lines = text.lines().peekable();
|
||
// signature + header text
|
||
let has_sig = lines.peek().is_some_and(|&l| l == "WEBVTT" || l.starts_with("WEBVTT ") || l.starts_with("WEBVTT\t"));
|
||
if has_sig {
|
||
for l in lines.by_ref() {
|
||
if l.trim().is_empty() {
|
||
break;
|
||
}
|
||
}
|
||
}
|
||
// blocks
|
||
let mut block: Vec<&str> = Vec::new();
|
||
let mut blocks: Vec<Vec<&str>> = Vec::new();
|
||
for l in lines {
|
||
if l.trim().is_empty() {
|
||
if !block.is_empty() {
|
||
blocks.push(std::mem::take(&mut block));
|
||
}
|
||
} else {
|
||
block.push(l);
|
||
}
|
||
}
|
||
if !block.is_empty() {
|
||
blocks.push(block);
|
||
}
|
||
let mut seen_cue = false;
|
||
for b in blocks {
|
||
let first = b[0].trim();
|
||
let timing_idx = b.iter().position(|line| parse_timing_line(line).is_some());
|
||
if timing_idx.is_none() && (first == "NOTE" || first.starts_with("NOTE ") || first.starts_with("NOTE\t")) {
|
||
doc.blocks.push(b.join("\n"));
|
||
continue;
|
||
}
|
||
if !seen_cue && timing_idx.is_none() && (first == "STYLE" || first == "REGION") {
|
||
doc.blocks.push(b.join("\n"));
|
||
continue;
|
||
}
|
||
let Some(timing_idx) = timing_idx else {
|
||
doc.warnings.push(format!("ignored block \"{first}\""));
|
||
continue;
|
||
};
|
||
let id = (timing_idx > 0).then(|| b[..timing_idx].join("\n").trim().to_string());
|
||
let timing = &b[timing_idx];
|
||
let Some((start, end, settings)) = parse_timing_line(timing) else {
|
||
doc.warnings.push(format!("bad timing line \"{}\"", timing.trim()));
|
||
continue;
|
||
};
|
||
seen_cue = true;
|
||
if end <= start {
|
||
doc.warnings.push(format!("cue at {} has end before start; skipped", format_clock(start, '.')));
|
||
continue;
|
||
}
|
||
let raw = b[timing_idx + 1..].iter().map(|l| l.trim_end()).collect::<Vec<_>>().join("\n");
|
||
let (speaker, text) = split_voice(&unescape_text(&raw));
|
||
doc.cues.push(Cue { start, end, text, speaker, id, settings: settings.to_string() });
|
||
}
|
||
if !has_sig && doc.cues.is_empty() && !text.trim().is_empty() {
|
||
return Err(Error::NotFormat("WebVTT"));
|
||
}
|
||
Ok(doc)
|
||
}
|
||
|
||
pub fn write(doc: &Document) -> String {
|
||
let mut out = String::from("WEBVTT\n\n");
|
||
for b in &doc.blocks {
|
||
let b = b.trim_end();
|
||
if !b.is_empty() {
|
||
out.push_str(b);
|
||
out.push_str("\n\n");
|
||
}
|
||
}
|
||
for c in &doc.cues {
|
||
if let Some(id) = c.id.as_deref().map(str::trim).filter(|i| !i.is_empty() && !i.contains("-->")) {
|
||
out.push_str(&id.replace(['\r', '\n'], " "));
|
||
out.push('\n');
|
||
}
|
||
out.push_str(&format!("{} --> {}", format_clock(c.start, '.'), format_clock(c.end, '.')));
|
||
if !c.settings.trim().is_empty() {
|
||
out.push(' ');
|
||
out.push_str(c.settings.trim());
|
||
}
|
||
out.push('\n');
|
||
let mut text = String::new();
|
||
if let Some(s) = c.speaker.as_deref().filter(|s| !s.is_empty()) {
|
||
// `>` would end the voice tag early and spill the rest of the name into the cue text, so it
|
||
// is written as the lookalike `›` (U+203A), and a line break as a space. A name ending
|
||
// in `-` gets a space before the closing `>`, so the two never read as an arrow.
|
||
let name = s.replace('>', "\u{203a}").replace(['\r', '\n'], " ");
|
||
text.push_str(&format!("<v {name}{}>", if name.ends_with('-') { " " } else { "" }));
|
||
}
|
||
text.push_str(&c.text);
|
||
for l in text.lines().filter(|l| !l.trim().is_empty()) {
|
||
out.push_str(&escape_text(l));
|
||
out.push('\n');
|
||
}
|
||
out.push('\n');
|
||
}
|
||
out
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
use crate::TICKS_PER_MS;
|
||
use crate::time::Tick;
|
||
|
||
const SAMPLE: &str = "WEBVTT - with header\nKind: captions\n\nSTYLE\n::cue { color: yellow }\n\nNOTE a comment\nspanning lines\n\nintro\n00:01.000 --> 00:00:03.500 line:90% align:start\n<v.loud Ann Smith>Hello there\nsecond line\n\n00:00:04.000 --> 00:00:05.000\n<i>plain</i> & tags\n";
|
||
|
||
#[test]
|
||
fn bom_without_signature_is_ignored() {
|
||
let d = parse("\u{feff}00:00:01.000 --> 00:00:02.000\nHi\n\n00:00:03.000 --> 00:00:04.000\nYo\n").unwrap();
|
||
assert_eq!(d.cues.len(), 2);
|
||
assert_eq!(d.cues[0].text, "Hi");
|
||
}
|
||
|
||
#[test]
|
||
fn style_region_identifier_cue_is_kept() {
|
||
for id in ["STYLE", "REGION"] {
|
||
let d = parse(&format!("WEBVTT\n\n{id}\n00:00:01.000 --> 00:00:02.000\nHi\n")).unwrap();
|
||
assert_eq!(d.cues.len(), 1, "{id}");
|
||
assert!(d.blocks.is_empty(), "{id}");
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn parse_sample() {
|
||
let d = parse(SAMPLE).unwrap();
|
||
assert_eq!(d.blocks, vec!["STYLE\n::cue { color: yellow }".to_string(), "NOTE a comment\nspanning lines".to_string()]);
|
||
assert_eq!(d.cues.len(), 2);
|
||
let c = &d.cues[0];
|
||
assert_eq!(c.id.as_deref(), Some("intro"));
|
||
assert_eq!(c.start.0, 1000 * TICKS_PER_MS);
|
||
assert_eq!(c.end.0, 3500 * TICKS_PER_MS);
|
||
assert_eq!(c.settings, "line:90% align:start");
|
||
assert_eq!(c.speaker.as_deref(), Some("Ann Smith"));
|
||
assert_eq!(c.text, "Hello there\nsecond line");
|
||
assert_eq!(d.cues[1].text, "<i>plain</i> & tags");
|
||
let w = write(&d);
|
||
assert!(w.starts_with("WEBVTT\n\nSTYLE\n::cue { color: yellow }\n\nNOTE a comment\nspanning lines\n\nintro\n00:00:01.000 --> 00:00:03.500 line:90% align:start\n<v Ann Smith>Hello there\n"));
|
||
assert_eq!(parse(&w).unwrap().cues, d.cues);
|
||
}
|
||
|
||
#[test]
|
||
fn speaker_with_gt_keeps_cue_text_intact() {
|
||
for (name, back_name) in [("A > B", "A \u{203a} B"), ("Tom & Jerry", "Tom & Jerry"), ("A > --", "A \u{203a} --"), ("A\nB", "A B")] {
|
||
let mut doc = Document::default();
|
||
doc.cues.push(Cue { start: Tick(0), end: Tick(TICKS_PER_MS), text: "Hi".into(), speaker: Some(name.into()), ..Default::default() });
|
||
let written = write(&doc);
|
||
assert!(!written.contains("-->"), "the voice tag keeps its own `>`: {written}");
|
||
let back = parse(&written).unwrap();
|
||
assert_eq!(back.cues[0].speaker.as_deref(), Some(back_name));
|
||
assert_eq!(back.cues[0].text, "Hi");
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn newline_in_cue_id_round_trips() {
|
||
let mut doc = Document::default();
|
||
doc.cues.push(Cue {
|
||
start: crate::Tick(0),
|
||
end: crate::Tick(254_016_000),
|
||
text: "x".into(),
|
||
speaker: None,
|
||
id: Some("first\nsecond".into()),
|
||
settings: String::new(),
|
||
});
|
||
|
||
let output = write(&doc);
|
||
assert!(output.contains("first second\n00:00:00.000 --> 00:00:00.001\n"), "{output:?}");
|
||
assert!(!output.contains("first\nsecond"));
|
||
assert_eq!(parse(&output).unwrap().cues[0].id.as_deref(), Some("first second"));
|
||
}
|
||
|
||
#[test]
|
||
fn voices() {
|
||
assert_eq!(split_voice("<v Bob>hi</v>"), (Some("Bob".into()), "hi".into()));
|
||
assert_eq!(split_voice("<v Bob>hi</v> <v Ann>yo</v>"), (None, "<v Bob>hi</v> <v Ann>yo</v>".into()));
|
||
assert_eq!(split_voice("<vx>hi"), (None, "<vx>hi".into()));
|
||
}
|
||
|
||
#[test]
|
||
fn multiple_voices_with_tab_separator() {
|
||
let text = "<v Ann>Hello <v\tBob>there";
|
||
assert_eq!(split_voice(text), (None, text.to_string()));
|
||
let source = format!("WEBVTT\n\n00:00:01.000 --> 00:00:02.000\n{text}\n");
|
||
let doc = parse(&source).unwrap();
|
||
assert_eq!(doc.cues[0].speaker, None);
|
||
assert_eq!(doc.cues[0].text, text);
|
||
assert_eq!(parse(&write(&doc)).unwrap().cues, doc.cues);
|
||
let closed = "<v Ann>Hi <v\tBob>x</v>";
|
||
assert_eq!(split_voice(closed), (None, closed.to_string()));
|
||
}
|
||
|
||
#[test]
|
||
fn note_prefixed_cue_id_is_a_cue() {
|
||
let d = parse("WEBVTT\n\nNOTE 1\n00:00:01.000 --> 00:00:02.000\nhello\n").unwrap();
|
||
assert_eq!(d.cues.len(), 1);
|
||
assert_eq!(d.cues[0].id.as_deref(), Some("NOTE 1"));
|
||
assert!(d.blocks.is_empty());
|
||
}
|
||
|
||
#[test]
|
||
fn without_signature() {
|
||
let d = parse("00:00:01.000 --> 00:00:02.000\nx\n").unwrap();
|
||
assert_eq!(d.cues.len(), 1);
|
||
assert!(parse("just text").is_err());
|
||
}
|
||
|
||
/// The writer escapes a cue's `-->` as `-->` (it would read as a timing line
|
||
/// otherwise), but the parser never decoded it, so `A --> B` came back as `A --> B` on
|
||
/// every VTT round trip.
|
||
#[test]
|
||
fn an_arrow_in_cue_text_survives_a_round_trip() {
|
||
let mut d = Document::default();
|
||
// A second of ticks: `format_clock` writes milliseconds, so a sub-millisecond end would
|
||
// re-parse as `end == start` and the cue would be dropped as backwards.
|
||
let ms = TICKS_PER_MS;
|
||
d.cues.push(Cue { start: Tick(0), end: Tick(1000 * ms), text: "A --> B".into(), ..Cue::default() });
|
||
let w = write(&d);
|
||
assert!(w.contains("A --> B"), "the writer still escapes it: {w}");
|
||
let back = parse(&w).unwrap();
|
||
assert_eq!(back.cues.len(), 1);
|
||
assert_eq!(back.cues[0].text, "A --> B", "the parser decodes the writer's escape");
|
||
assert_eq!(write(&back), w, "stable from there on");
|
||
}
|
||
|
||
/// A literal `-->` in cue text must not turn into `-->` on a round trip.
|
||
#[test]
|
||
fn a_literal_escaped_arrow_survives_a_round_trip() {
|
||
for text in ["a --> b", "a --> b", "--&gt;", "--->-->", "x --&amp;gt; y"] {
|
||
let mut d = Document::default();
|
||
d.cues.push(Cue { start: Tick(0), end: Tick(1000 * TICKS_PER_MS), text: text.into(), ..Cue::default() });
|
||
let back = parse(&write(&d)).unwrap();
|
||
assert_eq!(back.cues[0].text, text);
|
||
}
|
||
}
|
||
|
||
/// Only the writer's own `-->` escape is reversed: cue markup keeps its entities verbatim,
|
||
/// as `parse` has always done.
|
||
#[test]
|
||
fn other_entities_stay_verbatim_in_cue_text() {
|
||
let d = parse(SAMPLE).unwrap();
|
||
assert_eq!(d.cues[1].text, "<i>plain</i> & tags");
|
||
let w = write(&d);
|
||
assert!(w.contains("& tags"), "not re-escaped on the way out: {w}");
|
||
assert_eq!(parse(&w).unwrap().cues[1].text, "<i>plain</i> & tags");
|
||
}
|
||
}
|