mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-12 10:03:51 +00:00
craft-html: HTML documents on craft-flow (feature flow), from WordCraft (#1050)
* Add craft-wordmodel: WordCraft's document model without its editing code
WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.
Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.
* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs
The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.
* craft-mime: detect, for bytes that name no character set
is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.
* craft-wordmodel: keep the document operations (edit.rs) with the model
The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.
base64 0.23, as WordCraft and craft-offcrypto use.
* craft-codepage: one table per legacy code page, for craft-rtf, craft-dxf and craft-mime
craft-rtf and craft-dxf each carried tables for DOS 437, 737, 850, 852 and
857 (and craft-rtf one for 866). They are now one set in craft-codepage,
which takes craft-rtf's codepage module (Windows and Mac tables, OEM glyphs,
font charsets) and craft-dxf's DOS tables. craft_rtf::codepage re-exports it
unchanged; craft-dxf keeps its $DWGCODEPAGE names and Decoder and looks the
DOS tables up there.
Every byte of every code page either crate decodes is unchanged: the
public decoders of both crates were dumped over all 256 bytes of each page
(and the East Asian pairs) before and after, and the dumps are identical.
craft-mime now reads the DOS pages by their IANA names (IBM437, cp850,
ibm-852, 857...), which it read as UTF-8 before.
* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath
The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.
mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.
* craft-math: read MathLive's and other spellings of LaTeX math commands
\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).
* craft-math: equaSHion's equation corpus as test data
corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (7a6ef6b9), unmodified.
* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README
The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.
* Add craft-flow: the flow model the text formats map to, on craft-wordmodel
WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.
* Add craft-markdown: Markdown read and written, on craft-flow
WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.
* WIP: craft-latex sources from WordCraft (unmodified)
* Add craft-latex: LaTeX documents read and written, on craft-flow
WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.
* craft-math: latex_text, LaTeX math and the linear format converted as text
WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.
* craft-math: latex_text's group finders are public, for craft-latex's length parser
* craft-latex: math through craft_math::latex_text; label timing test by ratio
The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.
* Cargo.lock after the merge
* WIP: WordCraft's HTML document reader and writer, unmodified (crates/formats/src/html at 07f97737)
* craft-html: HTML documents on craft-flow, behind a flow feature
WordCraft's crates/formats/src/html (07f97737) as craft_html::doc: the
tag-soup builder, CSS subset, ruby and MathML reading, page decoding and
the writer, with its 44 tests. It reads tokens from this crate's
tokenizer through a small adapter (decoded text and attributes, names in
lower case) instead of its own lexer, and its character references are
this crate's. Lexer::set_pos is new: the ruby reader hands a token back.
The HTML parts of WordCraft's MathML and cross-format tests come along as
tests/doc.rs, the linear-time check as a ratio.
* craft-math: operator names take their LaTeX scripts directly again (restores #1019)
#1019 (22161d2d) wrote \sin^2\theta as \sin ^{2}{\theta } so it reads back as
the same tree. The squash merge of #1038, whose branch predated it, put
math_latex.rs and the corpus test back as they were before. This applies
#1019's change again, unchanged, with its test.
* Cargo.lock: craft-html's flow dependencies
---------
Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
This commit is contained in:
Dylan M. Taylor
co-authored by
shuff57
shuff57
parent
f3fe9de956
commit
db616480e2
Generated
+6
@@ -1290,6 +1290,12 @@ dependencies = [
|
||||
name = "craft-html"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"craft-codepage",
|
||||
"craft-flow",
|
||||
"craft-math",
|
||||
"craft-mime",
|
||||
"craft-wordmodel",
|
||||
"encoding_rs",
|
||||
"proptest",
|
||||
]
|
||||
|
||||
|
||||
@@ -144,7 +144,7 @@ craft-<name> = { git = "https://github.com/storytold/craft-libs", rev = "<commit
|
||||
heic-rs reads the container and HEVC pictures, and this crate converts to 16-bit RGB to match
|
||||
libheif (VUI range and matrix, chroma upsampling, `imir`, ICC on grid tiles). Also orientation,
|
||||
alpha, EXIF/XMP and thumbnails. It never panics, and apps put it behind a non-default feature.
|
||||
- [`craft-html`](crates/html/README.md): the bottom layer of reading and writing HTML: a tolerant tokenizer (text, start tags, end tags; elements to drop or to read as raw text are options), character references with the legacy names browsers accept without `;`, escaping, and the character set a page declares. No tree, no CSS, no rendering.
|
||||
- [`craft-html`](crates/html/README.md): the bottom layer of reading and writing HTML: a tolerant tokenizer (text, start tags, end tags; elements to drop or to read as raw text are options), character references with the legacy names browsers accept without `;`, escaping, and the character set a page declares. With the `flow` feature, HTML documents read into and written from the craft-flow model.
|
||||
- [`craft-i18n`](crates/i18n/README.md): translations over the embedded craft-i18n catalogs:
|
||||
lookup with fallback (`pt-br` → `pt` → English), plurals, language names, the system locale and
|
||||
a pseudo locale. The catalogs stay a fully optional build input; apps keep `tl!` and their
|
||||
|
||||
@@ -10,5 +10,21 @@ repository.workspace = true
|
||||
[lints]
|
||||
workspace = true
|
||||
|
||||
[features]
|
||||
# HTML documents: read into and written from the craft-flow model and a craft-wordmodel Document.
|
||||
flow = ["dep:craft-codepage", "dep:craft-flow", "dep:craft-math", "dep:craft-mime", "dep:craft-wordmodel", "dep:encoding_rs"]
|
||||
|
||||
[dependencies]
|
||||
craft-codepage = { workspace = true, optional = true }
|
||||
craft-flow = { workspace = true, optional = true }
|
||||
craft-math = { workspace = true, optional = true }
|
||||
craft-mime = { workspace = true, features = ["detect"], optional = true }
|
||||
craft-wordmodel = { workspace = true, optional = true }
|
||||
encoding_rs = { workspace = true, optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
proptest = { workspace = true }
|
||||
|
||||
[[test]]
|
||||
name = "doc"
|
||||
required-features = ["flow"]
|
||||
|
||||
+17
-1
@@ -52,12 +52,28 @@ linear.
|
||||
`escape` writes `& < > " '` as references; `escape_dq` leaves `'`, for content and attribute
|
||||
values in double quotes.
|
||||
|
||||
## HTML documents (feature `flow`)
|
||||
|
||||
`craft_html::doc` reads a page or a pasted fragment into the [craft-flow](../flow/README.md)
|
||||
model and a craft-wordmodel `Document`, and writes one as clean HTML. The reader is a tolerant
|
||||
tag-soup builder on this crate's tokenizer (a stack of open elements with HTML's implicit closes),
|
||||
with a CSS subset from `style` attributes and `<style>` rules with simple selectors in cascade
|
||||
order, tables with spans, lists, pictures (`data:` URIs, others through the caller's
|
||||
`ImageLoader`), ruby and MathML equations ([craft-math](../math/README.md)). `import_page` opens a
|
||||
web page as Word does (Times New Roman 12 pt, HTML's default spacing); `import` reads a fragment in
|
||||
the document's own formatting. `decode` / `decode_as` turn a page's bytes into text (BOM, declared
|
||||
charset, else detection). The writer gives semantic HTML with inline CSS, pictures as `data:`
|
||||
URIs, equations as MathML and ruby as `<ruby>`.
|
||||
|
||||
The feature brings in craft-flow, craft-wordmodel, craft-math, craft-mime and craft-codepage; a
|
||||
reader that only needs tokens (mail, clipboard tables) leaves it off.
|
||||
|
||||
## Not here
|
||||
|
||||
- The full HTML5 named reference table (about 2,200 names): only the names above.
|
||||
- Decoding bytes to text. `declared_charset` gives the label a page names; a decoder for it is
|
||||
the caller's (`craft-mime::charset` for the WHATWG labels).
|
||||
- Tree construction, implied end tags, CSS.
|
||||
- A DOM, and CSS beyond the subset the document reader applies.
|
||||
|
||||
## Provenance
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
//! HTML documents (feature `flow`): read into and written from the craft-flow model and a
|
||||
//! craft-wordmodel `Document`.
|
||||
//!
|
||||
//! Import is a tolerant tag-soup reader (no DOM): a tokenizer plus a stack of open elements with
|
||||
//! HTML's implicit closes for `p`, `li`, `tr`, `td`/`th` and headings. It understands `p`, `div`,
|
||||
//! `h1`–`h6`, `b`/`strong`, `i`/`em`, `u`, `s`/`strike`/`del`, `sup`/`sub`, `code`/`kbd`, `a`
|
||||
//! (links and `id`/`name` anchors), `br`, `hr`, `ul`/`ol`/`li` (nested), `table`/`tr`/`td`/`th`
|
||||
//! (colspan/rowspan), `img` (`data:` URIs, other sources through a caller's [`ImageLoader`]), `blockquote`, `pre`, `font`, `ruby`
|
||||
//! (`rb`/`rt`/`rp`: furigana and other phonetic guides, #1681), and the CSS
|
||||
//! properties `color`, `background-color`, `font-weight`, `font-style`, `font-size`,
|
||||
//! `font-family`, `text-decoration`, `vertical-align`, `text-align`, `white-space` and page
|
||||
//! breaks, and on paragraphs `margin`, `text-indent` and `line-height`. CSS comes from `style`
|
||||
//! attributes and from `style` elements' rules with simple selectors (`p`, `.note`, `p.note`),
|
||||
//! applied in cascade order (specificity, then source order, then the `style` attribute).
|
||||
//! Scripts are skipped; nesting is capped.
|
||||
//!
|
||||
//! Export writes clean semantic HTML with inline CSS, pictures as `data:` URIs and ruby as
|
||||
//! `<ruby>`.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use craft_wordmodel::props::LineSpacing;
|
||||
use craft_wordmodel::props::VAlign;
|
||||
use craft_wordmodel::{Align, Document};
|
||||
|
||||
use craft_flow::{
|
||||
self as model, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, ListInfo, Meta, Para, Spacing, base64_encode, data_uri, make_img, mime_of,
|
||||
parse_color,
|
||||
};
|
||||
|
||||
mod builder;
|
||||
mod css;
|
||||
mod entity;
|
||||
mod export;
|
||||
mod import;
|
||||
mod lexer;
|
||||
mod math_ruby;
|
||||
|
||||
pub use builder::ImageLoader;
|
||||
pub use css::css_len_pt;
|
||||
pub use entity::{decode, decode_as, decode_entity, unescape};
|
||||
pub use export::{esc, export, export_flow, export_fragment, inlines_html};
|
||||
#[cfg(test)]
|
||||
use import::parse_in;
|
||||
pub use import::{PAGE_FONT, import, import_on, import_page, import_with, parse, parse_with};
|
||||
|
||||
#[cfg(test)]
|
||||
use css::css_unescape;
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,167 @@
|
||||
use super::builder::decls;
|
||||
use super::import::PAGE_FONT;
|
||||
use super::{Align, Fmt, model, parse_color};
|
||||
|
||||
/// A CSS length (or font-size keyword) in points; `None` when it is not one.
|
||||
pub fn css_len_pt(v: &str) -> Option<f32> {
|
||||
let v = v.trim().to_ascii_lowercase();
|
||||
let num = |s: &str| s.trim().parse::<f32>().ok().filter(|x| x.is_finite());
|
||||
if let Some(x) = v.strip_suffix("pt") {
|
||||
num(x)
|
||||
} else if let Some(x) = v.strip_suffix("px") {
|
||||
num(x).map(|p| p * 0.75)
|
||||
} else if let Some(x) = v.strip_suffix("rem").or_else(|| v.strip_suffix("em")) {
|
||||
num(x).map(|p| p * 12.0)
|
||||
} else if let Some(x) = v.strip_suffix("in") {
|
||||
num(x).map(|p| p * 72.0)
|
||||
} else if let Some(x) = v.strip_suffix("cm") {
|
||||
num(x).map(|p| p * 72.0 / 2.54)
|
||||
} else if let Some(x) = v.strip_suffix("mm") {
|
||||
num(x).map(|p| p * 72.0 / 25.4)
|
||||
} else if let Some(x) = v.strip_suffix('%') {
|
||||
num(x).map(|p| p * 0.12)
|
||||
} else {
|
||||
match v.as_str() {
|
||||
"xx-small" => Some(7.0),
|
||||
"x-small" => Some(7.5),
|
||||
"small" | "smaller" => Some(10.0),
|
||||
"medium" => Some(12.0),
|
||||
"large" | "larger" => Some(13.5),
|
||||
"x-large" => Some(18.0),
|
||||
"xx-large" => Some(24.0),
|
||||
"xxx-large" => Some(36.0),
|
||||
_ => num(&v).map(|p| p * 0.75),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn strip_important(v: &str) -> &str {
|
||||
let s = v.trim();
|
||||
if let Some(pos) = s.rfind('!') {
|
||||
let after = s[pos + 1..].trim();
|
||||
if after.eq_ignore_ascii_case("important") {
|
||||
return s[..pos].trim();
|
||||
}
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
/// Decode CSS escapes (CSS Syntax §4.3.7): `\` and one to six hex digits, with one whitespace
|
||||
/// after them, is that code point; `\` and any other character is that character. CR LF, CR and
|
||||
/// form feed count as one newline, as CSS reads them (§3.3).
|
||||
pub(super) fn css_unescape(s: &str) -> String {
|
||||
let mut o = String::with_capacity(s.len());
|
||||
let mut it = s.chars().peekable();
|
||||
// Takes a newline (CR LF as one) or other whitespace off the front, if there is one.
|
||||
let skip_space = |it: &mut std::iter::Peekable<std::str::Chars>| match it.peek() {
|
||||
Some('\r') => {
|
||||
it.next();
|
||||
it.next_if_eq(&'\n');
|
||||
}
|
||||
Some(' ' | '\t' | '\n' | '\u{C}') => {
|
||||
it.next();
|
||||
}
|
||||
_ => {}
|
||||
};
|
||||
while let Some(c) = it.next() {
|
||||
if c != '\\' {
|
||||
o.push(c);
|
||||
continue;
|
||||
}
|
||||
let mut hex = String::new();
|
||||
while hex.len() < 6 && it.peek().is_some_and(char::is_ascii_hexdigit) {
|
||||
hex.extend(it.next());
|
||||
}
|
||||
if hex.is_empty() {
|
||||
// Before a newline it continues the line; at the end it is dropped.
|
||||
match it.peek() {
|
||||
Some('\r' | '\n' | '\u{C}') => skip_space(&mut it),
|
||||
Some(_) => o.extend(it.next()),
|
||||
None => {}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
skip_space(&mut it);
|
||||
let cp = u32::from_str_radix(&hex, 16).unwrap_or(0);
|
||||
o.push(char::from_u32(cp).filter(|c| *c != '\0').unwrap_or('\u{FFFD}'));
|
||||
}
|
||||
o
|
||||
}
|
||||
|
||||
/// Apply a `style` attribute to formatting / block properties.
|
||||
pub(super) fn apply_style(
|
||||
css: &str,
|
||||
f: &mut Fmt,
|
||||
align: &mut Option<Align>,
|
||||
preserve: &mut bool,
|
||||
page_break: &mut bool,
|
||||
page_break_after: &mut bool,
|
||||
) {
|
||||
for (k, v) in decls(css) {
|
||||
let lv = v.to_ascii_lowercase();
|
||||
match k.as_str() {
|
||||
"font-weight" => f.bold = lv.starts_with("bold") || lv.parse::<u32>().is_ok_and(|w| w >= 600),
|
||||
"font-style" => f.italic = lv == "italic" || lv == "oblique",
|
||||
"text-decoration" | "text-decoration-line" => {
|
||||
if lv.contains("none") {
|
||||
f.underline = false;
|
||||
f.strike = false;
|
||||
}
|
||||
if lv.contains("underline") {
|
||||
f.underline = true;
|
||||
}
|
||||
if lv.contains("line-through") {
|
||||
f.strike = true;
|
||||
}
|
||||
}
|
||||
"color" => {
|
||||
if let Some(c) = parse_color(v) {
|
||||
f.color = Some(c);
|
||||
}
|
||||
}
|
||||
"background-color" | "background" => {
|
||||
if let Some(c) = v.split_whitespace().find_map(parse_color) {
|
||||
f.background = Some(c);
|
||||
}
|
||||
}
|
||||
"font-size" => {
|
||||
if let Some(p) = css_len_pt(v).filter(|p| *p >= 1.0 && *p <= 1638.0) {
|
||||
f.size = Some(p);
|
||||
}
|
||||
}
|
||||
"font-family" => {
|
||||
let fam = css_unescape(v.split(',').next().unwrap_or("").trim().trim_matches(['"', '\'']));
|
||||
if model::is_mono(&fam) || fam.eq_ignore_ascii_case("monospace") {
|
||||
f.code = true;
|
||||
} else if !fam.is_empty() && !matches!(fam.to_ascii_lowercase().as_str(), "serif" | "sans-serif" | "inherit" | "initial") {
|
||||
f.font = model::kept_font(&fam).or(f.font.take());
|
||||
} else if f.font.is_some() {
|
||||
// A generic family replaces a face the text has (a page's Times New Roman);
|
||||
// text without one keeps the document's font.
|
||||
match fam.to_ascii_lowercase().as_str() {
|
||||
"serif" => f.font = Some(PAGE_FONT.into()),
|
||||
"sans-serif" => f.font = Some("Arial".into()),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
"vertical-align" => {
|
||||
f.sup = lv == "super";
|
||||
f.sub = lv == "sub";
|
||||
}
|
||||
"text-align" => {
|
||||
*align = match lv.as_str() {
|
||||
"center" => Some(Align::Center),
|
||||
"right" | "end" => Some(Align::Right),
|
||||
"justify" => Some(Align::Justify),
|
||||
"left" | "start" => Some(Align::Left),
|
||||
_ => *align,
|
||||
}
|
||||
}
|
||||
"white-space" => *preserve = lv.starts_with("pre"),
|
||||
"page-break-before" | "break-before" => *page_break = lv == "always" || lv == "page",
|
||||
"page-break-after" | "break-after" => *page_break_after = lv == "always" || lv == "page",
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
//! Character references (from the crate's core) and decoding a page's bytes to text.
|
||||
|
||||
/// The text of an entity body (`amp`, `#38`, `#x26`).
|
||||
pub fn decode_entity(name: &str) -> Option<String> {
|
||||
crate::decode_entity(name).map(String::from)
|
||||
}
|
||||
|
||||
pub use crate::unescape;
|
||||
|
||||
/// Decode HTML bytes: BOM, then a `charset` in the first 2 KB, else UTF-8 with a legacy fallback.
|
||||
pub fn decode(b: &[u8]) -> String {
|
||||
decode_as(b, None)
|
||||
}
|
||||
|
||||
/// [`decode`] in the `charset` its container states (a MIME part's, a content type's), when it
|
||||
/// states one. Single-byte Windows code pages (`windows-1250`…, `iso-8859-1`, `us-ascii` as 1252)
|
||||
/// are decoded as such, other labels by their WHATWG decoders.
|
||||
pub fn decode_as(b: &[u8], charset: Option<&str>) -> String {
|
||||
if b.starts_with(&[0xFF, 0xFE]) || b.starts_with(&[0xFE, 0xFF]) || b.starts_with(&[0xEF, 0xBB, 0xBF]) {
|
||||
return craft_mime::detect::decode(b);
|
||||
}
|
||||
let named = charset.map(str::to_ascii_lowercase).or_else(|| crate::declared_charset(b));
|
||||
let code_page = match named.as_deref().map(str::trim) {
|
||||
Some("iso-8859-1" | "latin1" | "us-ascii") => Some(1252),
|
||||
Some(cs) => cs
|
||||
.strip_prefix("windows-")
|
||||
.or_else(|| cs.strip_prefix("cp"))
|
||||
.and_then(|n| n.parse::<u16>().ok())
|
||||
.filter(|cp| craft_codepage::SINGLE_BYTE_CODE_PAGES.contains(cp)),
|
||||
None => None,
|
||||
};
|
||||
// Other labels (ISO 8859, KOI8, Shift_JIS, GBK, Big5, EUC-KR…) by their WHATWG decoders; a
|
||||
// UTF-16 one without a BOM means UTF-8, as in browsers.
|
||||
let other = named
|
||||
.as_deref()
|
||||
.and_then(|l| encoding_rs::Encoding::for_label(l.trim().as_bytes()))
|
||||
.filter(|e| ![encoding_rs::UTF_8, encoding_rs::UTF_16LE, encoding_rs::UTF_16BE].contains(e));
|
||||
match (code_page, other) {
|
||||
(Some(cp), _) => b.iter().map(|&c| craft_codepage::windows_byte(cp, c)).collect(),
|
||||
(None, Some(enc)) => enc.decode_without_bom_handling(b).0.into_owned(),
|
||||
(None, None) => match std::str::from_utf8(b) {
|
||||
Ok(s) => s.to_string(),
|
||||
Err(_) => craft_mime::detect::decode(b),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,393 @@
|
||||
use super::{Align, Document, FBlock, FTable, Flow, Fmt, Inline, Kind, Para, VAlign, base64_encode, mime_of, model};
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// Export
|
||||
|
||||
/// Escape text for HTML content and attribute values.
|
||||
pub fn esc(s: &str) -> String {
|
||||
let mut o = String::with_capacity(s.len() + 8);
|
||||
for c in s.chars() {
|
||||
match c {
|
||||
'&' => o.push_str("&"),
|
||||
'<' => o.push_str("<"),
|
||||
'>' => o.push_str(">"),
|
||||
'"' => o.push_str("""),
|
||||
'\u{000C}' | '\u{000E}' => {}
|
||||
_ => o.push(c),
|
||||
}
|
||||
}
|
||||
o
|
||||
}
|
||||
|
||||
/// A font name as a quoted CSS string. Quotes, `\` and control characters would end the string,
|
||||
/// and `;` and `,` the declaration or the family list for a reader that splits on them, so all of
|
||||
/// these are written as hex escapes ([`css_unescape`] reads them back).
|
||||
fn css_font_name(name: &str) -> String {
|
||||
let mut o = String::with_capacity(name.len() + 2);
|
||||
o.push('\'');
|
||||
for c in name.chars() {
|
||||
match c {
|
||||
'\0' => o.push('\u{FFFD}'),
|
||||
'\'' | '"' | '\\' | ';' | ',' => o.push_str(&format!("\\{:x} ", u32::from(c))),
|
||||
c if c.is_control() => o.push_str(&format!("\\{:x} ", u32::from(c))),
|
||||
c => o.push(c),
|
||||
}
|
||||
}
|
||||
o.push('\'');
|
||||
o
|
||||
}
|
||||
|
||||
fn align_css(a: Option<Align>) -> Option<&'static str> {
|
||||
match a? {
|
||||
Align::Center => Some("text-align:center"),
|
||||
Align::Right => Some("text-align:right"),
|
||||
Align::Justify | Align::Distribute => Some("text-align:justify"),
|
||||
Align::Left => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn text_html(t: &str, at_start: bool) -> String {
|
||||
let mut o = String::new();
|
||||
let mut prev_space = at_start;
|
||||
for c in t.chars() {
|
||||
match c {
|
||||
'\n' => {
|
||||
o.push_str("<br>");
|
||||
prev_space = true;
|
||||
continue;
|
||||
}
|
||||
'\t' => o.push_str("<span style=\"white-space:pre\">\t</span>"),
|
||||
' ' if prev_space => o.push_str(" "),
|
||||
_ => o.push_str(&esc(&c.to_string())),
|
||||
}
|
||||
prev_space = c == ' ';
|
||||
}
|
||||
o
|
||||
}
|
||||
|
||||
/// The tags that carry `f`'s character formatting: what opens them, and what closes them
|
||||
/// (innermost last, so the closes are written in reverse).
|
||||
fn fmt_tags(f: &Fmt) -> (String, Vec<&'static str>) {
|
||||
let mut open = String::new();
|
||||
let mut close: Vec<&str> = Vec::new();
|
||||
// A script or other non-web target is written as plain text (`is_safe_href`).
|
||||
if let Some(l) = f.link.as_deref().filter(|l| craft_wordmodel::link::is_safe_href(l)) {
|
||||
open.push_str(&format!("<a href=\"{}\">", esc(l)));
|
||||
close.push("</a>");
|
||||
}
|
||||
let mut tag = |o: &str, c: &'static str| {
|
||||
open.push_str(o);
|
||||
close.push(c);
|
||||
};
|
||||
if f.bold {
|
||||
tag("<strong>", "</strong>");
|
||||
}
|
||||
if f.italic {
|
||||
tag("<em>", "</em>");
|
||||
}
|
||||
if f.underline {
|
||||
tag("<u>", "</u>");
|
||||
}
|
||||
if f.strike {
|
||||
tag("<s>", "</s>");
|
||||
}
|
||||
if f.sup {
|
||||
tag("<sup>", "</sup>");
|
||||
} else if f.sub {
|
||||
tag("<sub>", "</sub>");
|
||||
}
|
||||
if f.code {
|
||||
tag("<code>", "</code>");
|
||||
}
|
||||
let mut css = Vec::new();
|
||||
if let Some(c) = f.color {
|
||||
css.push(format!("color:#{}", c.hex()));
|
||||
}
|
||||
if let Some(c) = f.background {
|
||||
css.push(format!("background-color:#{}", c.hex()));
|
||||
}
|
||||
if let Some(s) = f.size {
|
||||
css.push(format!("font-size:{}pt", trim_num(s)));
|
||||
}
|
||||
if let Some(fam) = &f.font {
|
||||
css.push(format!("font-family:{}", esc(&css_font_name(fam))));
|
||||
}
|
||||
if !css.is_empty() {
|
||||
open.push_str(&format!("<span style=\"{}\">", css.join(";")));
|
||||
close.push("</span>");
|
||||
}
|
||||
(open, close)
|
||||
}
|
||||
|
||||
/// Inline spans → HTML.
|
||||
pub fn inlines_html(inlines: &[Inline]) -> String {
|
||||
let mut o = String::new();
|
||||
for i in inlines.iter() {
|
||||
match i {
|
||||
Inline::Text(t, f) => {
|
||||
let (open, close) = fmt_tags(f);
|
||||
let at_start = o.is_empty();
|
||||
o.push_str(&open);
|
||||
o.push_str(&text_html(t, at_start));
|
||||
for c in close.iter().rev() {
|
||||
o.push_str(c);
|
||||
}
|
||||
}
|
||||
// A phonetic guide: the base text with its reading in `<rt>` (#1681). HTML ruby has
|
||||
// no alignment or size, so the guide's own attributes stay in the document model.
|
||||
Inline::Ruby { base, ruby, fmt } => {
|
||||
let (open, close) = fmt_tags(fmt);
|
||||
let at_start = o.is_empty();
|
||||
o.push_str(&open);
|
||||
o.push_str(&format!("<ruby>{}<rt>{}</rt></ruby>", text_html(base, at_start), text_html(ruby, false)));
|
||||
for c in close.iter().rev() {
|
||||
o.push_str(c);
|
||||
}
|
||||
}
|
||||
Inline::Image(img) => {
|
||||
o.push_str(&format!(
|
||||
"<img src=\"data:{};base64,{}\" alt=\"{}\" width=\"{}\" height=\"{}\">",
|
||||
mime_of(&img.ext),
|
||||
base64_encode(&img.data),
|
||||
esc(&img.alt),
|
||||
(img.w / 0.75).round(),
|
||||
(img.h / 0.75).round()
|
||||
));
|
||||
}
|
||||
Inline::Anchor(a) => o.push_str(&format!("<a id=\"{}\"></a>", esc(a))),
|
||||
Inline::Note { .. }
|
||||
| Inline::Comment { .. }
|
||||
| Inline::CommentEnd(_)
|
||||
| Inline::Shape { .. }
|
||||
| Inline::Group { .. }
|
||||
| Inline::FieldStart(_)
|
||||
| Inline::FieldEnd
|
||||
| Inline::AnchorEnd(_) => {}
|
||||
// ASK and SET show nothing (see `model::shows_nothing`).
|
||||
Inline::Field { instr, .. } if model::shows_nothing(instr) => {}
|
||||
Inline::Field { result, fmt, .. } | Inline::Control { text: result, fmt, .. } => {
|
||||
let (open, close) = fmt_tags(fmt);
|
||||
let at_start = o.is_empty();
|
||||
o.push_str(&open);
|
||||
o.push_str(&text_html(result, at_start));
|
||||
for c in close.iter().rev() {
|
||||
o.push_str(c);
|
||||
}
|
||||
}
|
||||
// No picture to show: the browser shows the alt text in its place.
|
||||
Inline::Figure(alt) | Inline::Chart(craft_flow::Chart { alt, .. }) => o.push_str(&format!("<img alt=\"{}\">", esc(alt))),
|
||||
// MathML, which browsers, Word and LibreOffice show as an equation (#816).
|
||||
Inline::Equation { linear, display } => o.push_str(&craft_math::mathml::linear_to_mathml(linear, *display)),
|
||||
}
|
||||
}
|
||||
o
|
||||
}
|
||||
|
||||
fn trim_num(v: f32) -> String {
|
||||
let s = format!("{:.2}", v);
|
||||
s.trim_end_matches('0').trim_end_matches('.').to_string()
|
||||
}
|
||||
|
||||
fn para_attrs(p: &Para) -> String {
|
||||
let mut css: Vec<&str> = Vec::new();
|
||||
if let Some(a) = align_css(p.align) {
|
||||
css.push(a);
|
||||
}
|
||||
if p.page_break {
|
||||
css.push("break-before:page;page-break-before:always");
|
||||
}
|
||||
if css.is_empty() { String::new() } else { format!(" style=\"{}\"", css.join(";")) }
|
||||
}
|
||||
|
||||
fn para_html(p: &Para) -> String {
|
||||
let body = inlines_html(&p.inlines);
|
||||
let attrs = para_attrs(p);
|
||||
match p.kind {
|
||||
Kind::Heading(n) => format!("<h{n}{attrs}>{body}</h{n}>", n = n.clamp(1, 6)),
|
||||
Kind::Title => format!("<h1{attrs}>{body}</h1>"),
|
||||
Kind::Rule => "<hr>".to_string(),
|
||||
_ if body.is_empty() => format!("<p{attrs}> </p>"),
|
||||
_ => format!("<p{attrs}>{body}</p>"),
|
||||
}
|
||||
}
|
||||
|
||||
fn table_html(t: &FTable, out: &mut String, depth: usize) {
|
||||
out.push_str("<table style=\"border-collapse:collapse\">\n");
|
||||
for row in &t.rows {
|
||||
out.push_str("<tr>");
|
||||
for c in row {
|
||||
if c.covered {
|
||||
continue;
|
||||
}
|
||||
let tag = if c.header { "th" } else { "td" };
|
||||
let mut attrs = String::new();
|
||||
if c.colspan > 1 {
|
||||
attrs.push_str(&format!(" colspan=\"{}\"", c.colspan));
|
||||
}
|
||||
if c.rowspan > 1 {
|
||||
attrs.push_str(&format!(" rowspan=\"{}\"", c.rowspan));
|
||||
}
|
||||
let v_css = match c.valign {
|
||||
Some(VAlign::Center) => "vertical-align:middle",
|
||||
Some(VAlign::Bottom) => "vertical-align:bottom",
|
||||
_ => "vertical-align:top",
|
||||
};
|
||||
let mut css = vec!["padding:4px 6px".to_string(), v_css.to_string()];
|
||||
if !t.borderless {
|
||||
css.insert(0, "border:1px solid #999".to_string());
|
||||
}
|
||||
if let Some(s) = c.shading {
|
||||
css.push(format!("background-color:#{}", s.hex()));
|
||||
}
|
||||
let single = match c.blocks.as_slice() {
|
||||
[FBlock::Para(p)] if p.kind == Kind::Normal && p.list.is_none() && !p.page_break => Some(p),
|
||||
_ => None,
|
||||
};
|
||||
if let Some(a) = single.and_then(|p| align_css(p.align)) {
|
||||
css.push(a.to_string());
|
||||
}
|
||||
out.push_str(&format!("<{tag}{attrs} style=\"{}\">", css.join(";")));
|
||||
match single {
|
||||
Some(p) => out.push_str(&inlines_html(&p.inlines)),
|
||||
None => {
|
||||
out.push('\n');
|
||||
blocks_html(&c.blocks, out, depth + 1);
|
||||
}
|
||||
}
|
||||
out.push_str(&format!("</{tag}>"));
|
||||
}
|
||||
out.push_str("</tr>\n");
|
||||
}
|
||||
out.push_str("</table>\n");
|
||||
}
|
||||
|
||||
fn blocks_html(blocks: &[FBlock], out: &mut String, depth: usize) {
|
||||
let mut i = 0;
|
||||
while let Some(b) = blocks.get(i) {
|
||||
match b {
|
||||
FBlock::Table(t) => {
|
||||
if depth < model::MAX_DEPTH {
|
||||
table_html(t, out, depth);
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
FBlock::Para(p) if p.list.is_some() => {
|
||||
// Nested lists from consecutive list paragraphs.
|
||||
let mut stack: Vec<(bool, bool)> = Vec::new(); // (ordered, li open)
|
||||
while let Some(FBlock::Para(q)) = blocks.get(i) {
|
||||
let Some(li) = q.list else { break };
|
||||
let lv = li.level.min(8) as usize;
|
||||
while stack.len() > lv + 1 {
|
||||
close_list(&mut stack, out);
|
||||
}
|
||||
if stack.len() == lv + 1 && stack.last().is_some_and(|s| s.0 != li.ordered) {
|
||||
close_list(&mut stack, out);
|
||||
}
|
||||
while stack.len() < lv + 1 {
|
||||
if let Some(top) = stack.last_mut()
|
||||
&& !top.1
|
||||
{
|
||||
out.push_str("<li>");
|
||||
top.1 = true;
|
||||
}
|
||||
out.push_str(if li.ordered { "<ol>" } else { "<ul>" });
|
||||
stack.push((li.ordered, false));
|
||||
}
|
||||
if let Some(top) = stack.last_mut() {
|
||||
if top.1 {
|
||||
out.push_str("</li>\n");
|
||||
}
|
||||
top.1 = true;
|
||||
}
|
||||
let body = inlines_html(&q.inlines);
|
||||
let attrs = para_attrs(q);
|
||||
out.push_str(&format!("<li{attrs}>{body}"));
|
||||
i += 1;
|
||||
}
|
||||
while !stack.is_empty() {
|
||||
close_list(&mut stack, out);
|
||||
}
|
||||
out.push('\n');
|
||||
}
|
||||
FBlock::Para(p) if p.kind == Kind::Code => {
|
||||
out.push_str("<pre><code>");
|
||||
let mut first = true;
|
||||
while let Some(FBlock::Para(q)) = blocks.get(i) {
|
||||
if q.kind != Kind::Code || q.list.is_some() {
|
||||
break;
|
||||
}
|
||||
if !first {
|
||||
out.push('\n');
|
||||
}
|
||||
first = false;
|
||||
out.push_str(&esc(&q.text()));
|
||||
i += 1;
|
||||
}
|
||||
out.push_str("</code></pre>\n");
|
||||
}
|
||||
FBlock::Para(p) if p.kind == Kind::Quote => {
|
||||
out.push_str("<blockquote>\n");
|
||||
while let Some(FBlock::Para(q)) = blocks.get(i) {
|
||||
if q.kind != Kind::Quote || q.list.is_some() {
|
||||
break;
|
||||
}
|
||||
out.push_str(¶_html(&Para { kind: Kind::Normal, ..q.clone() }));
|
||||
out.push('\n');
|
||||
i += 1;
|
||||
}
|
||||
out.push_str("</blockquote>\n");
|
||||
}
|
||||
FBlock::Para(p) => {
|
||||
out.push_str(¶_html(p));
|
||||
out.push('\n');
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn close_list(stack: &mut Vec<(bool, bool)>, out: &mut String) {
|
||||
if let Some((ordered, li_open)) = stack.pop() {
|
||||
if li_open {
|
||||
out.push_str("</li>");
|
||||
}
|
||||
out.push_str(if ordered { "</ol>" } else { "</ul>" });
|
||||
}
|
||||
}
|
||||
|
||||
/// A complete HTML document.
|
||||
pub fn export(doc: &Document) -> String {
|
||||
let flow = model::from_doc(doc);
|
||||
export_flow(&flow, doc.styles.default_chr.lang.as_deref().unwrap_or("en"))
|
||||
}
|
||||
|
||||
/// `doc`'s content as an HTML fragment (no head or style sheet): what a copy puts on the system
|
||||
/// clipboard beside its text, equations as MathML (#816).
|
||||
pub fn export_fragment(doc: &Document) -> String {
|
||||
let flow = model::from_doc(doc);
|
||||
let mut out = String::from("<meta charset=\"utf-8\">");
|
||||
blocks_html(&flow.blocks, &mut out, 0);
|
||||
out
|
||||
}
|
||||
|
||||
pub fn export_flow(flow: &Flow, lang: &str) -> String {
|
||||
let m = &flow.meta;
|
||||
let mut out = String::new();
|
||||
out.push_str("<!DOCTYPE html>\n");
|
||||
out.push_str(&format!("<html lang=\"{}\">\n<head>\n<meta charset=\"utf-8\">\n", esc(lang)));
|
||||
out.push_str("<meta name=\"generator\" content=\"WordCraft\">\n");
|
||||
out.push_str(&format!("<title>{}</title>\n", esc(&m.title)));
|
||||
for (k, v) in [("author", &m.author), ("description", &m.description), ("keywords", &m.keywords), ("subject", &m.subject)] {
|
||||
if !v.is_empty() {
|
||||
out.push_str(&format!("<meta name=\"{k}\" content=\"{}\">\n", esc(v)));
|
||||
}
|
||||
}
|
||||
out.push_str(
|
||||
"<style>\nbody{font-family:Aptos,Calibri,'Segoe UI',Arial,sans-serif;font-size:12pt;line-height:1.15;max-width:48em;margin:2em auto;padding:0 1em}\n\
|
||||
table{border-collapse:collapse;margin:0.5em 0}\nblockquote{margin:0.5em 2em;font-style:italic;color:#404040}\n\
|
||||
pre{background:#f5f5f5;padding:0.5em;font-family:'Courier New',monospace;font-size:10pt}\n</style>\n</head>\n<body>\n",
|
||||
);
|
||||
blocks_html(&flow.blocks, &mut out, 0);
|
||||
out.push_str("</body>\n</html>\n");
|
||||
out
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
use super::builder::{Builder, ImageLoader, parse_sheet};
|
||||
use super::lexer::{Lexer, Tok};
|
||||
use super::math_ruby::{read_math, read_ruby};
|
||||
use super::{Document, Flow, Inline, model};
|
||||
|
||||
/// Parse HTML text into the flow model.
|
||||
pub fn parse(s: &str) -> Flow {
|
||||
parse_with(s, &|_| None)
|
||||
}
|
||||
|
||||
/// [`parse`], loading pictures that aren't `data:` URIs through `images`.
|
||||
pub fn parse_with(s: &str, images: ImageLoader) -> Flow {
|
||||
parse_in(s, images, false)
|
||||
}
|
||||
|
||||
pub(super) fn parse_in(s: &str, images: ImageLoader, page: bool) -> Flow {
|
||||
// HTML normalizes input newlines before tokenization, not after decoding character references.
|
||||
let normalized = if s.contains('\r') {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
let mut chunks = s.split('\r');
|
||||
out.push_str(chunks.next().unwrap_or_default());
|
||||
for chunk in chunks {
|
||||
out.push('\n');
|
||||
out.push_str(chunk.strip_prefix('\n').unwrap_or(chunk));
|
||||
}
|
||||
std::borrow::Cow::Owned(out)
|
||||
} else {
|
||||
std::borrow::Cow::Borrowed(s)
|
||||
};
|
||||
// The lexer's offsets (a style element's text is cut out by them) are into the normalized text.
|
||||
let s: &str = &normalized;
|
||||
let mut lx = Lexer::new(s);
|
||||
let mut b = Builder::new(images, page);
|
||||
let mut guard = 0usize;
|
||||
while let Some(t) = lx.next() {
|
||||
guard += 1;
|
||||
if guard > 50_000_000 {
|
||||
break;
|
||||
}
|
||||
match t {
|
||||
Tok::Text(t) => b.text(&t),
|
||||
Tok::Start { name, attrs, self_close } => {
|
||||
b.close_head_for_start(&name);
|
||||
if craft_math::mathml::local(&name) == "math" && !self_close {
|
||||
let el = read_math(&mut lx, name, attrs);
|
||||
if !b.in_head()
|
||||
&& let Some((linear, display)) = craft_math::mathml::read(&el)
|
||||
{
|
||||
b.start_para().inlines.push(Inline::Equation { linear, display });
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if name == "ruby" && !self_close {
|
||||
let r = read_ruby(&mut lx);
|
||||
if !b.in_head() {
|
||||
b.push_ruby(&r.base, &r.ruby);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if matches!(name.as_str(), "script" | "style" | "noscript" | "template" | "textarea") && !self_close {
|
||||
let from = lx.pos();
|
||||
let to = lx.skip_past_end_tag(&name);
|
||||
if name == "style" {
|
||||
parse_sheet(s.get(from..to).unwrap_or(""), &mut b.sheet);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
b.start(&name, &attrs, self_close);
|
||||
}
|
||||
Tok::End(name) => b.end(&name),
|
||||
}
|
||||
}
|
||||
b.finish()
|
||||
}
|
||||
|
||||
/// Collapse white space inside ruby markup as body text does (HTML's white space only: an
|
||||
/// ideographic space is text), without the space the element's indentation leaves at the edges.
|
||||
pub(super) fn collapse_ws(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
for c in s.chars() {
|
||||
if matches!(c, ' ' | '\t' | '\n' | '\r' | '\u{000C}') {
|
||||
if !out.ends_with(' ') {
|
||||
out.push(' ');
|
||||
}
|
||||
} else {
|
||||
out.push(c);
|
||||
}
|
||||
}
|
||||
out.trim_matches(' ').to_string()
|
||||
}
|
||||
|
||||
/// Parse HTML into a document.
|
||||
pub fn import(s: &str) -> Document {
|
||||
model::begin_import();
|
||||
model::to_doc(&parse(s))
|
||||
}
|
||||
|
||||
/// [`import`], loading pictures that aren't `data:` URIs through `images`.
|
||||
pub fn import_with(s: &str, images: ImageLoader) -> Document {
|
||||
model::begin_import();
|
||||
model::to_doc(&parse_with(s, images))
|
||||
}
|
||||
|
||||
/// A web page opened as a document, as Word opens one (a file, or a document's HTML
|
||||
/// `w:altChunk`): its text is Times New Roman 12 pt on single lines, headings bold and black at
|
||||
/// the sizes browsers give them, and a `p` or heading whose CSS leaves a margin out (or writes it
|
||||
/// invalidly) gets HTML's automatic spacing there; a margin a span states replaces the one its
|
||||
/// paragraph states (both as Word's import does). [`import`] reads a pasted fragment, which takes
|
||||
/// the document's own formatting instead (but its CSS still applies).
|
||||
pub fn import_page(s: &str, images: ImageLoader) -> Document {
|
||||
model::begin_import();
|
||||
model::to_doc(&parse_in(s, images, true))
|
||||
}
|
||||
|
||||
/// [`import_page`] onto `page` (its size and margins), which tables size to: HTML gives no page
|
||||
/// size, so the caller picks it (the region's paper, #1381).
|
||||
pub fn import_on(s: &str, images: ImageLoader, page: &craft_wordmodel::SectionProps) -> Document {
|
||||
model::begin_import();
|
||||
model::to_doc_in(&parse_in(s, images, true), page.clone())
|
||||
}
|
||||
|
||||
/// The font of a web page's text when its CSS names none.
|
||||
pub const PAGE_FONT: &str = "Times New Roman";
|
||||
/// Heading sizes on a web page, `h1`–`h6`, points.
|
||||
pub(super) const PAGE_HEADING_SIZES: [f32; 6] = [24.0, 18.0, 13.5, 12.0, 10.0, 7.5];
|
||||
@@ -0,0 +1,50 @@
|
||||
//! The crate's tokenizer ([`crate::Lexer`]) as the document reader takes its tokens: text and
|
||||
//! attribute values decoded, element and attribute names in lower case.
|
||||
|
||||
use crate::{Token, unescape, unescape_attr};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub(super) enum Tok {
|
||||
Text(String),
|
||||
Start { name: String, attrs: Vec<(String, String)>, self_close: bool },
|
||||
End(String),
|
||||
}
|
||||
|
||||
pub(super) struct Lexer<'a> {
|
||||
inner: crate::Lexer<'a, 'static>,
|
||||
}
|
||||
|
||||
impl<'a> Lexer<'a> {
|
||||
pub(super) fn new(s: &'a str) -> Lexer<'a> {
|
||||
Lexer { inner: crate::Lexer::new(s) }
|
||||
}
|
||||
|
||||
/// Where the next token starts.
|
||||
pub(super) fn pos(&self) -> usize {
|
||||
self.inner.pos()
|
||||
}
|
||||
|
||||
/// Read again from `pos` (an earlier [`Lexer::pos`]).
|
||||
pub(super) fn set_pos(&mut self, pos: usize) {
|
||||
self.inner.set_pos(pos);
|
||||
}
|
||||
|
||||
/// Skip past the end tag of raw text element `tag`; returns where that end tag starts (the
|
||||
/// end of the input when there is none).
|
||||
pub(super) fn skip_past_end_tag(&mut self, tag: &str) -> usize {
|
||||
let from = self.inner.pos();
|
||||
from + self.inner.raw_content(tag).len()
|
||||
}
|
||||
|
||||
pub(super) fn next(&mut self) -> Option<Tok> {
|
||||
Some(match self.inner.next()? {
|
||||
Token::Text(t) => Tok::Text(unescape(t)),
|
||||
Token::Start(tag) => Tok::Start {
|
||||
attrs: tag.attrs.iter().map(|(n, v)| (n.to_ascii_lowercase(), unescape_attr(v))).collect(),
|
||||
name: tag.name,
|
||||
self_close: tag.self_close,
|
||||
},
|
||||
Token::End(name) => Tok::End(name),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
use super::builder::block_kind;
|
||||
use super::lexer::{Lexer, Tok};
|
||||
|
||||
/// A `<math>` element (#816): its tokens up to the matching end tag, as a tree. Nesting and size
|
||||
/// are bounded; an unclosed element ends with the input.
|
||||
pub(super) fn read_math(lx: &mut Lexer, name: String, attrs: Vec<(String, String)>) -> craft_math::mathml::El {
|
||||
use craft_math::mathml::{El, Item, MAX_NODES, local};
|
||||
let mut stack: Vec<El> = vec![El { name, attrs, children: Vec::new() }];
|
||||
let mut count = 0usize;
|
||||
let close = |stack: &mut Vec<El>| {
|
||||
if stack.len() > 1
|
||||
&& let Some(done) = stack.pop()
|
||||
&& let Some(parent) = stack.last_mut()
|
||||
{
|
||||
parent.children.push(Item::El(done));
|
||||
}
|
||||
};
|
||||
while let Some(t) = lx.next() {
|
||||
count += 1;
|
||||
if count > MAX_NODES * 4 {
|
||||
// Over the cap: drop the rest of this element so its tokens aren't read as HTML text.
|
||||
skip_math(lx, t);
|
||||
break;
|
||||
}
|
||||
match t {
|
||||
Tok::Text(t) => {
|
||||
if let Some(top) = stack.last_mut()
|
||||
&& !t.is_empty()
|
||||
{
|
||||
top.children.push(Item::Text(t));
|
||||
}
|
||||
}
|
||||
Tok::Start { name, attrs, self_close } => {
|
||||
let el = El { name, attrs, children: Vec::new() };
|
||||
if self_close || stack.len() > craft_wordmodel::math::MAX_DEPTH {
|
||||
if let Some(top) = stack.last_mut() {
|
||||
top.children.push(Item::El(el));
|
||||
}
|
||||
} else {
|
||||
stack.push(el);
|
||||
}
|
||||
}
|
||||
Tok::End(name) => {
|
||||
let name = local(&name).to_string();
|
||||
// Close up to the matching open element; a stray end tag is ignored.
|
||||
if let Some(k) = stack.iter().rposition(|e| local(&e.name) == name) {
|
||||
if k == 0 {
|
||||
break;
|
||||
}
|
||||
while stack.len() > k {
|
||||
close(&mut stack);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
while stack.len() > 1 {
|
||||
close(&mut stack);
|
||||
}
|
||||
stack.pop().unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Skip tokens (starting with `first`) up to and including the `</math>` that closes the element
|
||||
/// being read; nested `<math>` elements are counted. An unclosed element ends with the input.
|
||||
fn skip_math(lx: &mut Lexer, first: Tok) {
|
||||
use craft_math::mathml::local;
|
||||
let mut depth = 0usize;
|
||||
let mut t = Some(first);
|
||||
while let Some(tok) = t {
|
||||
match tok {
|
||||
Tok::Start { name, self_close: false, .. } if local(&name) == "math" => depth = depth.saturating_add(1),
|
||||
Tok::End(name) if local(&name) == "math" => {
|
||||
if depth == 0 {
|
||||
return;
|
||||
}
|
||||
depth = depth.saturating_sub(1);
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
t = lx.next();
|
||||
}
|
||||
}
|
||||
|
||||
/// A `<ruby>` element's base text and reading (#1681).
|
||||
#[derive(Default)]
|
||||
pub(super) struct RubyText {
|
||||
pub(super) base: String,
|
||||
pub(super) ruby: String,
|
||||
}
|
||||
|
||||
/// Which part of a `<ruby>` element the text being read belongs to.
|
||||
#[derive(Clone, Copy, PartialEq, Eq)]
|
||||
enum RubyPart {
|
||||
/// The text being annotated: an `<rb>` child, or the text outside `<rt>`/`<rp>`.
|
||||
Base,
|
||||
/// The reading: an `<rt>` child.
|
||||
Ruby,
|
||||
/// `<rp>` fallback parentheses, which are not text: browsers show them only where ruby isn't
|
||||
/// supported.
|
||||
Fallback,
|
||||
}
|
||||
|
||||
/// A block-level tag ends an open `<ruby>` implicitly, as it does in a browser.
|
||||
fn ends_ruby(t: &Tok) -> bool {
|
||||
match t {
|
||||
Tok::Start { name, .. } | Tok::End(name) => block_kind(name).is_some(),
|
||||
Tok::Text(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Read a `<ruby>` element's content: its base text from `<rb>` children (or the text outside
|
||||
/// them), its reading from `<rt>` children, and `<rp>` fallback parentheses dropped. Tolerant,
|
||||
/// like the rest of the reader: tags inside are ignored, an unclosed element ends at the next
|
||||
/// block-level tag (handed back to the caller) or with the input, and the tokens read are capped.
|
||||
pub(super) fn read_ruby(lx: &mut Lexer) -> RubyText {
|
||||
/// Most tokens one `<ruby>` element is read from: a missing end tag must not scan the rest of
|
||||
/// a hostile file here.
|
||||
const MAX_TOKENS: usize = 4096;
|
||||
let mut out = RubyText::default();
|
||||
let mut part = RubyPart::Base;
|
||||
let mut nested = 0usize;
|
||||
let mut count = 0usize;
|
||||
while count < MAX_TOKENS {
|
||||
count += 1;
|
||||
let start = lx.pos();
|
||||
let Some(t) = lx.next() else { break };
|
||||
if nested == 0 && ends_ruby(&t) {
|
||||
// The boundary ends the element: give the token back to the main reader.
|
||||
lx.set_pos(start);
|
||||
break;
|
||||
}
|
||||
match t {
|
||||
Tok::Text(t) => match part {
|
||||
RubyPart::Base => out.base.push_str(&t),
|
||||
RubyPart::Ruby => out.ruby.push_str(&t),
|
||||
RubyPart::Fallback => {}
|
||||
},
|
||||
// The self-closing slash is ignored on HTML elements (`<rt/>` opens an `rt`), as in a
|
||||
// browser's parser.
|
||||
Tok::Start { name, self_close, .. } => match name.as_str() {
|
||||
"rb" => part = RubyPart::Base,
|
||||
"rt" => part = RubyPart::Ruby,
|
||||
"rp" => part = RubyPart::Fallback,
|
||||
"ruby" if !self_close => nested += 1,
|
||||
_ => {}
|
||||
},
|
||||
Tok::End(name) => match name.as_str() {
|
||||
"rb" | "rt" | "rp" => part = RubyPart::Base,
|
||||
"ruby" if nested > 0 => nested -= 1,
|
||||
"ruby" => break,
|
||||
_ => {}
|
||||
},
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
@@ -0,0 +1,769 @@
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn numeric_entity_rejects_sign() {
|
||||
assert_eq!(unescape("a&#+65;b"), "a&#+65;b");
|
||||
assert_eq!(unescape("a&#x+41;b"), "a&#x+41;b");
|
||||
assert_eq!(unescape("aAAb"), "aAAb");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn important_declarations_win_the_cascade() {
|
||||
let color = |html: &str| fmt_of(paras(&parse(html))[0]).color;
|
||||
let red = Some(craft_wordmodel::Rgb(255, 0, 0));
|
||||
let blue = Some(craft_wordmodel::Rgb(0, 0, 255));
|
||||
assert_eq!(color("<style>p { color: red }</style><p style='color: blue'>x</p>"), blue);
|
||||
assert_eq!(color("<style>p { color: red !important }</style><p style='color: blue'>x</p>"), red);
|
||||
assert_eq!(color("<style>p { color: red !important }</style><p style='color: blue !important'>x</p>"), blue);
|
||||
assert_eq!(color("<p style='color: red !important; color: blue'>x</p>"), red);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_declared_charset_decodes_the_page() {
|
||||
// ISO 8859-2 (Central European): 0xA9 Š, 0xB9 š; named by the caller or the page.
|
||||
assert_eq!(decode_as(&[0xA9, b' ', 0xB9], Some("iso-8859-2")), "Š š");
|
||||
assert!(decode_as(b"<meta charset=iso-8859-2><p>\xA9</p>", None).contains("<p>Š</p>"));
|
||||
assert_eq!(decode_as(&[0x8A], Some("windows-1252")), "Š");
|
||||
// A page that says UTF-16 without a BOM is read as UTF-8, as browsers do.
|
||||
assert_eq!(decode_as("é".as_bytes(), Some("utf-16")), "é");
|
||||
}
|
||||
|
||||
fn texts(f: &Flow) -> Vec<String> {
|
||||
f.blocks
|
||||
.iter()
|
||||
.map(|b| match b {
|
||||
FBlock::Para(p) => format!("{:?}{:?}:{}", p.kind, p.list.map(|l| (l.ordered, l.level)), p.text()),
|
||||
FBlock::Table(t) => format!("table{}x{}", t.rows.len(), t.cols()),
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn table_style(html: &str) -> Option<String> {
|
||||
let doc = craft_flow::to_doc(&parse(html));
|
||||
doc.body.iter().find_map(|b| match &**b {
|
||||
craft_wordmodel::Block::Table(t) => Some(t.props.style.clone().unwrap_or_default()),
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn borderless_tables_get_no_grid_style() {
|
||||
assert_eq!(table_style("<table><tr><td>a<td>b</table>").as_deref(), Some(""));
|
||||
assert_eq!(table_style("<table style='border:none'><tr><td style='border:0'>a</table>").as_deref(), Some(""));
|
||||
assert_eq!(table_style("<table border=0><tr><td style='border: 0px none'>a</table>").as_deref(), Some(""));
|
||||
assert_eq!(table_style("<table border=1><tr><td>a</table>").as_deref(), Some("TableGrid"));
|
||||
assert_eq!(table_style("<table style='border:1px solid #000'><tr><td>a</table>").as_deref(), Some("TableGrid"));
|
||||
assert_eq!(table_style("<table><tr><td style='border:1px solid #999'>a</table>").as_deref(), Some("TableGrid"));
|
||||
// The last declaration decides: a `style` attribute's over the style sheet's.
|
||||
assert_eq!(table_style("<style>table { border: 1px solid }</style><table><tr><td>a</table>").as_deref(), Some("TableGrid"));
|
||||
assert_eq!(table_style("<style>table { border: 1px solid }</style><table style='border:none'><tr><td>a</table>").as_deref(), Some(""));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn borderless_tables_export_without_cell_borders() {
|
||||
// #438: a table without grid style and borders writes no cell borders and reopens borderless.
|
||||
for (src, grid) in [
|
||||
("<table style='border:none'><tr><td>A<td>B</table>", false),
|
||||
("<table><tr><td>A<td>B</table>", false),
|
||||
("<table border=1><tr><td>A<td>B</table>", true),
|
||||
] {
|
||||
let out = export(&craft_flow::to_doc(&parse(src)));
|
||||
assert_eq!(out.contains("border:1px"), grid, "{src}: {out}");
|
||||
assert_eq!(table_style(&out).as_deref(), Some(if grid { "TableGrid" } else { "" }), "{src}");
|
||||
assert!(out.contains(">A</td>") && out.contains(">B</td>"), "{out}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn input_newlines_preserve_preformatted_lines() {
|
||||
for newline in ["\n", "\r\n", "\r"] {
|
||||
let f = parse(&format!("<pre>{newline} Alpha{newline} <b>Beta</b>{newline}{newline} Gamma</pre>"));
|
||||
assert_eq!(texts(&f), ["CodeNone: Alpha", "CodeNone: Beta", "CodeNone:", "CodeNone: Gamma"], "{newline:?}");
|
||||
let FBlock::Para(p) = &f.blocks[1] else { panic!() };
|
||||
assert!(matches!(p.inlines.last(), Some(Inline::Text(t, f)) if t == "Beta" && f.bold));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn input_newlines_are_normalized_before_character_references() {
|
||||
let f = parse("<p style='white-space:pre'>A B\r\nC\rD</p><p>A\r\nB\rC</p>");
|
||||
assert_eq!(texts(&f), ["NormalNone:A\rB\nC\nD", "NormalNone:A B C"]);
|
||||
let f = parse("<p style='white-space:pre-wrap'>α\r\r\nβ γ</p><a href='https://example.test/a\r\nb\rc d'>link</a>");
|
||||
assert_eq!(texts(&f), ["NormalNone:α\n\nβ\rγ", "NormalNone:link"]);
|
||||
let FBlock::Para(p) = &f.blocks[1] else { panic!() };
|
||||
assert!(matches!(p.inlines.first(), Some(Inline::Text(_, f)) if f.link.as_deref() == Some("https://example.test/a\nb\nc\rd")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn css_preformatted_paragraphs_keep_edge_spaces() {
|
||||
// #439: `white-space:pre` keeps leading and trailing spaces like `<pre>`; normal
|
||||
// paragraphs still collapse.
|
||||
let f = parse(
|
||||
"<p style=\"white-space:pre\"> Alpha Beta </p><p style=\"white-space:pre-wrap\">\n Gamma \n</p><pre> Alpha </pre><p> Alpha Beta </p>",
|
||||
);
|
||||
assert_eq!(texts(&f), vec!["NormalNone: Alpha Beta ", "NormalNone: Gamma ", "CodeNone: Alpha ", "NormalNone:Alpha Beta"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tag_soup() {
|
||||
let f = parse(
|
||||
"<html><head><title>T & U</title><style>p{}</style><script>if(a<b)x</script></head><body><h1>Head</h1><p>One <b>bold<p>Two <i>it</i>\n text<ul><li>a<li>b<ol><li>c</ol></ul><table><tr><td>x<td colspan=2>y<tr><th>z</table><blockquote>q</blockquote><pre>l1\nl2</pre>",
|
||||
);
|
||||
assert_eq!(f.meta.title, "T & U");
|
||||
assert_eq!(
|
||||
texts(&f),
|
||||
vec![
|
||||
"Heading(1)None:Head",
|
||||
"NormalNone:One bold",
|
||||
"NormalNone:Two it text",
|
||||
"NormalSome((false, 0)):a",
|
||||
"NormalSome((false, 0)):b",
|
||||
"NormalSome((true, 1)):c",
|
||||
"table2x3",
|
||||
"QuoteNone:q",
|
||||
"CodeNone:l1",
|
||||
"CodeNone:l2",
|
||||
]
|
||||
);
|
||||
let FBlock::Para(p) = &f.blocks[1] else { panic!() };
|
||||
assert!(matches!(&p.inlines[1], Inline::Text(t, f) if t == "bold" && f.bold));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn omitted_head_keeps_body_text() {
|
||||
let f = parse("<!DOCTYPE html><html><head><meta charset=\"utf-8\"><title>T</title>\n<body><p>Hello world</p><p>Second</p></body></html>");
|
||||
assert_eq!(f.meta.title, "T");
|
||||
assert_eq!(texts(&f), vec!["NormalNone:Hello world", "NormalNone:Second"]);
|
||||
|
||||
for body in [
|
||||
"<p>Hello world</p>",
|
||||
"<div>Hello world</div>",
|
||||
"<span>Hello world</span>",
|
||||
"<a href='https://example.test'>Hello world</a>",
|
||||
"<custom>Hello world</custom>",
|
||||
"<textarea>ignored</textarea>Hello world",
|
||||
] {
|
||||
let f = parse(&format!(
|
||||
"<html><head><title>T & U</title><meta name='author' content='Ann'><style>p{{}}</style><script>ignored</script>{body}",
|
||||
));
|
||||
assert_eq!(f.meta.title, "T & U");
|
||||
assert_eq!(f.meta.author, "Ann");
|
||||
assert_eq!(texts(&f), vec!["NormalNone:Hello world"], "{body}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn styles_and_links() {
|
||||
let f = parse("<p style='text-align:center'><span style=\"color:#ff0000;font-weight:700\">red</span> <a href=\"http://x\">l</a><br>n</p>");
|
||||
let FBlock::Para(p) = &f.blocks[0] else { panic!() };
|
||||
assert_eq!(p.align, Some(Align::Center));
|
||||
assert!(matches!(&p.inlines[0], Inline::Text(t, f) if t == "red" && f.bold && f.color == Some(craft_wordmodel::Rgb(255,0,0))));
|
||||
assert!(p.inlines.iter().any(|i| matches!(i, Inline::Text(t, f) if t == "l" && f.link.as_deref() == Some("http://x"))));
|
||||
assert!(p.text().ends_with("l\nn"));
|
||||
}
|
||||
|
||||
/// The paragraphs of every cell of the first table, row by row.
|
||||
fn cell_paras(f: &Flow) -> Vec<Vec<Vec<Para>>> {
|
||||
let Some(FBlock::Table(t)) = f.blocks.iter().find(|b| matches!(b, FBlock::Table(_))) else { panic!("no table") };
|
||||
t.rows
|
||||
.iter()
|
||||
.map(|r| r.iter().map(|c| c.blocks.iter().filter_map(|b| if let FBlock::Para(p) = b { Some(p.clone()) } else { None }).collect()).collect())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn all_bold(p: &Para) -> bool {
|
||||
!p.inlines.is_empty() && p.inlines.iter().all(|i| matches!(i, Inline::Text(_, f) if f.bold))
|
||||
}
|
||||
|
||||
fn any_bold(p: &Para) -> bool {
|
||||
p.inlines.iter().any(|i| matches!(i, Inline::Text(_, f) if f.bold))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn header_cells_are_bold_unless_their_css_says_otherwise() {
|
||||
// #1381: a <th> is bold, as in browsers and Word; `font-weight:normal` on it or inside it wins.
|
||||
let f = parse(
|
||||
"<table><thead><tr><th>Head</th><th style='font-weight:normal'>Plain</th><th><span style='font-weight:400'>Inner</span></th></tr></thead><tbody><tr><td>a<td>b<td>c</tbody></table>",
|
||||
);
|
||||
let rows = cell_paras(&f);
|
||||
assert!(all_bold(&rows[0][0][0]), "{:?}", rows[0][0]);
|
||||
assert!(!any_bold(&rows[0][1][0]), "{:?}", rows[0][1]);
|
||||
assert!(!any_bold(&rows[0][2][0]), "{:?}", rows[0][2]);
|
||||
assert!(!any_bold(&rows[1][0][0]), "td stays regular");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cell_text_align_reaches_the_cell_paragraphs() {
|
||||
// #1381: `text-align` on a td/th, or inherited from its tr/thead/tbody/table, aligns the
|
||||
// cell's paragraphs; a paragraph's own text-align inside the cell wins.
|
||||
let f = parse(
|
||||
"<table style='text-align:center'><tbody style='text-align:justify'><tr style='text-align:right'>\
|
||||
<td style='text-align:right;'>r</td><td>inherited</td><td style='text-align:right'><p style='text-align:center'>own</p></td>\
|
||||
<td align=right>attr</td></tr><tr><td>body</td></tr></tbody><tr><td>table</td></tr></table>",
|
||||
);
|
||||
let rows = cell_paras(&f);
|
||||
assert_eq!(rows[0][0][0].align, Some(Align::Right), "td style");
|
||||
assert_eq!(rows[0][1][0].align, Some(Align::Right), "from tr");
|
||||
assert_eq!(rows[0][2][0].align, Some(Align::Center), "own paragraph wins");
|
||||
assert_eq!(rows[0][3][0].align, Some(Align::Right), "align attribute");
|
||||
assert_eq!(rows[1][0][0].align, Some(Align::Justify), "from tbody");
|
||||
assert_eq!(rows[2][0][0].align, Some(Align::Center), "from table");
|
||||
// `align` on a table places the table; it doesn't align the text in its cells.
|
||||
let f = parse("<table align=center><tr><td>x</td></tr></table>");
|
||||
assert_eq!(cell_paras(&f)[0][0][0].align, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reporter_sample_header_bold_and_right_aligned_cells() {
|
||||
// #1381: the reporter's table.
|
||||
let f = parse(
|
||||
"<table border=\"1\" style=\"width:100%;border-collapse:collapse;\">\n<thead><tr style=\"background-color:#e8f0fa;\"><th>Indicatore</th><th>Valore atteso</th></tr></thead>\n<tbody>\n<tr><td>Audit</td><td style=\"text-align:right;\">4</td></tr>\n</tbody></table>",
|
||||
);
|
||||
let rows = cell_paras(&f);
|
||||
assert!(all_bold(&rows[0][0][0]) && all_bold(&rows[0][1][0]), "{:?}", rows[0]);
|
||||
assert_eq!(rows[1][1][0].align, Some(Align::Right));
|
||||
assert_eq!(rows[1][0][0].align, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn entities() {
|
||||
assert_eq!(unescape("a <b> AB &bogus; &"), "a <b> AB &bogus; &");
|
||||
assert_eq!(unescape("− © &EURO; ™ ½ ¾"), "− © € ™ ½ ¾");
|
||||
assert_eq!(unescape("θ λ Δ ∇ ∑ √ ≠ ≤ ≥ ♥"), "θ λ Δ ∇ ∑ √ ≠ ≤ ≥ ♥");
|
||||
}
|
||||
|
||||
/// Browsers decode numeric and legacy named references without their `;` in text,
|
||||
/// but keep `&name=` and `&nameX` intact inside attribute values (query strings).
|
||||
#[test]
|
||||
fn references_without_semicolon() {
|
||||
assert_eq!(unescape("© 2024 Acme"), "© 2024 Acme");
|
||||
assert_eq!(unescape("© 2024 Acme"), "© 2024 Acme");
|
||||
assert_eq!(unescape("© 2024 & <"), "© 2024 & <");
|
||||
assert_eq!(unescape("&bogus &# &#x &"), "&bogus &# &#x &");
|
||||
assert_eq!(unescape(" x"), "\u{00A0}x");
|
||||
let f = parse("<p><a href=\"/q?a=1©=2®2=3&b=<\">l</a> © 2024</p>");
|
||||
let FBlock::Para(p) = &f.blocks[0] else { panic!() };
|
||||
assert!(p.inlines.iter().any(|i| matches!(i, Inline::Text(t, f) if t == "l" && f.link.as_deref() == Some("/q?a=1©=2®2=3&b=<"))));
|
||||
assert!(p.text().ends_with("© 2024"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wbr_word_break_opportunity() {
|
||||
let f = parse("<p>Super<wbr>cali<wbr>fragilistic</p>");
|
||||
let Some(FBlock::Para(p)) = f.blocks.first() else { panic!("expected para") };
|
||||
assert_eq!(p.text(), "Super\u{200B}cali\u{200B}fragilistic");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn page_break_after_and_before() {
|
||||
let f = parse("<p style='page-break-after: always'>First</p><p>Second</p>");
|
||||
let paras: Vec<_> = f.blocks.iter().filter_map(|b| if let FBlock::Para(p) = b { Some(p) } else { None }).collect();
|
||||
assert_eq!(paras.len(), 2);
|
||||
assert!(!paras[0].page_break);
|
||||
assert!(paras[1].page_break);
|
||||
|
||||
let f2 = parse("<div style='break-after: page'><p>Part 1</p></div><p>Part 2</p>");
|
||||
let paras2: Vec<_> = f2.blocks.iter().filter_map(|b| if let FBlock::Para(p) = b { Some(p) } else { None }).collect();
|
||||
assert_eq!(paras2.len(), 2);
|
||||
assert!(!paras2[0].page_break);
|
||||
assert!(paras2[1].page_break);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn font_size_relative_and_absolute() {
|
||||
let f = parse("<font size=\"+1\">larger</font><font size=\"-1\">smaller</font><font size=\"5\">huge</font>");
|
||||
let inls = match &f.blocks[0] {
|
||||
FBlock::Para(p) => &p.inlines,
|
||||
_ => panic!("expected para"),
|
||||
};
|
||||
let sizes: Vec<Option<f32>> = inls
|
||||
.iter()
|
||||
.map(|i| match i {
|
||||
Inline::Text(_, fmt) => fmt.size,
|
||||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(sizes, vec![Some(13.5), Some(10.0), Some(18.0)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decode_with_charset_declarations() {
|
||||
let iso = b"<html><head><meta charset=\"iso-8859-1\"></head><body>\xE9l\xE8ve</body></html>";
|
||||
assert!(decode(iso).contains("élève"));
|
||||
|
||||
let win = b"<html><head><meta charset='windows-1252'></head><body>\x80100</body></html>";
|
||||
assert!(decode(win).contains("€100"));
|
||||
|
||||
let gbk = b"<html><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=gbk\"></head><body>\xC4\xE3\xBA\xC3</body></html>";
|
||||
assert!(decode(gbk).contains("你好"));
|
||||
|
||||
let xml = b"<?xml version=\"1.0\" encoding=\"utf-8\"?><html><body>hello</body></html>";
|
||||
assert!(decode(xml).contains("hello"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn raw_text_elements_skip_only_matching_end_tag() {
|
||||
let f = parse("<script>var a = \"</scripted>\"; var b = \"<b>malicious</b>\";</script><p>Hello</p>");
|
||||
assert_eq!(f.blocks.len(), 1);
|
||||
let p = match &f.blocks[0] {
|
||||
FBlock::Para(p) => p,
|
||||
b => panic!("expected para, got {b:?}"),
|
||||
};
|
||||
assert_eq!(p.text(), "Hello");
|
||||
|
||||
let f = parse("<style>/* </styles> */ p { color: red; }</style><p>World</p>");
|
||||
assert_eq!(f.blocks.len(), 1);
|
||||
let p = match &f.blocks[0] {
|
||||
FBlock::Para(p) => p,
|
||||
b => panic!("expected para, got {b:?}"),
|
||||
};
|
||||
assert_eq!(p.text(), "World");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn leading_space_survives_round_trip() {
|
||||
let doc = Document::from_text(" lead");
|
||||
let html = export(&doc);
|
||||
assert!(!html.contains("<p> lead"), "{html}");
|
||||
let back = import(&html);
|
||||
assert_eq!(back.plain_text(Default::default()).replace('\u{a0}', " "), " lead");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn css_escapes_read_every_kind_of_newline() {
|
||||
// A backslash before CR LF, CR, LF or form feed continues the line; after hex digits, one
|
||||
// of them (CR LF as one) ends the escape.
|
||||
assert_eq!(css_unescape("A\\\r\nB\\\rC\\\nD\\\u{C}E"), "ABCDE");
|
||||
assert_eq!(css_unescape("\\41\r\nB \\42\u{C}C \\43 D"), "AB BC CD");
|
||||
assert_eq!(css_unescape("x\\'y\\"), "x'y");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn font_names_are_written_as_css_strings() {
|
||||
// A font name could end the CSS string or the declaration and add CSS of its own (#3739).
|
||||
let fonts = ["Arial\n;position:fixed", "Back\\slash", "O'Brien \"Sans\"", "A,B", "Times New Roman", "M+ 1p"];
|
||||
let mut p = Para::default();
|
||||
for (i, font) in fonts.iter().enumerate() {
|
||||
p.push_text(&format!("t{i}"), &Fmt { font: Some((*font).into()), ..Default::default() });
|
||||
}
|
||||
let html = export(&model::to_doc(&Flow { blocks: vec![FBlock::Para(p)], meta: Meta::default() }));
|
||||
assert!(html.contains("font-family:'Times New Roman'") && html.contains("font-family:'M+ 1p'"), "{html}");
|
||||
let styles: Vec<&str> = html.split(" style=\"").skip(1).filter_map(|s| s.split('"').next()).filter(|s| s.contains("font-family")).collect();
|
||||
assert_eq!(styles.len(), fonts.len(), "{html}");
|
||||
for s in styles {
|
||||
assert!(!s.contains([';', '\n']) && s.matches('\'').count() == 2, "{s}");
|
||||
}
|
||||
// Read back, each run has its font as it was.
|
||||
let back = parse(&html);
|
||||
let runs: Vec<(String, Option<String>)> = back
|
||||
.blocks
|
||||
.iter()
|
||||
.filter_map(|b| if let FBlock::Para(p) = b { Some(p) } else { None })
|
||||
.flat_map(|p| p.inlines.iter())
|
||||
.filter_map(|i| if let Inline::Text(t, f) = i { Some((t.clone(), f.font.as_deref().map(String::from))) } else { None })
|
||||
.collect();
|
||||
for (i, font) in fonts.iter().enumerate() {
|
||||
assert!(runs.contains(&(format!("t{i}"), Some(font.to_string()))), "{font:?} in {runs:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// A document that reads "I read " then a phonetic guide over `base`, then " daily.".
|
||||
fn ruby_doc(base: &str, ruby: &str) -> Document {
|
||||
let props = craft_wordmodel::CharProps::default();
|
||||
let mut p = craft_wordmodel::Paragraph::with_text("I read ", props.clone());
|
||||
let n = p.len();
|
||||
p.insert_object(n, craft_flow::ruby_object(base.into(), ruby.into(), 20.0), &props).unwrap();
|
||||
let n = p.len();
|
||||
p.insert_text(n, " daily.", &props).unwrap();
|
||||
let mut d = Document::new();
|
||||
d.body = vec![craft_wordmodel::para_block(p)];
|
||||
d
|
||||
}
|
||||
|
||||
/// The first flow ruby: its base text and reading.
|
||||
fn ruby_of(f: &Flow) -> Option<(String, String)> {
|
||||
f.blocks.iter().find_map(|b| match b {
|
||||
FBlock::Para(p) => p.inlines.iter().find_map(|i| match i {
|
||||
Inline::Ruby { base, ruby, .. } => Some((base.clone(), ruby.clone())),
|
||||
_ => None,
|
||||
}),
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
|
||||
/// The first document ruby: its base text and reading.
|
||||
fn doc_ruby(d: &Document) -> Option<(String, String)> {
|
||||
d.body.iter().find_map(|b| match &**b {
|
||||
craft_wordmodel::Block::Para(p) => p.objects.iter().find_map(|o| match o {
|
||||
craft_wordmodel::InlineObject::Ruby { base, ruby, .. } => Some((base.clone(), ruby.clone())),
|
||||
_ => None,
|
||||
}),
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
|
||||
/// The first document ruby: base size, ruby size, raise and alignment.
|
||||
fn doc_ruby_attrs(d: &Document) -> Option<(f32, f32, f32, craft_wordmodel::para::RubyAlign)> {
|
||||
d.body.iter().find_map(|b| match &**b {
|
||||
craft_wordmodel::Block::Para(p) => p.objects.iter().find_map(|o| match o {
|
||||
craft_wordmodel::InlineObject::Ruby { base_size, size, raise, align, .. } => Some((*base_size, *size, *raise, *align)),
|
||||
_ => None,
|
||||
}),
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ruby_exports_as_ruby_markup() {
|
||||
// #1681: the base text with its reading in `<rt>`, not plain base text.
|
||||
let out = export(&ruby_doc("漢字", "かんじ"));
|
||||
assert!(out.contains("<p>I read <ruby>漢字<rt>かんじ</rt></ruby> daily.</p>"), "{out}");
|
||||
// The base and the reading are escaped like any other text.
|
||||
let out = export(&ruby_doc("A&B", "x<y"));
|
||||
assert!(out.contains("<ruby>A&B<rt>x<y</rt></ruby>"), "{out}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ruby_imports_with_its_surrounding_text() {
|
||||
let f = parse("<p>I read <ruby>漢字<rt>かんじ</rt></ruby> daily.</p>");
|
||||
assert_eq!(ruby_of(&f), Some(("漢字".into(), "かんじ".into())));
|
||||
assert_eq!(texts(&f), vec!["NormalNone:I read 漢字 daily."]);
|
||||
let FBlock::Para(p) = &f.blocks[0] else { panic!() };
|
||||
assert!(matches!(&p.inlines[0], Inline::Text(t, _) if t == "I read "), "{:?}", p.inlines);
|
||||
assert!(matches!(p.inlines.last(), Some(Inline::Text(t, _)) if t == " daily."), "{:?}", p.inlines);
|
||||
// One document object, holding the base text and the reading.
|
||||
assert_eq!(doc_ruby(&craft_flow::to_doc(&f)), Some(("漢字".into(), "かんじ".into())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ruby_rb_and_rp_fallbacks_import_as_one_guide() {
|
||||
// `<rb>` holds the base text and `<rp>` the fallback parentheses for browsers without
|
||||
// ruby: the parentheses are not text, and the base pieces join.
|
||||
let f = parse("<p><ruby><rb>漢</rb><rb>字</rb><rp>(</rp><rt>かんじ</rt><rp>)</rp></ruby></p>");
|
||||
assert_eq!(ruby_of(&f), Some(("漢字".into(), "かんじ".into())));
|
||||
assert_eq!(texts(&f), vec!["NormalNone:漢字"]);
|
||||
assert_eq!(doc_ruby(&craft_flow::to_doc(&f)), Some(("漢字".into(), "かんじ".into())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ruby_import_uses_the_phonetic_guide_defaults() {
|
||||
// HTML states no alignment, size or raise: the Phonetic Guide's own defaults over a
|
||||
// 20 pt run (half the base size, raised clear of the base text, centred)…
|
||||
let d = crate::doc::import("<p style='font-size:20pt'><ruby>漢字<rt>かんじ</rt></ruby></p>");
|
||||
let Some((base_size, size, raise, align)) = doc_ruby_attrs(&d) else { panic!("no ruby") };
|
||||
assert_eq!((base_size, size, align.ooxml()), (20.0, 10.0, "center"));
|
||||
assert!((raise - 18.0).abs() < 0.001, "{raise}");
|
||||
// …and the guide's 10.5 pt fallback base size when the source states none.
|
||||
let d = crate::doc::import("<p><ruby>漢字<rt>かんじ</rt></ruby></p>");
|
||||
let Some((base_size, size, raise, _)) = doc_ruby_attrs(&d) else { panic!("no ruby") };
|
||||
assert_eq!((base_size, size), (10.5, 5.0));
|
||||
assert!((raise - 9.4).abs() < 0.001, "{raise}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ruby_survives_an_html_round_trip() {
|
||||
let html = export(&ruby_doc("漢字", "かんじ"));
|
||||
assert_eq!(ruby_of(&parse(&html)), Some(("漢字".into(), "かんじ".into())));
|
||||
let back = crate::doc::import(&html);
|
||||
assert_eq!(doc_ruby(&back), Some(("漢字".into(), "かんじ".into())));
|
||||
assert_eq!(back.plain_text(Default::default()), "I read 漢字 daily.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ruby_keeps_the_run_formatting() {
|
||||
let props = craft_wordmodel::CharProps { bold: Some(true), ..Default::default() };
|
||||
let mut p = craft_wordmodel::Paragraph::with_text("I read ", props.clone());
|
||||
let n = p.len();
|
||||
p.insert_object(n, craft_flow::ruby_object("漢字".into(), "かんじ".into(), 20.0), &props).unwrap();
|
||||
let mut d = Document::new();
|
||||
d.body = vec![craft_wordmodel::para_block(p)];
|
||||
assert!(export(&d).contains("<strong><ruby>漢字<rt>かんじ</rt></ruby></strong>"), "{}", export(&d));
|
||||
// Importing it back keeps the formatting for the base text and the reading.
|
||||
let f = parse("<p><strong><ruby>漢字<rt>かんじ</rt></ruby></strong></p>");
|
||||
let FBlock::Para(p) = &f.blocks[0] else { panic!() };
|
||||
assert!(matches!(&p.inlines[0], Inline::Ruby { fmt, .. } if fmt.bold), "{:?}", p.inlines);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_ruby_keeps_its_text() {
|
||||
// No reading, an empty reading, a missing base, an unclosed element and a block boundary
|
||||
// closing it: none of them panics, and the text inside is kept.
|
||||
let cases: [(&str, &[&str]); 7] = [
|
||||
("<p>a <ruby>漢字</ruby> b</p>", &["NormalNone:a 漢字 b"]),
|
||||
("<p>a <ruby>漢字<rt></rt></ruby> b</p>", &["NormalNone:a 漢字 b"]),
|
||||
("<p>a <ruby><rt>かんじ</rt></ruby> b</p>", &["NormalNone:a かんじ b"]),
|
||||
("<p>a <ruby><rt></rt></ruby> b</p>", &["NormalNone:a b"]),
|
||||
("<ruby>", &[]),
|
||||
("<p>a <ruby>漢字<rt>かんじ</rt> b</p><p>next</p>", &["NormalNone:a 漢字 b", "NormalNone:next"]),
|
||||
("<p>a <ruby>漢字<rt>かんじ</rt></p><p>next</p>", &["NormalNone:a 漢字", "NormalNone:next"]),
|
||||
];
|
||||
for (src, want) in cases {
|
||||
assert_eq!(texts(&parse(src)), want, "{src}");
|
||||
}
|
||||
// Ruby in ruby: the inner element's text joins the outer guide's.
|
||||
assert_eq!(ruby_of(&parse("<p><ruby>漢字<ruby>字<rt>じ</rt></ruby><rt>かんじ</rt></ruby></p>")), Some(("漢字字".into(), "じかんじ".into())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn small_big_and_css_font_sizes() {
|
||||
let f = parse(
|
||||
"<p><small>small</small> <big>big</big> <span style=\"font-size: xxx-large\">huge</span> <span style=\"font-size: smaller\">tiny</span></p>",
|
||||
);
|
||||
let FBlock::Para(p) = &f.blocks[0] else { panic!() };
|
||||
let sz = |i: usize| match &p.inlines[i] {
|
||||
Inline::Text(_, f) => f.size,
|
||||
_ => None,
|
||||
};
|
||||
assert_eq!(sz(0), Some(10.0));
|
||||
assert_eq!(sz(2), Some(14.0));
|
||||
assert_eq!(sz(4), Some(36.0));
|
||||
assert_eq!(sz(6), Some(10.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn css_important_casing_and_spacing() {
|
||||
let f = parse(
|
||||
"<p style='color: #ff0000 !IMPORTANT; font-size: 16pt ! important; font-weight: 700 !Important; text-align: center !IMPORTANT'>Styled</p>",
|
||||
);
|
||||
assert_eq!(f.blocks.len(), 1);
|
||||
let p = match &f.blocks[0] {
|
||||
FBlock::Para(p) => p,
|
||||
b => panic!("expected para, got {b:?}"),
|
||||
};
|
||||
assert_eq!(p.align, Some(Align::Center));
|
||||
match &p.inlines[0] {
|
||||
Inline::Text(_, fmt) => {
|
||||
assert_eq!(fmt.color, Some(craft_wordmodel::props::Rgb(0xFF, 0x00, 0x00)));
|
||||
assert_eq!(fmt.size, Some(16.0));
|
||||
assert!(fmt.bold);
|
||||
}
|
||||
i => panic!("expected text inline, got {i:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_cell_vertical_alignment() {
|
||||
let f = parse("<table><tr><td valign=top>top</td><td valign=\"center\">mid</td><td style=\"vertical-align: bottom\">bot</td></tr></table>");
|
||||
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
|
||||
assert_eq!(t.rows[0][0].valign, Some(VAlign::Top));
|
||||
assert_eq!(t.rows[0][1].valign, Some(VAlign::Center));
|
||||
assert_eq!(t.rows[0][2].valign, Some(VAlign::Bottom));
|
||||
let doc = craft_flow::to_doc(&f);
|
||||
let craft_wordmodel::Block::Table(dt) = &*doc.body[0] else { panic!() };
|
||||
assert_eq!(dt.rows[0].cells[0].props.valign, Some(VAlign::Top));
|
||||
assert_eq!(dt.rows[0].cells[1].props.valign, Some(VAlign::Center));
|
||||
assert_eq!(dt.rows[0].cells[2].props.valign, Some(VAlign::Bottom));
|
||||
let html = export(&doc);
|
||||
assert!(html.contains("vertical-align:top"));
|
||||
assert!(html.contains("vertical-align:middle"));
|
||||
assert!(html.contains("vertical-align:bottom"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn many_formatted_leading_breaks_trim_quickly() {
|
||||
// #3705: a paragraph opening with alternating bold and italic `<br>` was trimmed one
|
||||
// span at a time with `Vec::remove(0)`.
|
||||
let html = format!("<p>{}x</p>", "<b><br></b><i><br></i>".repeat(100_000));
|
||||
let t = std::time::Instant::now();
|
||||
let f = parse(&html);
|
||||
// About a second on a loaded machine; trimming one span at a time ran past 6 minutes.
|
||||
assert!(t.elapsed() < std::time::Duration::from_secs(20), "{:?}", t.elapsed());
|
||||
let texts: Vec<String> = f.blocks.iter().filter_map(|b| if let FBlock::Para(p) = b { Some(p.text()) } else { None }).collect();
|
||||
assert_eq!(texts, ["x"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn font_size_at_the_integer_limits_does_not_overflow() {
|
||||
// `<font size>` offsets were added to 3 unchecked, so the extreme values panicked in debug builds.
|
||||
for size in ["+2147483647", "--2147483648", "-2147483647", "+99999999999"] {
|
||||
let f = parse(&format!("<font size=\"{size}\">x</font>"));
|
||||
assert!(texts(&f).iter().any(|t| t.ends_with(":x")), "size={size}");
|
||||
}
|
||||
// The offsets saturate, then clamp to the 1..=7 scale: 7 is 36 pt, 1 is 7.5 pt.
|
||||
let size = |html: &str| match &parse(html).blocks[0] {
|
||||
FBlock::Para(p) => p.inlines.iter().find_map(|i| match i {
|
||||
Inline::Text(_, fmt) => fmt.size,
|
||||
_ => None,
|
||||
}),
|
||||
_ => None,
|
||||
};
|
||||
assert_eq!(size("<font size=\"+2147483647\">x</font>"), Some(36.0));
|
||||
assert_eq!(size("<font size=\"--2147483648\">x</font>"), Some(36.0));
|
||||
assert_eq!(size("<font size=\"-2147483647\">x</font>"), Some(7.5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_comments_do_not_swallow_the_rest() {
|
||||
for html in ["<p>a<!-->b</p><p>c</p>", "<p>a<!--->b</p><p>c</p>"] {
|
||||
let doc = crate::doc::import(html);
|
||||
assert_eq!(doc.plain_text(Default::default()), "ab\nc", "{html}");
|
||||
}
|
||||
}
|
||||
|
||||
fn paras(f: &Flow) -> Vec<&Para> {
|
||||
f.blocks.iter().filter_map(|b| if let FBlock::Para(p) = b { Some(p) } else { None }).collect()
|
||||
}
|
||||
fn fmt_of(p: &Para) -> Fmt {
|
||||
p.inlines.iter().find_map(|i| if let Inline::Text(_, f) = i { Some(f.clone()) } else { None }).unwrap_or_default()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn style_sheet_rules_apply_in_cascade_order() {
|
||||
let f = parse(
|
||||
"<style><!-- p { color: #00ff00; font-size: 10pt } .big { font-size: 20pt } p.big { font-weight: bold } \
|
||||
/* a comment { } */ @media print { p { color: #ff0000 } } div p, li:hover { font-style: italic } .000003 { text-align: center } --></style>\
|
||||
<p>plain</p><p class='big x'>big</p><p class=big style='font-size: 8pt'>inline wins</p><p class=000003>digits</p>",
|
||||
);
|
||||
let ps = paras(&f);
|
||||
let (plain, big, inline, digits) = (fmt_of(ps[0]), fmt_of(ps[1]), fmt_of(ps[2]), fmt_of(ps[3]));
|
||||
assert_eq!((plain.color, plain.size, plain.bold, plain.italic), (Some(craft_wordmodel::Rgb(0, 255, 0)), Some(10.0), false, false));
|
||||
assert_eq!((big.size, big.bold), (Some(20.0), true));
|
||||
assert_eq!(inline.size, Some(8.0));
|
||||
assert_eq!(ps[3].align, Some(Align::Center));
|
||||
assert_eq!(digits.size, Some(10.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn paragraph_css_states_spacing_indents_and_line_height() {
|
||||
let f = parse(
|
||||
"<style>p.a { margin: 6pt 1in 3pt; text-indent: -18pt; line-height: 150% }</style>\
|
||||
<p class=a>a</p><p style='margin-top: 14,4px; margin-bottom: auto; line-height: 20pt'>b</p><p>c</p>",
|
||||
);
|
||||
let ps = paras(&f);
|
||||
assert_eq!(
|
||||
ps[0].spacing,
|
||||
Spacing {
|
||||
before: Some(6.0),
|
||||
after: Some(3.0),
|
||||
left: Some(72.0),
|
||||
right: Some(72.0),
|
||||
first: Some(-18.0),
|
||||
line: Some(LineSpacing::Multiple(1.5)),
|
||||
..Spacing::default()
|
||||
}
|
||||
);
|
||||
// Values that don't parse (a comma decimal, `auto`) state nothing.
|
||||
assert_eq!(ps[1].spacing, Spacing { line: Some(LineSpacing::Exactly(20.0)), ..Spacing::default() });
|
||||
assert_eq!(ps[2].spacing, Spacing::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_page_opens_as_word_opens_one() {
|
||||
let html = "<style>p.k { margin-bottom: 13px } span.z { margin: 0in }</style>\
|
||||
<p>first</p><h1>Head</h1><p class=k><span class=z>kept</span></p><p class=k>own</p><div>bare</div><p> </p>";
|
||||
let f = parse_in(html, &|_| None, true);
|
||||
let ps = paras(&f);
|
||||
// Times New Roman 12 pt on single lines; headings bold and black at a browser's sizes.
|
||||
let first = fmt_of(ps[0]);
|
||||
assert_eq!((first.font.as_deref(), first.size), (Some(PAGE_FONT), Some(12.0)));
|
||||
assert_eq!(ps[0].spacing.line, Some(LineSpacing::Multiple(1.0)));
|
||||
let cell = parse_in("<table><tr><td>cell</td></tr></table>", &|_| None, true);
|
||||
let FBlock::Table(t) = &cell.blocks[0] else { panic!() };
|
||||
let FBlock::Para(cp) = &t.rows[0][0].blocks[0] else { panic!() };
|
||||
assert_eq!(cp.spacing.line, Some(LineSpacing::Multiple(1.0)));
|
||||
let head = fmt_of(ps[1]);
|
||||
assert_eq!((head.bold, head.size, head.color), (true, Some(24.0), Some(craft_wordmodel::Rgb(0, 0, 0))));
|
||||
// A colour the heading inherits stays.
|
||||
// A generic family names a face on a page; pasted text keeps the document's font.
|
||||
let generic = parse_in("<p style='font-family: sans-serif'>a</p><p style='font-family: serif'>b</p>", &|_| None, true);
|
||||
let fonts: Vec<Option<String>> = paras(&generic).iter().map(|p| fmt_of(p).font.map(|f| f.to_string())).collect();
|
||||
assert_eq!(fonts, [Some("Arial".to_string()), Some(PAGE_FONT.to_string())]);
|
||||
assert_eq!(fmt_of(paras(&parse("<p style='font-family: sans-serif'>a</p>"))[0]).font, None);
|
||||
let inherited = parse_in("<body style='color: #000080'><h1>Head</h1></body>", &|_| None, true);
|
||||
assert_eq!(fmt_of(paras(&inherited)[0]).color, Some(craft_wordmodel::Rgb(0, 0, 0x80)));
|
||||
// Automatic spacing where a p or heading leaves its margins out (layout drops it at the
|
||||
// top of the page and of a cell).
|
||||
let auto = |p: &Para| (p.spacing.before_auto, p.spacing.after_auto, p.spacing.before, p.spacing.after);
|
||||
assert_eq!(auto(ps[0]), (true, true, None, None));
|
||||
assert_eq!(auto(ps[1]), (true, true, None, None));
|
||||
// A span's margin replaces the one its paragraph states; the one it leaves out stays automatic.
|
||||
assert_eq!(auto(ps[2]), (true, false, None, Some(0.0)));
|
||||
assert_eq!(auto(ps[3]), (true, false, None, Some(9.75)));
|
||||
// Other blocks get no automatic spacing.
|
||||
assert_eq!(auto(ps[4]), (false, false, None, None));
|
||||
// The document says so, and the empty paragraph's mark keeps its size.
|
||||
let d = model::to_doc(&f);
|
||||
let props = |i: usize| match &*d.body[i] {
|
||||
craft_wordmodel::Block::Para(p) => p.clone(),
|
||||
b => panic!("{b:?}"),
|
||||
};
|
||||
assert_eq!((props(0).props.space_before_auto, props(0).props.space_after_auto), (Some(true), Some(true)));
|
||||
assert_eq!((props(2).props.space_after_auto, props(2).props.space_after), (None, Some(0.0)));
|
||||
assert_eq!(props(5).mark.size, Some(12.0));
|
||||
// `<p> </p>` keeps the size of its space for its line.
|
||||
assert!(ps[5].inlines.is_empty());
|
||||
assert_eq!(ps[5].mark.as_ref().and_then(|m| m.size), Some(12.0));
|
||||
// A fragment (a paste) keeps the document's own formatting.
|
||||
let pasted = parse(html);
|
||||
assert_eq!(fmt_of(paras(&pasted)[0]).font, None);
|
||||
assert_eq!(paras(&pasted)[0].spacing, Spacing::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cells_give_their_columns_widths() {
|
||||
let f = parse(
|
||||
"<style>td.n { width: 42px }</style><table border=1><tr><td colspan=3>t</td></tr><tr><td class=n>1</td><td width=200>a</td><td style='width: 2in'>b</td></tr></table>",
|
||||
);
|
||||
let FBlock::Table(t) = &f.blocks[0] else { panic!("{:?}", f.blocks) };
|
||||
assert_eq!(t.widths, [31.5, 150.0, 144.0]);
|
||||
// A column without a width leaves them all to the layout.
|
||||
let f = parse("<table><tr><td width=100>a</td><td>b</td></tr></table>");
|
||||
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
|
||||
assert!(t.widths.is_empty());
|
||||
// A cell beside a cell spanning rows from above gives its own column its width.
|
||||
let f =
|
||||
parse("<table><tr><td rowspan=2>a</td><td width=200>b</td></tr><tr><td width=300>c</td></tr><tr><td width=100>d</td><td>e</td></tr></table>");
|
||||
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
|
||||
assert_eq!(t.widths, [75.0, 150.0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cells_take_their_css_borders() {
|
||||
use craft_wordmodel::props::{Border, BorderStyle};
|
||||
let f = parse(
|
||||
"<style>td.b { border-top: solid windowtext 1px; border-bottom: 2pt double #ff0000; border-left: none; border-right: 0px solid; vertical-align: bottom }</style><table><tr><td class=b>a</td><td>b</td></tr></table>",
|
||||
);
|
||||
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
|
||||
// The style sheet's other cell properties apply too.
|
||||
assert_eq!(t.rows[0][0].valign, Some(craft_wordmodel::props::VAlign::Bottom));
|
||||
let b = t.rows[0][0].borders.expect("borders");
|
||||
assert_eq!(b.top, Some(Border::single(0.75)));
|
||||
assert_eq!(b.bottom, Some(Border { style: BorderStyle::Double, width: 2.0, color: Some(craft_wordmodel::Rgb(255, 0, 0)), space: 0.0 }));
|
||||
assert_eq!(b.left.map(|l| l.style), Some(BorderStyle::None));
|
||||
// A border of no width is none (OOXML has no zero-width line).
|
||||
assert_eq!(b.right.map(|r| r.style), Some(BorderStyle::None));
|
||||
assert_eq!(t.rows[0][1].borders, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn style_sheet_edge_cases() {
|
||||
let f = parse_in(
|
||||
"<style>@charset \"utf-8\"; @import url(a.css); p.a { color: #0000ff } body { margin: 20px } p.b { margin: 10 5% medium }</style>\
|
||||
<body><p class=a>a</p>bare<p class=b>b</p></body>",
|
||||
&|_| None,
|
||||
true,
|
||||
);
|
||||
let ps = paras(&f);
|
||||
// A rule right after statement at-rules still applies.
|
||||
assert_eq!(fmt_of(ps[0]).color, Some(craft_wordmodel::Rgb(0, 0, 255)));
|
||||
// The body's margins aren't the paragraphs' indents.
|
||||
assert_eq!((ps[1].spacing.left, ps[1].spacing.before), (None, None));
|
||||
// Bare numbers, percentages and keywords state no margin: automatic spacing stays.
|
||||
assert_eq!((ps[2].spacing.before_auto, ps[2].spacing.left), (true, None));
|
||||
// The `style` attribute wins even over a sheet with more declarations than are read.
|
||||
let many = (0..600).map(|i| format!("x-{i}: 1")).collect::<Vec<_>>().join(";");
|
||||
let f = parse(&format!("<style>p {{ {many}; color: #ff0000 }}</style><p style='color: #00ff00'>c</p>"));
|
||||
assert_eq!(fmt_of(paras(&f)[0]).color, Some(craft_wordmodel::Rgb(0, 255, 0)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hostile_style_sheets_never_panic() {
|
||||
// One declaration block shared by many selectors isn't copied for each.
|
||||
let selectors = vec!["p"; 20_000].join(",");
|
||||
let huge = format!("<style>{selectors} {{ color: red; {} }}</style><p>x</p>", "x: y;".repeat(200_000));
|
||||
let _ = parse_in(&huge, &|_| None, true);
|
||||
let many = ".a { color: red } ".repeat(20_000);
|
||||
for css in ["{", "}", "p {", "@media { p { x } ", "p { color: red", "/* open", ".{}", "p..a{}", "\u{0}.é{x:y}", many.as_str()] {
|
||||
let _ = parse_in(&format!("<style>{css}</style><p class='a é'>x</p><td style='width: 1e40px'>y</td>"), &|_| None, true);
|
||||
}
|
||||
}
|
||||
@@ -110,6 +110,18 @@ impl<'a, 'o> Lexer<'a, 'o> {
|
||||
self.pos
|
||||
}
|
||||
|
||||
/// Goes back (or forward) to `pos`, a byte offset an earlier [`Lexer::pos`] gave, to read
|
||||
/// from there again: for a reader that looked at a token and hands it back. An offset past the
|
||||
/// end, or inside a character, moves to the next character boundary.
|
||||
pub fn set_pos(&mut self, pos: usize) {
|
||||
let mut pos = pos.min(self.src.len());
|
||||
while !self.src.is_char_boundary(pos) {
|
||||
pos += 1;
|
||||
}
|
||||
self.pos = pos;
|
||||
self.raw = None;
|
||||
}
|
||||
|
||||
/// The raw content of element `name`, whose start tag was the last token: everything up to
|
||||
/// its end tag (any letter case), or to the end of the input when it has none. The end tag
|
||||
/// is consumed. For `style` and `script`, whose content is not markup.
|
||||
|
||||
@@ -175,3 +175,18 @@ proptest::proptest! {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_token_can_be_read_again() {
|
||||
let mut l = Lexer::new("é<b>x</b>");
|
||||
assert_eq!(l.next(), Some(Token::Text("é")));
|
||||
let at = l.pos();
|
||||
assert!(matches!(l.next(), Some(Token::Start(t)) if t.name == "b"));
|
||||
l.set_pos(at);
|
||||
assert!(matches!(l.next(), Some(Token::Start(t)) if t.name == "b"));
|
||||
// Inside a character or past the end: the next boundary.
|
||||
l.set_pos(1);
|
||||
assert!(matches!(l.next(), Some(Token::Start(t)) if t.name == "b"));
|
||||
l.set_pos(1000);
|
||||
assert_eq!(l.next(), None);
|
||||
}
|
||||
|
||||
@@ -34,6 +34,8 @@
|
||||
#[doc = include_str!("../README.md")]
|
||||
struct ReadmeExamples;
|
||||
|
||||
#[cfg(feature = "flow")]
|
||||
pub mod doc;
|
||||
mod entity;
|
||||
mod lex;
|
||||
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
//! HTML documents (feature `flow`): the HTML parts of WordCraft's cross-format and MathML tests
|
||||
//! (`crates/formats/src/tests.rs`, `mathml.rs`): equations through HTML, hostile input, entities.
|
||||
|
||||
use craft_flow::{FBlock, Inline};
|
||||
use craft_html::doc::{import, inlines_html, parse};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
fn equations(html: &str) -> Vec<(String, bool)> {
|
||||
let mut out = Vec::new();
|
||||
for b in &parse(html).blocks {
|
||||
if let FBlock::Para(p) = b {
|
||||
for i in &p.inlines {
|
||||
if let Inline::Equation { linear, display } = i {
|
||||
out.push((linear.clone(), *display));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn html_export_round_trips_equations() {
|
||||
for (linear, display) in [("y=2x_s", false), ("x=(-b±√(b^2-4ac))/2a", true), ("∑_(k=1)^n▒k", false), ("■(a&b@c&d)", false)] {
|
||||
let html = inlines_html(&[Inline::Equation { linear: linear.into(), display }]);
|
||||
assert!(html.starts_with("<math"), "{html}");
|
||||
assert_eq!(equations(&format!("<p>Before {html} after</p>")), vec![(linear.to_string(), display)], "{html}");
|
||||
}
|
||||
// Text after the equation still arrives; an unclosed element ends with the input.
|
||||
assert!(import("<p><math><mi>x</mi></math> tail</p>").plain_text(Default::default()).contains("tail"));
|
||||
assert!(!equations("<p><math><mfrac><mi>x").is_empty());
|
||||
let deep = format!("<math>{}<mi>x</mi>{}</math>", "<mrow>".repeat(5000), "</mrow>".repeat(5000));
|
||||
assert!(equations(&deep).len() <= 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn math_over_the_token_cap_is_skipped_to_its_end() {
|
||||
// Past the cap, the rest of the element is dropped, not read as body text.
|
||||
let html = format!("<p><math>{}<mrow><mi>z</mi></mrow></math> tail</p>", "<mi>y</mi>".repeat(craft_math::mathml::MAX_NODES * 2));
|
||||
let mut text = String::new();
|
||||
for b in &parse(&html).blocks {
|
||||
if let FBlock::Para(p) = b {
|
||||
for i in &p.inlines {
|
||||
if let Inline::Text(t, _) = i {
|
||||
text.push_str(t);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(text.trim(), "tail");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deep_nesting_is_bounded() {
|
||||
let html = "<div><table><tr><td>".repeat(5000) + "x";
|
||||
assert!(import(&html).plain_text(Default::default()).contains('x'));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn entities_in_text() {
|
||||
let d = import("<p><&Aé &nosuchentity; A</p>");
|
||||
assert_eq!(d.plain_text(Default::default()).trim(), "<&Aé &nosuchentity; A");
|
||||
// An entity name longer than the scan window is left alone.
|
||||
let long = format!("<p>&{};</p>", "a".repeat(40));
|
||||
assert!(import(&long).plain_text(Default::default()).contains(&"a".repeat(40)));
|
||||
}
|
||||
|
||||
/// A run of `&#` is read in linear time: four times as much takes well under the sixteen times
|
||||
/// as long a quadratic read would (a ratio, so a busy machine does not fail it).
|
||||
#[test]
|
||||
fn a_run_of_references_takes_linear_time() {
|
||||
let fastest = |n: usize| {
|
||||
let text = "&#".repeat(n);
|
||||
(0..3)
|
||||
.map(|_| {
|
||||
let t = Instant::now();
|
||||
let _ = import(&text).plain_text(Default::default());
|
||||
t.elapsed()
|
||||
})
|
||||
.min()
|
||||
.unwrap_or_default()
|
||||
};
|
||||
let (small, large) = (fastest(125_000), fastest(500_000));
|
||||
assert!(large < small * 8 + Duration::from_millis(50), "{small:?} for 125,000, {large:?} for 500,000");
|
||||
}
|
||||
|
||||
proptest::proptest! {
|
||||
#[test]
|
||||
fn fuzz_html(s in "(<(/)?(p|b|i|ul|ol|li|table|tr|td|th|pre|blockquote|a href=x|img src=data:,|br|h2|span style='color:red')>|[a-z &;#<>\"'=]){0,120}") {
|
||||
let d = import(&s);
|
||||
let _ = craft_html::doc::export(&d);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user