mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-12 08:52:32 +00:00
Dylan M. Taylorandshuff57
bdf06f1029
* Add craft-wordmodel: WordCraft's document model without its editing code
WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.
Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.
* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs
The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.
* craft-mime: detect, for bytes that name no character set
is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.
* craft-wordmodel: keep the document operations (edit.rs) with the model
The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.
base64 0.23, as WordCraft and craft-offcrypto use.
* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath
The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.
mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.
* craft-math: read MathLive's and other spellings of LaTeX math commands
\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).
* craft-math: equaSHion's equation corpus as test data
corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (7a6ef6b9), unmodified.
* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README
The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.
* Add craft-flow: the flow model the text formats map to, on craft-wordmodel
WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.
* Add craft-markdown: Markdown read and written, on craft-flow
WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.
* WIP: craft-latex sources from WordCraft (unmodified)
* Add craft-latex: LaTeX documents read and written, on craft-flow
WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.
* craft-math: latex_text, LaTeX math and the linear format converted as text
WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.
* craft-math: latex_text's group finders are public, for craft-latex's length parser
* craft-latex: math through craft_math::latex_text; label timing test by ratio
The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.
---------
Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
craft-math
Equations as one tree, and the syntaxes it is read from and written to:
| Module | Syntax | Read | Write |
|---|---|---|---|
math |
the tree: MNode, MRun, Math (an equation with its kept source OMML), modelled on the Office Math object model (ECMA-376 Part 1 §22.1) |
||
math_linear |
UnicodeMath, the linear format Word shows (x=(-b±√(b^2-4ac))/2a) |
math::parse_linear |
math::to_linear |
math_latex |
LaTeX math, including MathLive's spellings | parse_latex |
to_latex |
latex_text |
LaTeX math as text, keeping the source's spacing (for LaTeX documents) | latex_to_linear |
linear_to_latex |
omml |
Office Math Markup (m:oMath, DOCX/PPTX/XLSX) |
read_omath, read_str |
write_equation, write_str |
mathml |
presentation MathML (HTML, ODF formulas, the clipboard) | parse, read, to_tree |
to_mathml, linear_to_mathml |
Beside them: math_symbols (LaTeX names of symbols), math_tree (the arguments of a node, a walk
over every run), symbol_fonts (Symbol, Wingdings and the other symbol fonts as Unicode) and
Rgb (a run's colour).
use craft_math::{Math, math, math_latex, mathml, omml};
let tree = math_latex::parse_latex(r"\frac{a}{b}+\sqrt{x}");
assert_eq!(math::to_linear(&tree), "a/b+√x");
// OMML out and back.
let eq = Math { nodes: tree.clone(), ..Default::default() };
let xml = omml::write_str("", false, &eq, None);
let read = omml::read_str(&xml).unwrap();
assert_eq!(read.nodes, tree);
// MathML carries the linear format in an annotation, so it reads back exactly.
let ml = mathml::linear_to_mathml("a/b+√x", true);
assert_eq!(mathml::parse(&ml).and_then(|m| mathml::read(&m)), Some(("a/b+√x".to_string(), true)));
Reading OMML
omml::read_omath takes an m:oMath element of a craft_xml tree read with the canonical
prefixes of omml::NAMESPACES (m, w and mc, transitional and strict URIs), so a document
reader passes the element it already has. omml::read_str and omml::parse read a string. The
source XML is kept in Math::omml; write_equation writes it back unchanged while the
equation's colour is the same and the XML is well formed, so a saved document carries the
equation as Word wrote it.
Limits
Nesting is bounded by math::MAX_DEPTH in every reader and writer; omml::parse reads at most
omml::MAX_ELEMENTS elements and mathml::parse keeps at most mathml::MAX_NODES; the LaTeX
reader looks at 20,000 characters. Nothing panics on any input.
Provenance
The tree, the linear format, LaTeX, the symbol names, the tree walk, the symbol fonts and Rgb
are WordCraft's (crates/doc), as moved into craft-wordmodel; OMML is WordCraft's DOCX reader
and writer (crates/docx/src/read/math.rs, write/math.rs) and MathML its formats crate's
(crates/formats/src/mathml.rs), moved onto craft-xml. The MathLive command spellings and the
round-trip corpus in tests/corpus/ come from equaSHion (storytold/craft-libs#29).