The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.
\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).
The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.
mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.
* Add craft-sheet: the cell positions, values and tables the spreadsheet formats share
craft-xls carried its own CellRef, Range, Value, CellError, Visibility, sheet-name
quoting, built-in number format table and indexed palette. craft-xlsx and craft-ods
need the same ones, so they move to a crate with no dependencies and craft-xls
re-exports them: an application gets one Value and one CellRef whichever file it read.
Added on the way: A1 parsing (CellRef::parse, Range::parse), Range::count and a1,
CellError::parse and the newer error values SpreadsheetML stores as text,
builtin_format_id for writers.
* Add craft-xlsx: the part layer of XLSX and XLSB workbooks
Package::open finds the workbook part through craft-opc and reads the workbook
part, shared strings (runs and phonetic readings), the styles tables as indexed
tables, and a worksheet's rows one at a time with the rest of the part as a tree.
Binary workbooks give the same workbook, strings, rows and merged cells from their
records. Every other part stays reachable through Package::opc.
From GridCraft crates/xlsx (7e0d96d7): read.rs, sheet_read.rs, styles.rs, xml.rs
(string items, xstring escapes), xlsb.rs and its tests, split from GridCraft's
document model.
* craft-xlsx: XLSB opens up to 4 million cells, records counted once over the workbook
Binary workbooks over 1 million cells were refused, a quarter of what .xls allows,
and an allowance of 5 million records per part refused ordinary files: Excel writes
four records per non-empty row besides the cells, and one BrtCellBlank for every
formatted empty cell. Cells are now 4 million over the workbook (also the most
shared strings), text 256 MiB, and records 40 million for every part together, as
in craft-xls.
Ported from storytold/gridcraft#3807.
---------
Co-authored-by: kiwi4boy <kiwi4boy@gmail.com>
A strict read of the manifest for a reader that refuses a damaged package
instead of opening what it can, as GridCraft's ODS reader does today
(crates/xlsx/src/ods.rs check_manifest): malformed or unbalanced XML, any
document type declaration, undefined entities, content outside the root,
a root that is not manifest:manifest by namespace, a file entry without a
full-path, or more entries than the reader keeps are errors
(Error::Manifest).
The format half of PhotoCraft's Paint.NET import (crates/io/src/pdn.rs and pdn/nrbf.rs at
abeb16c96): the PDN3 header, the .NET binary serialization graph, the layer list and blend
modes, and the chunked gzip pixel blocks. Surfaces, blend-mode mapping and warnings stay in
the app.
MailCraft crates/ical at a44a450: content lines, DATE/DATE-TIME values,
durations and text, the common subset of recurrence rules, and VTIMEZONE
definitions read and written, with their tests. Times are chrono values; the
zone database the writer needs is the caller's, through ZoneRules. The event
and to-do mapping onto MailCraft's model stays in MailCraft.
Co-authored-by: Martin Ball <98639962+benandez@users.noreply.github.com>
* craft-raw: the file-format layer of camera raw files, with DNG and CRW
A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).
The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.
Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.
* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)
RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.
Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.
* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C
Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.
ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.
* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding
Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.
* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over
Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.
* craft-raw: test for the Deflate padding bound
* craft-raw: Pentax PEF and Samsung SRW
Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.
* craft-raw: Canon CR2, with sRAW and mRAW
Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).
* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved
Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).
* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)
* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level
---------
Co-authored-by: radialmonster <radial@gmail.com>
* craft-audio: a program config element in the shape of a channel configuration gets its speakers
An AAC stream with channel configuration 0 whose program config element lists
a centre, a front pair and then exactly the elements of configuration 3 to 6
decoded as C L R Ls Rs LFE and was reported as an unordered layout, so a
caller reading it in WAV order sent the centre to the left speaker. Such a
stream now has the configuration's layout and WAV order. Any other element
list keeps the decoded order, as before.
Ported from storytold/filmcraft#4280 (aac_stream_order in filmcraft-codecs).
* craft-audio: the ADTS program config oracle expects 5.1 in WAV order
ffmpeg 9 writes its 5.1 program config element centre first, so the stream
now has 5.1's speakers: each decoded channel must match ffmpeg's channel of
the same index. An element list in another order keeps the old check.
---------
Co-authored-by: mpratsaki <110930793+mpratsaki@users.noreply.github.com>
The outlook feature adds MailCraft's reader of PidTagRtfCompressed
([MS-OXRTFCP]) and of the text and encapsulated HTML ([MS-OXRTFEX]) of the
RTF inside, from crates/pst/src/rtf.rs at a44a450. Font charsets use this
crate's charset_code_page table; code pages are decoded by craft-mime.
Co-authored-by: Martin Ball <98639962+benandez@users.noreply.github.com>
* Add craft-xcf: GIMP XCF reader from PhotoCraft
The format half of PhotoCraft's XCF import (crates/io/src/xcf.rs at abeb16c96): the header,
precisions, property lists with their length repairs, parasites, layer structures and item
paths, channels, and the uncompressed, RLE and zlib tiles. The mapping onto a document (blend
modes, surfaces, profiles, warnings) stays in the app.
* craft-xcf: an RLE run of zero length is an error
A long run (op 127 or 128) with a 16-bit length of 0 consumed input without producing output,
so a tile of them was work the pixel budget never charged.
Ported from storytold/photocraft#11138.
* craft-xcf: channel names are charged to the metadata budget
Layer names already were. Many channel pointers to one channel with a large name made a copy
per read, multiplying a small file into gigabytes of retained names.
Ported from storytold/photocraft#11138.
craft-xls carried its own CellRef, Range, Value, CellError, Visibility, sheet-name
quoting, built-in number format table and indexed palette. craft-xlsx and craft-ods
need the same ones, so they move to a crate with no dependencies and craft-xls
re-exports them: an application gets one Value and one CellRef whichever file it read.
Added on the way: A1 parsing (CellRef::parse, Range::parse), Range::count and a1,
CellError::parse and the newer error values SpreadsheetML stores as text,
builtin_format_id for writers.
LZW, ASCII85, ASCIIHex, RunLength, the TIFF and PNG predictors, bounded CCITT
fax decoding (through hayro-ccitt) with Group 4 encoding (through fax), and
the lossless JBIG2 encoder with its MQ coder, from pdfcraft-filters at
PdfCraft revision 80e26e674d1aa33078e83c8139875047f3289eb6.
The crate sits below craft-pdf and craft-tiff: it knows no container. The
PDF filter chain, Flate and the bounded JBIG2 decoding entry are not part of
it.
* craft-zip: ZIP preflight, bounded entry read and Budget (from craft-archive, #129)
The ZIP half of the craft-archive proposal (#126, #129): an allocation-free check of the end
records and central directory that runs before a ZIP reader indexes the archive, a bounded
entry read that distrusts the declared size, and the named Budget both charge. The XML half
of #129 is not carried: craft-xml bounds XML now.
* craft-zip: Archive (entry-count guard, duplicate names, budgeted reads) and Writer
check_entry_count is craft-opc's guard: the declared entry count, ZIP64 included, checked on
the raw bytes before zip parses the central directory. Archive reads entries within a per-entry
limit and a total Budget through read_entry. Duplicate names are found by counting the central
directory headers against the entries zip kept. Writer is craft-opc's ZipOut: stored or deflated
entries, ZIP64 for large ones, one fixed timestamp.
* craft-idml, craft-epub: read and write through craft-zip
Both crates drop their own bounded ZIP reads and zip writers. They gain the entry-count check
before the archive is indexed (ZIP64 included) and archives whose bytes depend only on their
content (one fixed timestamp).
---------
Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif
(read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse,
write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers.
They share Metadata, DateTime, Gps and Orientation.
Changed in the move:
- TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own.
- The app namespace is a parameter: parse_xmp_with, write_xmp_with and
MergeRules::namespaces take it, and XmpData no longer has lc_settings.
- The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and
craft-raw read those containers; from_embedded takes the blocks they find.
* Add craft-drawingml: units, colours, preset shapes, custom geometry and text warps
The parts of DrawingML that need no document model, moved from DeckCraft
(60f91f78): crates/color, crates/geom (preset, warp, Xfrm and the EMU
conversions) and crates/pptx/src/custgeom.rs with CustomPath from
crates/model.
The code is unchanged except that custgeom reads craft-xml trees and writes
to a craft-xml Writer instead of DeckCraft's own XML helper, and matches
elements by local name. The 59 tests of the moved code come with it; six
more cover the custom geometry limits.
* drawingml: evaluate a custom geometry's guide lists once per coordinate space
read_cust_geom rebuilt Guides and evaluated avLst and gdLst again for every
path: up to 1,000 paths times up to 10,000 guides, each name copied and
hashed.
Guides are now cached per path coordinate space (w, h). At most 16 spaces
and 4 MiB of guide names are kept per geometry; paths in further spaces are
left out. The first space is always kept, and a geometry without guides is
not capped.
Ported from storytold/deckcraft#1789. A list counts as having guides only
when it has gd children, as that PR's review asked.
* drawingml: the model types for fills, lines, text, tables, charts and diagrams
Moved from DeckCraft (60f91f78) crates/model/src: style.rs, text.rs,
table.rs, chart.rs, numfmt.rs and trendline.rs, plus Diagram, DiagramNode
and the MediaId and SlideId newtypes from lib.rs.
The files are unchanged except that deckcraft_color:: is crate::color:: and
Fill and TableBorder allow clippy::large_enum_variant. Their 60 tests come
with them. Readers and writers follow.
* drawingml: the theme model (colour, font and format schemes)
Moved from DeckCraft crates/model/src/theme.rs at 60f91f78: FontSet, ScriptFont, FontScheme,
FormatScheme, Theme and default_format. The app's theme gallery, its variants and its own default
theme stay in the app. Theme::default() is a plain theme built in place, so nothing here depends
on a gallery.
* craft-dxf: model, the drawing above the tags (header, tables, blocks, layouts)
Document::read indexes a tag list into header variables, table records,
blocks with their entities, the ENTITIES records (a POLYLINE with its
vertices, an INSERT with its attributes) and the objects, each a slice of the
tag list. The typed table records, LAYOUT and DICTIONARY are made on request.
Also the colour index palette, colour, lineweight and transparency values,
extended data lookup and the object coordinate system.
Merged from CADCraft's cadcraft-io dxf_read (5a6aa9b6) and VectorCraft's
vectorcraft-cad import (566f7208). Neither app's document types appear.
* craft-dxf: model entities (geometry, text, MTEXT columns, hatches, dimensions)
Entity::read types an entity record: the properties every entity has, and
LINE, POINT, CIRCLE, ARC, ELLIPSE, LWPOLYLINE, POLYLINE with its vertices
(2D, 3D, mesh, polyface), SPLINE, RAY, XLINE, TEXT, MTEXT with columns,
ATTDEF, INSERT with its attributes (multiline ones included), DIMENSION,
LEADER, SOLID, TRACE, 3DFACE, HATCH (polyline and edge loops, pattern lines,
seeds, gradient), VIEWPORT, IMAGE, WIPEOUT and TOLERANCE. Other types stay
readable from the record's tags. model::text has the caret encoding of
strings, both ways.
Merged from CADCraft's cadcraft-io dxf_read (5a6aa9b6) and VectorCraft's
vectorcraft-cad import (566f7208).
* craft-dxf: a hatch spline edge without fit data leaves the loop's source count
Before R2010 a spline edge has no 97 of its own, so the 97 after it counts the
loop's source entities. It was read as the edge's fit point count and the
sources were lost.
* README: keep main's crate list after the merge
* craft-pdf: the standard security handler from PdfCraft (`crypt`)
Adds the `crypt` feature: PdfCraft's pdfcraft-crypt crate (revision
2b65ab000bfe0997dec5fbed2769d3da3799355b) as craft_pdf::crypt, with its unit,
integration and property tests. The code is unchanged; doc comments and test
imports name the new module.
It decrypts and encrypts strings and streams for revisions 2 to 6 and creates
encryption dictionaries. The reader keeps its own decryptor.
* craft-pdf: the crypt unit tests build for wasm
The property test in src/crypt/tests.rs needs proptest, which is a
dev-dependency of the non-wasm targets only. It is now compiled for those
targets only; the other unit tests still build for wasm.
* hayro-jbig2: bound a stream's total decode work and what a page retains; no i32 overflow
Port of pdfcraft 85c10051 (patches 5-9), 82439525 (patch 10) and 10107d94 (patch 12).
* hayro-jbig2: PdfCraft's JBIG2 follow-ups (refinement fixes, file header, work accounting, dictionary pixel cap)
Ports pdfcraft 72e763ad, b7ec3669, e3bc4e85, 01d3479f, b9e068aa, 858ce960, 07f44038, e38b9d18, 8717a085, 58c30af3 and the hayro-jbig2 part of e2464e14 (with the hayro-jbig2 part of 0ed33990 they build on).
* vendor/README.craft-reader.md: one #831 paragraph, the follow-ups after it
* craft-cms: the changes PhotoCraft made to its copy since extraction
PhotoCraft kept editing crates/cms after craft-cms was extracted from it at
storytold/photocraft@b3222ee8. This ports its 41 commits up to
storytold/photocraft@0143b148 with a three-way merge per file, keeping the
changes craft-cms made on its own (Result-returning API, white preservation,
FOGRA39, named colours, profile ID).
Ported: the .3dl shaper, Mesh header, UTF-16 and .look fixes in lutfile; the
checked ICC offsets and the per-profile curve sample budget; same_bytes and the
byte-confirmed transform cache; the blocked matrix/TRC path and the 8-bit
first-stage table; GpuMatrixShaper; cicp_rgb; the ACEScg and Dot Gain 20 %
built-ins (the latter now the default gray); bisection for a table's flat ends;
PhotoCraft spelled with a capital C in the built-in descriptions, with the
shipped coated CMYK profile's bytes as PhotoCraft regenerated them.
Not taken, because craft-cms already does it another way: Curve::inverse_table
(inverse64_many), the effective-dimension loop in multilinear interpolation,
the curve-count and mluc checks, and the .cube DOMAIN writer.
The 8-bit first-stage table checks device white as the exact path does.
Refs #779
* craft-cms: Lut3d::from_fn
PhotoCraft's proof and HDR display tables are built from a chain of several
transforms, which it wrote into Lut3d's fields. Those are private here, so
from_fn samples any RGB function, with the same size bounds as from_transform.
Refs #779
* craft-cms: README names Lut3d::from_fn
* craft-cms: a UTF-8 byte-order mark before a .cube or .3dl file is skipped
A LUT file saved with a UTF-8 BOM failed to parse: the mark stayed on the
first keyword. decode_text now strips it, as it already handles UTF-16.
Ported from storytold/photocraft#9776.
* craft-cms: a .cube data line with a field that is not a number is an error
A data line's fields after the first were parsed with filter_map, so
'0.25 nope 0.5 0.75' silently became the three numbers that did parse. Every
field must now parse.
Ported from storytold/photocraft#10112.
* craft-cms tests: .look data is pinned to hand-written little-endian bytes
The test built its input with the same to_le_bytes the parser mirrors, so a
wrong byte order on both sides would pass. The bytes are now written out.
Ported from storytold/photocraft#10389.
* craft-cms: a 1D .cube whose domain the 33-point bake cannot hold is refused
A 1D LUT is resampled onto a fixed 33-point grid. A steep curve over a
narrow DOMAIN_MIN/DOMAIN_MAX (or LUT_1D_INPUT_RANGE) does not survive that:
a two-node ramp over 0.24..0.26 baked 42 RGB8 levels off. The parser now
measures the baked table against the exact curve for a non-default domain
and refuses one that shifts by more than two RGB8 levels, naming the
channel. Default-domain tables and gentle curves are unchanged.
Ported from storytold/photocraft#9101.
* craft-cms: only a quote that opens a .cube TITLE protects '#'
A quote anywhere on a TITLE line made the text up to the next quote part of
the title, so 'TITLE Look # "draft"' took its comment into the title. Only
a quote at the start of the value opens a quoted title now; an unterminated
one is an error, and an unquoted title ends at '#' like any other line.
Ported from storytold/photocraft#8506 (its lutfile.rs part).
* craft-cms: GamutCheck::mask_rgba computes a whole gamut-warning mask in parallel
A caller building a gamut-warning mask made one delta_e call per pixel on
one thread. mask_rgba fills the mask (255 where alpha > 0 and the pixel is
out of gamut) in 16K-pixel chunks on the rayon pool, serial below that and
on wasm, and returns the count. The result is identical to out_of_gamut per
pixel. A check not built from an RGB source profile is refused before
anything is written, since an RGBA pixel cannot stand for a CMYK, gray or
Lab colour.
Ported from storytold/photocraft#9748.
---------
Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
Co-authored-by: Max Mittelstädt <62167861+TheGalitube@users.noreply.github.com>
Co-authored-by: smittie2000 <smittie2000@gmail.com>
Co-authored-by: bfenty <1398236+bfenty@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: vangel76 <vangel1976@googlemail.com>
* craft-aep: a flat read-only tree index, borrowed text documents, fewer allocations
The container part of storytold/effectcraft#3302:
- `TreeIndex`: the chunk tree as one flat array in depth-first order,
borrowed from the input, for readers that only look. Same chunks counted
against the same limits and the same errors as `parse`. An `.aepx` is
indexed by `index_aepx` into one buffer. The typed view reads through
either tree (`Node`, `Root`, `Project::from_index`).
- Chunk parser: each list of siblings is counted, then reserved once.
`.aepx`: hex decoded into an output sized once, attribute values without
an entity borrowed. `write` reserves the exact output size.
- Text documents are parsed into a tree that borrows from the body; the
style and paragraph defaults are built once.
- Layer markers and Source Text are read where they are stored instead of
searched for.
- Container accessors that only tests used are removed.
What the importer and exporter in EffectCraft reach across the crate
boundary is public here: `Root`, `Draft`, `AepxDrafts`, `index_aepx`,
`TreeIndex::from_drafts`, `Project::of`, `ProjectSettings::of`,
`Node::marker_list`, `Root::file_version` and `saved_by_version`,
`StyleRun::SAMPLED`. `TimeBase::NTSC` holds the rates as plain
numerator and denominator instead of an EffectCraft type.
* craft-aep: one decoder for an effect parameter definition (`Pard`)
The container part of storytold/effectcraft#3319: the fields of a `pard`
record (type code, name, default, valid range, slider range, popup item
count) are read by one type, `Pard`, which `EffectParam` now reads
through. It is public so that an exporter's tests can read back what
they encode instead of keeping a second decoder.
The rest of that PR (import and export bounds, loss reports) is in
EffectCraft's importer and exporter.
---------
Co-authored-by: vangel76 <vangel1976@googlemail.com>
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks
Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0,
U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a
leading or trailing <br/> was trimmed away.
* craft-captions: TTML/DFXP export drops characters XML 1.0 forbids
Ported from storytold/filmcraft#4467. A control character in cue text made the
exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the
review of that pull request asked for.
* craft-captions: WebVTT cue-text arrow escape is reversible
Ported from storytold/filmcraft#4437. A literal --> in cue text came back as
--> after a write and parse; the writer now adds one amp; level and the reader
removes exactly one.
* craft-captions: a WebVTT speaker name with > no longer spills into the cue text
Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in
place of > inside the voice tag. Also from that pull request's review: a line
break in a name is written as a space, and a name ending in - is kept apart
from the tag's closing >.
* craft-captions: an SRT text line that looks like a timing line survives a round trip
Ported from storytold/filmcraft#4448. The writer marks such a line with a
leading zero-width space and the reader removes it, so the cue is not split.
* craft-captions: back-to-back SCC captions with the same text stay separate cues
Ported from storytold/filmcraft#3808. Every End of Caption now closes the open
cue. MCC's CEA-608 path uses the same decoder.
* craft-captions: an SCC pop-on load may run across the previous caption's erase
Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely
follows another popped on about a second late, because its load had to be one
unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed
between them.
* craft-captions: CEA-708 export starts each caption at its window's top left
Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now
sends DeleteWindows before DefineWindow in place of the ClearWindows after it:
a window made anew is empty with the pen at its top left in any decoder, for
the same number of bytes, so cue timing is unchanged.
* craft-captions: TTML/DFXP export keeps literal < and > text
Ported from storytold/filmcraft#2830. The writer treated every < with a later >
as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its
middle. Only a well-formed tag is markup now; a > inside a quoted attribute
value does not end the tag.
* craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written
Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could
collide with the style ID, and a generated c1 could take the ID of a later cue.
Valid explicit IDs are reserved first; missing, invalid or repeated ones get a
replacement that is in no one's way. IDs are checked as XML NCNames.
* craft-captions: compare the craft-xml and roxmltree views of TTML documents
The TTML reader moved from roxmltree to craft-xml. This dev-only test checks
that both hand it the same elements, attributes and text, by namespace, on
TTML documents and on the writer's output. roxmltree becomes a dev-dependency,
which the cue ID tests use too.
* craft-captions: use craft-timeline's exact time instead of a private copy
craft-captions carried a 290-line subset of the tick, frame rate and timecode
field code because craft-libs had none when it was added. craft-timeline now
has the whole module with the same tick unit. craft_captions::time is a
re-export of craft_timeline::time, so every public path still resolves and
the types are now the same ones craft-timeline's callers hold.
* craft-captions: SCC and MCC time codes count from the document's start timecode
Ported from the caption-format part of storytold/filmcraft#4131
(filmcraft#4130). Document::start_timecode is the label of time zero; the SCC
and MCC writers count from the 29.97 frame with the same label, so the label
carries across rates. Readers note the rate and drop-frame mode of a file's
labels in Document::labels, and Document::count_from counts cues back from a
start time code. charset_warnings names the time codes the file has, which
that pull request's review found missing.
---------
Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
Co-authored-by: SeeRay11 <searay112023@gmail.com>
Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com>
Co-authored-by: JonRC <jonarc06@gmail.com>
Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com>
The container every OpenDocument file shares, moved from WordCraft
(crates/formats/src/odt/crypt.rs, odt/flat.rs, the package reading and
writing in odt/import.rs and odt/package.rs, crates/doc/src/zip_guard.rs
at 06c2ae6f) so that the ODT and ODS readers build on one copy.
- Package and PackageWriter: the ZIP with its mimetype and manifest,
within Limits (entries, bytes per file, bytes per package).
- Manifest: read by namespace whatever the prefix, as GridCraft's ODS
reader did; written as LibreOffice lays it out.
- crypt: AES-256-CBC with PBKDF2, read and written, with the per-file and
per-package key-derivation budgets.
- flat: a flat document to a package and back, for any media type (the
WordCraft code wrote the text one only).
- zip_guard: the entry-count check before zip parses a directory.
No document content is read here.
PdfCraft's pdfcraft-prc crate (revision
80e26e674d1aa33078e83c8139875047f3289eb6), moved whole with its tests. It reads
PRC (ISO 14739-1), the model format of PDF 3D annotations: the container, the
bit-level primitives, the schema, tessellations and the product tree.
The code is unchanged. The package and its README are renamed, the doctests
and tests import craft_prc, and the Double code table taken from prc-rs (MIT)
has its row in ATTRIBUTION.md and its licence in licenses/prc-rs.
The bottom layer every HTML reader in the Craft apps carried its own copy
of. The tokenizer is MailCraft's (crates/html/src/tokenizer.rs), with what
it dropped and capped turned into Options and Limits, a self-closing flag,
and raw_content for callers that read a style sheet's text. The character
references are WordCraft's (crates/formats/src/html/entity.rs at 06c2ae6f):
its name table and the 94 legacy names that decode without a semicolon,
with MailCraft's Latin-1 and Greek tables and its handling of numbers too
long to be numbers.
No tree, no CSS, no decoding of bytes.
* craft-zip: ZIP preflight, bounded entry read and Budget (from craft-archive, #129)
The ZIP half of the craft-archive proposal (#126, #129): an allocation-free check of the end
records and central directory that runs before a ZIP reader indexes the archive, a bounded
entry read that distrusts the declared size, and the named Budget both charge. The XML half
of #129 is not carried: craft-xml bounds XML now.
* craft-zip: Archive (entry-count guard, duplicate names, budgeted reads) and Writer
check_entry_count is craft-opc's guard: the declared entry count, ZIP64 included, checked on
the raw bytes before zip parses the central directory. Archive reads entries within a per-entry
limit and a total Budget through read_entry. Duplicate names are found by counting the central
directory headers against the entries zip kept. Writer is craft-opc's ZipOut: stored or deflated
entries, ZIP64 for large ones, one fixed timestamp.
---------
Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
One walk over a GIF's blocks, without decompressing pixels: the logical
screen, each frame's position, delay, disposal and transparent colour, the
loop count and the first frame's colour table. Replaces PhotoCraft's two
block walkers and gives the other apps a header-only animation probe.
* craft-audio: integer PCM of WAV and AIFF files bit for bit, and minimal files around it
wav_pcm and aiff_pcm (feature decode-native) return a file's integer PCM as stored
(RawPcm: little-endian interleaved bytes, 8-bit unsigned, with channels, rate and
bits), built on the existing WAV and AIFF chunk parsers. pcm_wav_file,
pcm_wav_header, pcm_aiff_file and pcm_aiff_header (feature pcm-encode) write the
44-byte canonical WAVE header and the plain AIFF header around such bytes.
For essence embedded in AAF and OMF files, which is moved bit for bit and whose
descriptors hold a bare header as a summary. The writers are FilmCraft's
(filmcraft-interchange wav.rs); the readers replace its second WAV/AIFF parser
with this crate's.
* craft-timeline: exact media time, reports, media paths, ALE and CMX 3600 EDL
New crate for editorial timeline interchange (#782), moved from FilmCraft at
4bec22bd:
- time: filmcraft-time whole (Tick, FrameRate, TimeRange, SMPTE timecode,
feet+frames, typed-timecode parsing) with its tests.
- report: Report / Level / ReportEntry.
- path: file URL and path conversion, relative references joined to a folder.
- rate: timebase+ntsc and floating rates to exact rates, MAX_RATE.
- ale: AleDoc, parse, write, standard heading and columns.
- edl: the parser (events, lines, timecode fields, M2, AUD, comment extensions),
the frame-rate guess, and the line layout of the writer, on frame counts.
- Format, detect_text, decode_text.
Each format module gives the document as the format writes it. Mapping onto an
application's timeline stays in the application.
* craft-timeline: strip a UTF-8 byte order mark when parsing ALE files
An ALE saved with a BOM was refused as "not an ALE file": the mark made the
first line something other than "Heading".
Ported from storytold/filmcraft#4531.
* craft-aaf: AAF (Edit Protocol) and OMF 2.0 readers and writers over one composition model
New crate (#782), moved from FilmCraft's filmcraft-interchange at 4bec22bd:
the mob-style model types of comp.rs, aaf/{ids,store,read,write}.rs and
omf/{bento,read,write}.rs, with their structural and hostile-input tests
rebuilt on model documents instead of FilmCraft projects.
The container is craft-cfb, time and reports are craft-timeline's, and the
WAVE/AIFF wrappers of embedded PCM are craft-audio's. The writer's identity
(company, product, version, AAF product id) is passed in instead of being
FilmCraft's. The private tags and properties FilmCraft wrote keep their names
so existing files read back.
The mapping between the model and an application's project stays in the
application.
---------
Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
wav_pcm and aiff_pcm (feature decode-native) return a file's integer PCM as stored
(RawPcm: little-endian interleaved bytes, 8-bit unsigned, with channels, rate and
bits), built on the existing WAV and AIFF chunk parsers. pcm_wav_file,
pcm_wav_header, pcm_aiff_file and pcm_aiff_header (feature pcm-encode) write the
44-byte canonical WAVE header and the plain AIFF header around such bytes.
For essence embedded in AAF and OMF files, which is moved bit for bit and whose
descriptors hold a bare header as a summary. The writers are FilmCraft's
(filmcraft-interchange wav.rs); the readers replace its second WAV/AIFF parser
with this crate's.
* Add craft-drawingml: units, colours, preset shapes, custom geometry and text warps
The parts of DrawingML that need no document model, moved from DeckCraft
(60f91f78): crates/color, crates/geom (preset, warp, Xfrm and the EMU
conversions) and crates/pptx/src/custgeom.rs with CustomPath from
crates/model.
The code is unchanged except that custgeom reads craft-xml trees and writes
to a craft-xml Writer instead of DeckCraft's own XML helper, and matches
elements by local name. The 59 tests of the moved code come with it; six
more cover the custom geometry limits.
* drawingml: evaluate a custom geometry's guide lists once per coordinate space
read_cust_geom rebuilt Guides and evaluated avLst and gdLst again for every
path: up to 1,000 paths times up to 10,000 guides, each name copied and
hashed.
Guides are now cached per path coordinate space (w, h). At most 16 spaces
and 4 MiB of guide names are kept per geometry; paths in further spaces are
left out. The first space is always kept, and a geometry without guides is
not capped.
Ported from storytold/deckcraft#1789. A list counts as having guides only
when it has gd children, as that PR's review asked.
* drawingml: the model types for fills, lines, text, tables, charts and diagrams
Moved from DeckCraft (60f91f78) crates/model/src: style.rs, text.rs,
table.rs, chart.rs, numfmt.rs and trendline.rs, plus Diagram, DiagramNode
and the MediaId and SlideId newtypes from lib.rs.
The files are unchanged except that deckcraft_color:: is crate::color:: and
Fill and TableBorder allow clippy::large_enum_variant. Their 60 tests come
with them. Readers and writers follow.
* craft-psd: Photoshop PSD/PSB reader and writer, moved from PhotoCraft
PhotoCraft's `photocraft-psd` crate (crates/psd at storytold/photocraft
63b641b55e54bf8f5c83efd977fdb16a2c255431) as `craft-psd`: the lossless
PSD/PSB model, reader, writer and builder, descriptors, the Photoshop
resource files that share its primitives (.abr, .asl, .aco, .act, .ase,
.grd, .pat, .atn, .csh, .shc) and the Photoshop blocks of a layered TIFF.
The crate had no dependency on any other PhotoCraft crate.
Code is unchanged apart from the crate name, craft-libs' rustfmt width,
comments that named other PhotoCraft crates, issue references (now
`photocraft#N`), and the corpus tests' directory (`CRAFT_PSD_CORPUS` or
`corpus/` at the workspace root). Tests, proptests, fuzz targets and
fixtures move with it.
Part of #778, #786.
* craft-psd: engine_data, the text engine document of type layers
The EngineData reader and writer from PhotoCraft's photocraft-text
(crates/text/src/engine_data.rs at storytold/photocraft 63b641b5): a
tree of values that is written back in Photoshop's layout, with strings
stored as plain bytes kept as plain bytes.
From EffectCraft's effectcraft-psd (crates/psd/src/engine_data.rs at
storytold/effectcraft 072e1af9), ported onto that tree: `first_style`,
the first style run as a typed `TextStyle` with bounded sizes and
spacings, and `minimal`, a one-run document for writers and fixtures.
`Value::index` is new.
Both apps had their own parser for the same syntax.
* craft-psd: engine_data::parse_lossy reads what it can of a damaged document
EffectCraft's EngineData reader was best effort: a truncated or damaged
document still gave the text and style it held. `parse_lossy` does that
on the same parser: a truncated dictionary, array or string keeps what
it has, a byte that starts no key is skipped, a token that is no number
becomes a name, and garbage gives an empty dictionary. `parse` stays
strict and is unchanged in what it accepts.
The parser's `value` is split into one function per kind of value, and
reads with `get` instead of indexing.
* craft-mime: strict base64, mailto links and code page labels from MailCraft
What MailCraft's mime and charset crates still had that craft-mime did not:
strict RFC 4648 base64 (crates/charset), mailto: links (RFC 6068,
crates/mime) and the Windows code page to charset label table (crates/pst).
With these, MailCraft's charset crate has nothing craft-mime lacks.
* Add craft-vcard: vCard reader and writer from MailCraft
MailCraft crates/vcard at a44a450: vCard 2.1, 3.0 and 4.0 read, 3.0 written,
with its limits and tests. The model is the crate's own Contact (the vCard
fields of MailCraft's); the birthday is a chrono::NaiveDate. Character sets
and strict base64 come from craft-mime.
---------
Co-authored-by: Martin Ball <98639962+benandez@users.noreply.github.com>
* craft-media: MPEG-2 transport and program stream demuxer from FilmCraft (feature `mpegts`)
Moves filmcraft-mpegts (storytold/filmcraft@4bec22b) into craft_media::mpegts.
ByteSource is the crate's trait, Error is written without thiserror so the
feature has no dependencies, and Codec::codec_id() and Stream::codec_params()
map streams to CodecId. Adds Container::MpegTs and Container::MpegPs, an
ffprobe oracle test over ten ffmpeg-made files, and hostile-input tests.
* craft-media: module order after merging main
`craft_lottie::model` is the Lottie document as the file has it: one
struct per object of the lottie-spec 1.0 object model (animation, assets,
layers by type, transform, masks, effects, shape items, text, markers,
fonts, glyphs), one field per key. `Animation::read` reads within
`Limits` (bytes, nesting, layers, assets, shape items); `to_json` writes.
Keys the model does not name, values of another type than expected, and
layers and shape items of an unknown type stay in `extra` and are written
back, so a document survives a read and a write. Animatable properties
stay JSON (`Animated`) and are read with `read_prop`.
* craft-svg paths: a shape carries the name of its Inkscape layer
Shape::layer is the inkscape:label (else the id) of the innermost
<g inkscape:groupmode="layer"> around the element, so a CAD or cutter
import can keep the drawing's layers.
* craft-svg guard: bound attributes per element and over the document; find entities holding markup
too_wide is the attribute check of storytold/figcraft#2251: an XML
reader checks each attribute against the earlier ones on its element,
so an element with millions of them is quadratic work before any other
limit applies. entity_holds_markup lets a caller refuse a document type
whose entities become elements, as PhotoCraft does.
* craft-svg: feature `tree`, usvg behind one bounded entry point
usvg 0.48.1 (Apache-2.0 OR MIT), re-exported as craft_svg::tree::usvg,
so every app parses with one version and one set of checks:
tree::parse runs svgz, the entity, attribute and nesting guards, the
reference-chain height (VectorCraft's depth::height, on this crate's
cascade), then usvg. tree::image_resolver bounds embedded SVGs
(FigCraft's budget::resolver). tree::draw bounds what drawing a tree
with resvg allocates (PhotoCraft's svg/limits.rs).
Part of #778 and #789.
craft-captions carried a 290-line subset of the tick, frame rate and timecode
field code because craft-libs had none when it was added. craft-timeline now
has the whole module with the same tick unit. craft_captions::time is a
re-export of craft_timeline::time, so every public path still resolves and
the types are now the same ones craft-timeline's callers hold.
The format half of PhotoCraft's OpenRaster import and export (crates/io/src/ora.rs at
abeb16c96): the archive and its mimetype, stack.xml as a tree of layers and stacks with the
attributes the file has, the 0.0.6 group isolation rule, and a writer. PNG coding, blend-mode
mapping, rendering and warnings stay in the app.
stack.xml is read with craft-xml and the archive with the zip crate.
What MailCraft's mime and charset crates still had that craft-mime did not:
strict RFC 4648 base64 (crates/charset), mailto: links (RFC 6068,
crates/mime) and the Windows code page to charset label table (crates/pst).
With these, MailCraft's charset crate has nothing craft-mime lacks.
Co-authored-by: Martin Ball <98639962+benandez@users.noreply.github.com>
* craft-y4m: YUV4MPEG2 reader
FilmCraft's Y4M reader as a crate: the stream header, colour space layouts,
a frame index built from the FRAME lines through a read callback, and plane
ranges. The media source and frame types stay in the app.
* craft-y4m: a header parameter starting with a multi-byte character is ignored
Port of storytold/filmcraft#4562: the tag of a stream header parameter is its
first character, not its first byte, so a parameter such as `éX` is skipped
like any other unknown tag instead of being split inside the character.
---------
Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks
Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0,
U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a
leading or trailing <br/> was trimmed away.
* craft-captions: TTML/DFXP export drops characters XML 1.0 forbids
Ported from storytold/filmcraft#4467. A control character in cue text made the
exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the
review of that pull request asked for.
* craft-captions: WebVTT cue-text arrow escape is reversible
Ported from storytold/filmcraft#4437. A literal --> in cue text came back as
--> after a write and parse; the writer now adds one amp; level and the reader
removes exactly one.
* craft-captions: a WebVTT speaker name with > no longer spills into the cue text
Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in
place of > inside the voice tag. Also from that pull request's review: a line
break in a name is written as a space, and a name ending in - is kept apart
from the tag's closing >.
* craft-captions: an SRT text line that looks like a timing line survives a round trip
Ported from storytold/filmcraft#4448. The writer marks such a line with a
leading zero-width space and the reader removes it, so the cue is not split.
* craft-captions: back-to-back SCC captions with the same text stay separate cues
Ported from storytold/filmcraft#3808. Every End of Caption now closes the open
cue. MCC's CEA-608 path uses the same decoder.
* craft-captions: an SCC pop-on load may run across the previous caption's erase
Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely
follows another popped on about a second late, because its load had to be one
unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed
between them.
* craft-captions: CEA-708 export starts each caption at its window's top left
Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now
sends DeleteWindows before DefineWindow in place of the ClearWindows after it:
a window made anew is empty with the pen at its top left in any decoder, for
the same number of bytes, so cue timing is unchanged.
* craft-captions: TTML/DFXP export keeps literal < and > text
Ported from storytold/filmcraft#2830. The writer treated every < with a later >
as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its
middle. Only a well-formed tag is markup now; a > inside a quoted attribute
value does not end the tag.
* craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written
Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could
collide with the style ID, and a generated c1 could take the ID of a later cue.
Valid explicit IDs are reserved first; missing, invalid or repeated ones get a
replacement that is in no one's way. IDs are checked as XML NCNames.
* craft-captions: compare the craft-xml and roxmltree views of TTML documents
The TTML reader moved from roxmltree to craft-xml. This dev-only test checks
that both hand it the same elements, attributes and text, by namespace, on
TTML documents and on the writer's output. roxmltree becomes a dev-dependency,
which the cue ID tests use too.
---------
Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com>
Co-authored-by: SeeRay11 <searay112023@gmail.com>
Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com>
Co-authored-by: JonRC <jonarc06@gmail.com>
Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com>
Moved from WordCraft's crates/docx/src/crypt.rs (07f97737): agile and
standard decryption, agile encryption, the ISO and legacy protection
password hashes. The compound file is read and written with craft-cfb
and the descriptor parsed with craft-xml.
From GridCraft's crates/xlsx/src/crypto.rs (5cf52c5f): check(), named
unsupported schemes, rights-managed files, the password key encryptor
chosen by its URI, and independent key vectors.
The [MS-PST] reader of MailCraft crates/pst at a44a450, moved as it is: the
node database, heap, property and table contexts, folders, messages,
recipients, attachments, named properties, recurrence blobs and list
streams, with its limits, salvage and tests.
Left in MailCraft for now: the compressed-RTF and RTF text reading and the
code-page name table, which are not part of the .pst format.
Co-authored-by: Martin Ball <98639962+benandez@users.noreply.github.com>
The OpenEXR naming layer PhotoCraft and EffectCraft each have: layer and
channel names, the channels that make a colour picture, and Cryptomatte IDs,
metadata keys, manifests and rank channels. No decoder dependency; decoding
follows in later changes.
* Add craft-kra: Krita archives, maindoc.xml and tiled pixel files, from PhotoCraft
The format half of PhotoCraft's Krita import (crates/io/src/kra.rs and kra/tiles.rs at
abeb16c96): the archive and its mimetype, maindoc.xml as a tree of nodes with the attributes
Krita writes, the tiled pixel files and their LZF compression. The mapping onto a document
(blend modes, surfaces, warnings) stays in the app.
maindoc.xml is read with craft-xml and the archive with the zip crate.
* craft-kra: a default pixel is read under a one-pixel limit
The .defaultpixel member (1 to 8 bytes) was inflated under the 1 GiB layer-file limit before
its size was checked.
Ported from storytold/photocraft#11138.
* craft-kra: the tile table is not reserved from the declared count
A tiled file may declare one record per eight bytes of its size. Reserving that many up front
is 3 GiB for a 1 GiB member that holds none of them. Reserve 65,536 records at first and grow
as records are read, never past the declared count.
Ported from storytold/photocraft#13309.
Dylan M. Taylor