mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-12 08:52:32 +00:00
craft-xls-objects
1410
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dylan M. Taylor
|
832e3456e9 | Merge main into craft-xls-objects | ||
|
Dylan M. Taylor
|
d59bcf56c9 |
craft-timeline: FCPXML on the neutral model (#987)
* craft-timeline: a neutral timeline model and the import builder model: a Project of bins and items (media, sequences, subclips, and items no interchange format carries), sequences of tracks with clips, transitions, markers and effects with keyframes. Cut down from FilmCraft's project model to what Final Cut Pro 7 XML, FCPXML and OpenTimelineIO describe; EffectCraft already uses that shape for its interchange. Three things stay the application's: effects (looked up through a Catalog the caller supplies), generators beyond the common ones (Generator::Other with an opaque payload), and per-object extension JSON (ext) that a format can carry unread. fx: the effect and parameter ids the formats name themselves, and Standard, a catalog of just those. build: FilmCraft's import Builder and shared helpers (filmcraft-interchange common.rs at 4bec22bd) on the model, with the effect registry replaced by the catalog, and the bounds every import ends with (Imported::bound, from lib.rs). * craft-timeline: Final Cut Pro 7 XML (xmeml) on the neutral model fcp7: FilmCraft's xmeml reader and writer (filmcraft-interchange fcp7.rs at 4bec22bd) moved onto craft_timeline::model, behind feature xml. - XML is read with craft-xml, strictly: at most 256 levels of nesting (the limit EffectCraft checked with its own depth scan before handing documents over), 2 000 000 nodes, 256 MiB; a document type declaration with an internal subset and entities XML does not define are refused. It is written with craft-xml's Writer. - Effects other than motion, opacity, time remap, audio levels and pan are looked up through the caller's Catalog, by id and by name, and written with the catalog's stable name and category. - An application generator round-trips as <namespace>:<payload> in effectid (FilmCraft wrote filmcraft:<json>); transparent video, counting leader and bars and tone are written by name, which the reader already took. - Finding the sequence a clip item is in is one walk of the document instead of an ancestors() lookup per clip item. tests/it: FilmCraft's FCP7 integration tests (the hand-written Premiere-style document, the rich round trip, the 48-case property round trip, split mono and stereo channels, sub-frame spans, absurd timebases and start timecodes, the link-group bound) on the model, with a test catalog for the four application effects they use. * craft-timeline: FCPXML on the neutral model fcpxml: FilmCraft's FCPXML 1.9-1.11 reader and writer (filmcraft-interchange fcpxml.rs at 4bec22bd) moved onto craft_timeline::model, behind feature xml. XML is read with craft-xml as fcp7 is; filters other than the transform, blend and volume adjustments go through the caller's Catalog by name, and an effect resource's uid is <namespace>.<effect id> (FilmCraft wrote filmcraft.<id>). ExportOptions gains fcpxml_version. tests/it/fcpxml.rs: FilmCraft's FCPXML integration tests on the model (the hand-written connected-clips document, hostile lane numbers, standalone video stills, self-referencing compound media, round trips at 1.9, 1.10 and 1.11, the property round trip with its regression seed, percent-decoded relative URLs). The FCP7 tests export to FCPXML again where FilmCraft's did. |
||
|
Dylan M. TaylorandMartin Ball
|
d197b1e72f |
Add craft-dav: WebDAV multistatus reader from MailCraft (#975)
MailCraft crates/dav/src/xml.rs at a44a450, re-based onto craft-xml: a multistatus read fail-closed into a bounded tree with fixed prefixes for the DAV, CalDAV, CardDAV and related namespaces, and the properties of a response's successful propstats. Its tests come along. Co-authored-by: Martin Ball <98639962+benandez@users.noreply.github.com> |
||
|
Dylan M. Taylor
|
b85549a5b4 |
craft-dxf: model objects (page setup, GROUP, IMAGEDEF, SORTENTSTABLE, SPATIAL_FILTER, DIMASSOC, SCALE, MLINESTYLE), HELIX, dimension overrides (#955)
* craft-dxf: model, the drawing above the tags (header, tables, blocks, layouts) Document::read indexes a tag list into header variables, table records, blocks with their entities, the ENTITIES records (a POLYLINE with its vertices, an INSERT with its attributes) and the objects, each a slice of the tag list. The typed table records, LAYOUT and DICTIONARY are made on request. Also the colour index palette, colour, lineweight and transparency values, extended data lookup and the object coordinate system. Merged from CADCraft's cadcraft-io dxf_read (5a6aa9b6) and VectorCraft's vectorcraft-cad import (566f7208). Neither app's document types appear. * craft-dxf: model entities (geometry, text, MTEXT columns, hatches, dimensions) Entity::read types an entity record: the properties every entity has, and LINE, POINT, CIRCLE, ARC, ELLIPSE, LWPOLYLINE, POLYLINE with its vertices (2D, 3D, mesh, polyface), SPLINE, RAY, XLINE, TEXT, MTEXT with columns, ATTDEF, INSERT with its attributes (multiline ones included), DIMENSION, LEADER, SOLID, TRACE, 3DFACE, HATCH (polyline and edge loops, pattern lines, seeds, gradient), VIEWPORT, IMAGE, WIPEOUT and TOLERANCE. Other types stay readable from the record's tags. model::text has the caret encoding of strings, both ways. Merged from CADCraft's cadcraft-io dxf_read (5a6aa9b6) and VectorCraft's vectorcraft-cad import (566f7208). * craft-dxf: a hatch spline edge without fit data leaves the loop's source count Before R2010 a spline edge has no 97 of its own, so the 97 after it counts the loop's source entities. It was read as the edge's fit point count and the sources were lost. * craft-dxf: model::write, a drawing as tags (R12 to 2018) Drawing::tags writes the header, classes, the nine symbol tables, blocks, entities and (R2000 and later) the dictionaries, plot style placeholder and layouts around the layers, linetypes, styles, dimension styles, blocks and entities it is given, in the model's own types. entity_tags writes one entity for a caller with its own file. Handles given with the content are kept; the rest are numbered after them. The file skeleton is VectorCraft's vectorcraft-cad writer.rs (566f7208); the MTEXT column groups are CADCraft's dxf_read/mtext_dxf.rs column_tags (5a6aa9b6). * craft-dxf: model objects (page setup, groups, image definitions, draw order, clip filters, DIMASSOC, scales, multiline styles), HELIX and dimension overrides model::objects types the OBJECTS records both readers and CADCraft's passes use: the AcDbPlotSettings groups of a LAYOUT, GROUP, IMAGEDEF, SORTENTSTABLE, SPATIAL_FILTER, DIMASSOC, SCALE and MLINESTYLE, and dim_overrides reads the DSTYLE list of a dimension's ACAD extended data. Document gains objects_of and dictionary_names (the name a dictionary lists an object under). HELIX is an entity kind, read and written. From CADCraft's cadcraft-io dxf_read/objects.rs, document.rs, dimassoc.rs, entities.rs, dxf_mline.rs, dxf_clip.rs and dxf_image.rs (5a6aa9b6). * README: keep main's crate list after the merge |
||
|
Dylan M. Taylor
|
97f4971666 |
craft-pdf: the PDF filter chain from PdfCraft (filters), on craft-stream-codecs (#1033)
* Add craft-stream-codecs: the codec modules of PdfCraft's stream filters LZW, ASCII85, ASCIIHex, RunLength, the TIFF and PNG predictors, bounded CCITT fax decoding (through hayro-ccitt) with Group 4 encoding (through fax), and the lossless JBIG2 encoder with its MQ coder, from pdfcraft-filters at PdfCraft revision 80e26e674d1aa33078e83c8139875047f3289eb6. The crate sits below craft-pdf and craft-tiff: it knows no container. The PDF filter chain, Flate and the bounded JBIG2 decoding entry are not part of it. * craft-pdf: the PDF filter chain from PdfCraft (`filters`) Adds the `filters` feature: the /Filter chain of pdfcraft-filters (PdfCraft revision 80e26e674d1aa33078e83c8139875047f3289eb6) as craft_pdf::filters, over the codecs in craft-stream-codecs. It adds Flate, strict and tolerant chain decoding, byte accounting and a shared work allowance, and encoding, with the chain's unit tests, its property tests and its fuzz target. The code is unchanged apart from the imports of the moved codecs; FilterError is craft-stream-codecs' CodecError under its old name. |
||
|
|
a7a00c7b0a |
craft-raw: Olympus ORF (uncompressed) (#1032)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW A new crate for #587 and #778: bytes in, the sensor data out as a neutral RawImage (samples, CFA, levels, active area, crop, orientation, colour tags, opcode lists, profile tables), and back for DNG. One Cargo feature per format: dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw). The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw, unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile and gaintable, the model and probe. Demosaicing and everything after it stay with the develop pipeline. Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the default crop's per-tag defaults, an unusable crop falling back to the active area, non-finite white levels, the white default of a linearized image, the AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their part of the image, and the CinemaDNG frame tags. * craft-raw: Hasselblad 3FR and FFF (feature hasselblad) RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add. Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds, and the 8-bit RGB preview handed on as a baseline TIFF. Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests. Ported from PhotoCraft's reader: four BlackLevel values without a BlackLevelRepeatDim are one per CFA site in a file that is not a DNG. * craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak), moved from its vendor modules at 5ff567a4 with their tests; the MRW block container comes from lightcraft-meta. src/shared.rs holds the level helpers the maker decoders share. ERF and MEF state no white balance: the estimate LightCraft made from the preview is develop-side work, so wb_multipliers is None for them here. * craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal size is mostly padding outside the image is refused before it is inflated. * craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an image whose pixel bytes exceed the file and still takes a smaller complete one. * craft-raw: test for the Deflate padding bound * craft-raw: Pentax PEF and Samsung SRW Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module. * craft-raw: Canon CR2, with sRAW and mRAW Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs; the temperature and tint arithmetic stays with the developer). * craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview). * WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests) * craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level * craft-raw: Panasonic RW2 / RWL / RAW, every raw format * craft-raw: RW2 format 8 refuses strips stating fewer bits than samples Ported from storytold/lightcraft#3910. * craft-raw: RW2 format 8 refuses samples above the declared bit depth Ported from storytold/photocraft#11838. * craft-raw: the RW2 format 8 robustness seed carries a valid stream Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes. * craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections * craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits Ported from storytold/lightcraft#3910. * craft-raw: test RAF orientation and camera from the embedded JPEG's Exif * WIP craft-raw: Olympus ORF * craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview) * craft-raw: Olympus ORF, the uncompressed layouts, with its maker-note preview and WB_GLevel --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: wnanbei <wnanbei@gmail.com> |
||
|
Dylan M. Taylor
|
34aa8c1cfd |
craft-xlsx: read parts with canonical prefixes so kept trees can be written back (#1031)
The part layer read every part by local name, so extLst children (x14:, xr:, xm:), mc:AlternateContent and r:id lost their namespaces and could not be written back. Parts are now read with one fixed prefix per namespace (the main namespace unprefixed), mc:Ignorable and mc:Choice/@Requires are rewritten to match, and ns::declare prepares a kept tree for writing. craft-xml: a namespace listed with the empty prefix reads unprefixed. |
||
|
Dylan M. Taylor
|
1e3d3968c1 |
Add craft-markdown: Markdown read and written, on craft-flow (stacked on #1025) (#1030)
* Add craft-wordmodel: WordCraft's document model without its editing code WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX, RTF, ODT and other text-format crates can read into and write from one model. Module paths, type names and serde attributes are unchanged: an app that re-exports the crate keeps its native file format. Left in the app: range editing (edit.rs), IME text and equation editing. Not model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs. * craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs The DOCX reader and writer walk an equation's runs. The walk and the child accessors it uses were in WordCraft's equation editor (math_edit.rs), which stays in the app; they are structure, not editing. * craft-wordmodel: keep the document operations (edit.rs) with the model The DOCX and RTF readers use them: alternative-format chunks are merged into a document with adopt_blocks, and a reader formats ranges with format_range and inserts objects. They operate on the document, not on a view, so they belong beside it. The tests that drove them come back with them; the paragraph walks stay in walk.rs and edit re-exports them. base64 0.23, as WordCraft and craft-offcrypto use. * Add craft-flow: the flow model the text formats map to, on craft-wordmodel WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow, FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and from_doc, picture sniffing, colours, base64 and the Dropped report. Only the paths changed (wordcraft_doc is craft_wordmodel); the report helpers are public so the format crates can call them; the image re-encoding helper stays with WordCraft's RTF writer. * Add craft-markdown: Markdown read and written, on craft-flow WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests, moved onto craft-flow and craft-wordmodel. Character references come from craft-html. The table cell budget lives in craft-flow, which names it in its warning. Two test lines that wrote plain text through WordCraft's txt module stay in WordCraft. |
||
|
Dylan M. TaylorandJordi Altayó
|
9eb25f7139 |
craft-drawingml: extra named theme colours and tint rows (from DeckCraft #1746) (#1029)
* Add craft-drawingml: units, colours, preset shapes, custom geometry and text warps The parts of DrawingML that need no document model, moved from DeckCraft (60f91f78): crates/color, crates/geom (preset, warp, Xfrm and the EMU conversions) and crates/pptx/src/custgeom.rs with CustomPath from crates/model. The code is unchanged except that custgeom reads craft-xml trees and writes to a craft-xml Writer instead of DeckCraft's own XML helper, and matches elements by local name. The 59 tests of the moved code come with it; six more cover the custom geometry limits. * drawingml: evaluate a custom geometry's guide lists once per coordinate space read_cust_geom rebuilt Guides and evaluated avLst and gdLst again for every path: up to 1,000 paths times up to 10,000 guides, each name copied and hashed. Guides are now cached per path coordinate space (w, h). At most 16 spaces and 4 MiB of guide names are kept per geometry; paths in further spaces are left out. The first space is always kept, and a geometry without guides is not capped. Ported from storytold/deckcraft#1789. A list counts as having guides only when it has gd children, as that PR's review asked. * drawingml: the model types for fills, lines, text, tables, charts and diagrams Moved from DeckCraft (60f91f78) crates/model/src: style.rs, text.rs, table.rs, chart.rs, numfmt.rs and trendline.rs, plus Diagram, DiagramNode and the MediaId and SlideId newtypes from lib.rs. The files are unchanged except that deckcraft_color:: is crate::color:: and Fill and TableBorder allow clippy::large_enum_variant. Their 60 tests come with them. Readers and writers follow. * drawingml: the theme model (colour, font and format schemes) Moved from DeckCraft crates/model/src/theme.rs at 60f91f78: FontSet, ScriptFont, FontScheme, FormatScheme, Theme and default_format. The app's theme gallery, its variants and its own default theme stay in the app. Theme::default() is a plain theme built in place, so nothing here depends on a gallery. * drawingml: readers and writers for fills, lines, text, charts, themes, table styles and diagrams Moved from DeckCraft crates/pptx/src at 60f91f78: read/dml.rs, write/dml.rs, read/chart.rs, read/chartex.rs, write/chart.rs, read/theme.rs, write/theme.rs, tables.rs and dgm.rs, onto craft-xml. Package lookups (Imp, Part, Exp, Out) are the caller's part::Source and part::Sink. The chart frame budget and kept-part capture stay with the presentation reader. * drawingml: what WordCraft and GridCraft read that DeckCraft's copy did not Every ST_PresetColorVal name has its colour, in any case and with the dk/lt/med abbreviations; a sysClr without lastClr takes the usual value of its name; a Strict percentage (50%) reads anywhere a percentage does; a chart series keeps its c:f formulas and the writer uses them. * craft-drawingml: extra named theme colours and explicit tint rows Ported from the format part of storytold/deckcraft#1746 by @jagjordi: ColorScheme::{extra, tints, computed_tints}, ExtraColor, GalleryVariant, variants_of, and the a:clrScheme extension that keeps them in a file. * drawingml xmlx: copy only used, bounded namespace declarations onto kept fragments and mc branches From storytold/deckcraft#1740 (988a4920), which landed after the extraction commit 60f91f78. --------- Co-authored-by: Jordi Altayó <jordiag@kth.se> |
||
|
Dylan M. Taylor
|
aa84415bae |
craft-gif, craft-webp: bounded frame decoding through the image crate (feature frames) (#1028)
* craft-gif: bounded frame decoding through the image crate (feature frames) * craft-webp: bounded frame decoding through the image crate (feature frames) Like craft-gif's: every frame composited, a still as one frame, canvas cap checked from the headers, stops at the frame or byte limit or at a damaged frame and says which, optional merging of identical frames. |
||
|
Dylan M. Taylor
|
12917a6cf3 |
craft-stream-codecs: bounded JBIG2 decoding (decode_jbig2_within), on #886 (#1026)
* hayro-jbig2: bound a stream's total decode work and what a page retains; no i32 overflow Port of pdfcraft 85c10051 (patches 5-9), 82439525 (patch 10) and 10107d94 (patch 12). * hayro-jbig2: PdfCraft's JBIG2 follow-ups (refinement fixes, file header, work accounting, dictionary pixel cap) Ports pdfcraft 72e763ad, b7ec3669, e3bc4e85, 01d3479f, b9e068aa, 858ce960, 07f44038, e38b9d18, 8717a085, 58c30af3 and the hayro-jbig2 part of e2464e14 (with the hayro-jbig2 part of 0ed33990 they build on). * Add craft-stream-codecs: the codec modules of PdfCraft's stream filters LZW, ASCII85, ASCIIHex, RunLength, the TIFF and PNG predictors, bounded CCITT fax decoding (through hayro-ccitt) with Group 4 encoding (through fax), and the lossless JBIG2 encoder with its MQ coder, from pdfcraft-filters at PdfCraft revision 80e26e674d1aa33078e83c8139875047f3289eb6. The crate sits below craft-pdf and craft-tiff: it knows no container. The PDF filter chain, Flate and the bounded JBIG2 decoding entry are not part of it. * craft-stream-codecs: bounded JBIG2 decoding (decode_jbig2_within) The JBIG2 decoding entry of pdfcraft-filters (PdfCraft revision 80e26e674d1aa33078e83c8139875047f3289eb6, src/jbig2/decode.rs) with its tests. It spends from the caller's work allowance through hayro-jbig2's Image::decode_within and charge_reading, which come with fix/sec-hayro-jbig2-followups (#886). * vendor/README.craft-reader.md: one #831 paragraph, the follow-ups after it |
||
|
Dylan M. Taylor
|
baf6bd58f0 |
Add craft-flow: the flow model the text formats map to (#1025)
* Add craft-wordmodel: WordCraft's document model without its editing code WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX, RTF, ODT and other text-format crates can read into and write from one model. Module paths, type names and serde attributes are unchanged: an app that re-exports the crate keeps its native file format. Left in the app: range editing (edit.rs), IME text and equation editing. Not model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs. * craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs The DOCX reader and writer walk an equation's runs. The walk and the child accessors it uses were in WordCraft's equation editor (math_edit.rs), which stays in the app; they are structure, not editing. * craft-wordmodel: keep the document operations (edit.rs) with the model The DOCX and RTF readers use them: alternative-format chunks are merged into a document with adopt_blocks, and a reader formats ranges with format_range and inserts objects. They operate on the document, not on a view, so they belong beside it. The tests that drove them come back with them; the paragraph walks stay in walk.rs and edit re-exports them. base64 0.23, as WordCraft and craft-offcrypto use. * Add craft-flow: the flow model the text formats map to, on craft-wordmodel WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow, FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and from_doc, picture sniffing, colours, base64 and the Dropped report. Only the paths changed (wordcraft_doc is craft_wordmodel); the report helpers are public so the format crates can call them; the image re-encoding helper stays with WordCraft's RTF writer. |
||
|
|
336619e3e4 |
craft-psd: fixes ported from open PhotoCraft pull requests (#1024)
* craft-psd: raise the 64-bit .abr sample budget to 2 GiB Photoshop writes brush packs up to its 2 GB .abr limit; a 1 GiB budget refused valid packs over it. 32-bit targets keep 512 MiB. Ported from storytold/photocraft#13282 (photocraft#5949). * craft-psd: FELS is checked and transcoded as filter effects FELS holds the smart-filter masks of linked smart objects in the FEid layout. check_structure now re-parses it, and a layered TIFF transcodes it field by field: a little-endian save dropped the block as unknown. Ported from storytold/photocraft#8275 (photocraft#8260); the test is new. * craft-psd: a layered TIFF keeps external smart-object links An external linked file (liFE) holds the descriptor of its link, and from version 3 its bytes. The TIFF transcoder skipped both, so the record failed to convert and a little-endian save dropped the lnkE block. Ported from storytold/photocraft#6072; the test is new. * craft-psd: the example and the save test write into fresh private directories The big_psb example and the save_and_open test used a predictable temp directory name, which a planted directory or symlink could take over. They now create a new owner-only directory, and the example removes a half-written fixture from its own default folder. Ported from storytold/photocraft#13266 (its crates/psd part). * craft-psd: the pixel preflight sees the merged image's compression and size from_bytes_and_merged_preflight_info and from_source_and_merged_preflight_info hand the preflight the merged image's compression and encoded length before the image is read, so a caller can estimate the memory a decode will need (an opaque RGB image stored raw needs no inflate buffer). A missing compression field is None, and a refusal takes precedence over it. Ported from storytold/photocraft#11288 (its crates/psd part); the test is new. * craft-psd: typed layer selection ids (resource 1069) ResourceData::LayerSelectionIds: the selected layers' ids in stored order, duplicates and zero kept. The count is checked against the whole payload before anything is allocated; truncated or trailing data is malformed, not a partial selection. Ported from storytold/photocraft#10268 (its crates/psd part). * craft-psd: README matches the code The typed blocks tsly, lmgm and vmgm, vector masks and slices typed by path and slices, the resources ImageResource::parsed types, the ATN round-trip caveats and version-12 reading, and the corpus test's known-bad file. Ported from storytold/photocraft#11600 (its crates/psd part), with the resource list brought up to date. * craft-psd: atn_dump escapes control characters from the file A set's names, warnings, paths and errors are printed with control characters escaped, so a file cannot drive the terminal. Ported from storytold/photocraft#11218 (merged into PhotoCraft's copy after craft-psd was moved). * craft-psd: the .abr budget test checks the 64-bit bound at compile time clippy refuses an assertion whose value is constant. --------- Co-authored-by: rgoza1996 <97354241+rgoza1996@users.noreply.github.com> Co-authored-by: Korg1ch <57859048+Korg1ch@users.noreply.github.com> Co-authored-by: PJaYOfficial <1597623+PJaYOfficial@users.noreply.github.com> Co-authored-by: acilione <antoninocilione96@gmail.com> Co-authored-by: kirubhakaran12 <27127513+kirubhakaran12@users.noreply.github.com> Co-authored-by: Nicolas Falesy <112726756+nicolasfalesy@users.noreply.github.com> |
||
|
|
df469bb2bd |
craft-metadata: byte-preserving Exif and XMP edits for export, from PhotoCraft (#1022)
* craft-metadata: EXIF, IPTC-IIM, XMP and GPX from LightCraft lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif (read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse, write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers. They share Metadata, DateTime, Gps and Orientation. Changed in the move: - TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own. - The app namespace is a parameter: parse_xmp_with, write_xmp_with and MergeRules::namespaces take it, and XmpData no longer has lc_settings. - The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and craft-raw read those containers; from_embedded takes the blocks they find. * craft-metadata: a RW2 reads ISO and the rest of its Exif from the JpgFromRaw A RW2's own Exif IFD is a short subset. What it lacks is filled from the Exif of the embedded JpgFromRaw, then from the ISO tags of the Panasonic IFD0 (0x0017, 0x0037), each only in its recorded field type. 65535 in a SHORT ISOSpeedRatings is the overflow marker and states no ISO. The private tags are read only in a RW2 stream, under the caller's parse options. Ported from storytold/lightcraft#3518. * craft-metadata: the Exif rating is written, and 65535 reads as rejected write_exif writes Rating (0x4746), with 0xFFFF for a rejected photo, and from_tiff reads 65535 as -1 instead of clamping it to 5. Ported from storytold/lightcraft#1995. * craft-metadata: one shutter-speed formatter, exact to four digits format_exposure_time shows a duration within 0.1% of 1/n second as 1/n and anything else to four significant digits, so 0.3 s, 1/2.8 s and 0.625 s are no longer rounded to a reciprocal or one decimal. Metadata::exposure_display and the ExposureTime tag row both use it. Ported from storytold/lightcraft#1692. * craft-metadata: JPEG XL containers give their Exif and xml boxes The first Exif and xml box of a JPEG XL container, plain or Brotli-wrapped in brob, honouring the Exif box's TIFF header offset. A brob payload is capped at 32 MiB. A bare codestream has no metadata. Ported from storytold/lightcraft#3756. * craft-metadata: XMP Resource structs flatten their attribute fields A property with rdf:parseType="Resource" may give its fields as attributes as well as child elements. Only the children were flattened into XmpData::properties; the attributes now are too, as for a nested rdf:Description. Ported from storytold/lightcraft#1226 (the XMP part). * craft-metadata: a capture date of just a year or a month stays as stated XMP dates may stop at the year (1983) or the month (1983-04). DateTime now has a DatePrecision; such a date writes back to XMP as stated and only in photoshop:DateCreated, and write_exif leaves it out, since an Exif date cannot say it. Ported from storytold/lightcraft#3502. * craft-metadata: Fujifilm in-camera recipes from the maker note fuji module: the film simulation and the rest of the in-camera recipe (grain, colour chrome, tone, dynamic range, white balance and its shift, sharpness, noise reduction, clarity) from a Fujifilm maker note. recipe_from_exif, recipe_from_bytes (TIFF, JPEG, RAF) and read_recipe_file, which reads only as far as the metadata. from_embedded fills Metadata::fuji_recipe and file_tag_rows lists the recipe under Fujifilm. Ported from storytold/lightcraft#1209. * craft-metadata: the linear-time scan test compares times, not a fixed bound The test asserted that 200 000 attributes scan in under 5 s, which failed on a loaded machine. It now scans 25 000 and 100 000 attributes, takes the best of three runs of each, and asserts that four times the attributes take less than ten times as long. Linear scans give about 4; a quadratic duplicate check gave 18.6 and fails. * craft-metadata: interleave the linear-time scan test's runs The best-of-three ratio still failed once under load. The test now alternates five runs of 50 000 and 200 000 attributes and compares the fastest of each, so a burst of load slows both sizes. Six runs at load average 55 gave ratios of 3.9 to 5.4 against the bound of 10. * craft-metadata: XmpData::toolkit, the x:xmptk of the packet The name of the program that wrote a packet, from x:xmptk on x:xmpmeta (or x:xaptk on the older x:xapmeta), whatever prefix the packet binds to adobe:ns:meta/. write_xmp_with writes XmpWrite::toolkit there, and the test reads it back, escaped characters included. * craft-metadata: mpf, the Multi-Picture index of a JPEG mpf::images lists the images an APP2 MPF index (CIPA DC-007) puts after the primary image, with their file offsets; the index is read with craft-tiff, in either byte order, up to 16 entries. mpf::write inserts an index after the primary's leading APPn segments and appends the other images, with the primary listed as the representative baseline image: the layout LightCraft's gain map JPEG writer uses, for any number of images. mpf::markers lists the marker segments before the first scan, for APPn payloads such as ISO 21496-1 gain map metadata. * craft-metadata: edit, byte-preserving Exif and XMP edits from PhotoCraft PhotoCraft's orientation, resolution and Exif colour-space code (codecs orientation.rs without the pixel transforms, resolution.rs, exif_color.rs), at photocraft f1b5e7d72, as the edit module: - upright_exif and upright_xmp set Orientation to 1 in place, in the entry's own type and width, every well-formed duplicate included. - exif_with_resolution and xmp_with_resolution set the resolution in place and remove entries that cannot be rewritten; the _adding_missing variant appends a new IFD0 when there are none. - export_exif and export_xmp do both and drop the source Exif thumbnail, with the overlap scan bounded. - exif_orientation, exif_resolution, xmp_resolution, photoshop_resolution, exif_color_space and resolution read the same values with the editors' rules (first entry wins, SHORT or LONG only, BigTIFF accepted). The tests that call these functions directly move with them (the codec round trips stay in PhotoCraft): the orientation, resolution, thumbnail and compaction tests, the comparison-counting complexity test and the colour space tests. * craft-metadata: exif_with_pixel_size, the written size in PixelX/YDimension A camera records its sensor's size in the Exif IFD's PixelXDimension and PixelYDimension, which after a crop, resize or rotation names a size the exported file does not have. exif_with_pixel_size rewrites each one that holds one SHORT or LONG, in place and as a LONG, in classic and BigTIFF blocks, with the JPEG Exif prefix kept. Ported from storytold/photocraft#12407 (the format part). --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: Sebastian Berschneider <berschi@gmail.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Michael Anderson <1427851+endthestart@users.noreply.github.com> Co-authored-by: Ilya Dorman <ilya@codeforhumans.dev> Co-authored-by: Clyde C Brown <clyde.c.brown@pm.me> Co-authored-by: MohammadTeimori <teimori.mohammad@gmail.com> Co-authored-by: vangel76 <vangel1976@googlemail.com> |
||
|
Dylan M. TaylorandJustin A. Wilson
|
45177e282e |
craft-webp: animated WebP writer, lossy through craft-video's VP8 encoder (depends on #969) (#1021)
* craft-video: VP8 key-frame encoder (`vp8-encode` feature) Moves effectcraft's effectcraft-vp8enc here: the lossy picture of a WebP image or animation frame, written from RFC 6386. Rate-distortion choice between the 16x16 and B_PRED intra modes, per-frame token probability updates, loop filter level from the quantizer. `vp8` now turns on `vp8-decode` and `vp8-encode`, as `vp9` does; the decoder's tests and examples require `vp8-decode`. The tests decode the encoder's frames with this crate's decoder: with the loop filter off it reproduces the encoder's reconstruction exactly. * craft-webp: animated WebP writer, lossy through craft-video (feature vp8) The VP8X/ANIM/ANMF container around frames another encoder made (lossy with an alpha plane, or lossless VP8L), and with feature vp8 whole lossy animations from RGBA frames through craft-video's VP8 key-frame encoder. --------- Co-authored-by: Justin A. Wilson <waj334@gmail.com> |
||
|
Dylan M. Taylorandshuff57
|
22161d2d6a |
craft-math: operator names take their LaTeX scripts directly (\sin^2\theta round trip) (#1019)
* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath
The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.
mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.
* craft-math: read MathLive's and other spellings of LaTeX math commands
\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).
* craft-math: equaSHion's equation corpus as test data
corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (
|
||
|
Dylan M. Taylorandljkx
|
24903d320d |
craft-idml: resources model (styles, swatches, stroke styles, colour groups, preferences) (#1017)
* craft-idml: a typed model of the layout (spreads, pages, guides, page items, graphics, geometry) The first step of the typed IDML model. Spreads and parent spreads, pages, guides, page items (nested), placed graphics and their links are typed; each keeps its attributes and Properties as written and reads them through accessors, path geometry among them. Elements the model does not type are kept where they stood, so a node read and written again reads the same. * craft-idml: the story model (ranges, content, tables, notes, anchored items) Stories as IDML writes them: paragraph and character ranges, Content with the special characters InDesign writes as <?ACE n?>, breaks, the containers text sits in (hyperlink sources, tracked changes, XML elements, cross-reference sources, footnotes, notes), tables with their cells, and page items anchored in the text. Each node keeps its values as written; text is gathered without recursion. * craft-idml: read and write ICML (InCopy Markup) The format layer of storytold/designcraft#2248: an ICML file told by its InCopyInterchange instruction, read within the size of one package part, its references to package parts left out with a warning, a file without a story refused, and written with the two instructions that mark it. Choosing the story to place and mapping it stay with the application. * craft-idml: the resources model (style groups, swatches, stroke styles, colour groups, preferences) Style groups as trees of groups and styles for each kind, each style with its group path, BasedOn and NextStyle; colours, tints, gradients with their stops, mixed inks and the None swatch; inks, custom stroke styles, colour groups; and the preference elements by name. Values are kept as written and read through accessors. --------- Co-authored-by: ljkx <177671075+ljkx@users.noreply.github.com> |
||
|
|
b471b84fd4 |
craft-raw: Fujifilm RAF (#1016)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW A new crate for #587 and #778: bytes in, the sensor data out as a neutral RawImage (samples, CFA, levels, active area, crop, orientation, colour tags, opcode lists, profile tables), and back for DNG. One Cargo feature per format: dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw). The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw, unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile and gaintable, the model and probe. Demosaicing and everything after it stay with the develop pipeline. Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the default crop's per-tag defaults, an unusable crop falling back to the active area, non-finite white levels, the white default of a linearized image, the AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their part of the image, and the CinemaDNG frame tags. * craft-raw: Hasselblad 3FR and FFF (feature hasselblad) RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add. Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds, and the 8-bit RGB preview handed on as a baseline TIFF. Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests. Ported from PhotoCraft's reader: four BlackLevel values without a BlackLevelRepeatDim are one per CFA site in a file that is not a DNG. * craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak), moved from its vendor modules at 5ff567a4 with their tests; the MRW block container comes from lightcraft-meta. src/shared.rs holds the level helpers the maker decoders share. ERF and MEF state no white balance: the estimate LightCraft made from the preview is develop-side work, so wb_multipliers is None for them here. * craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal size is mostly padding outside the image is refused before it is inflated. * craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an image whose pixel bytes exceed the file and still takes a smaller complete one. * craft-raw: test for the Deflate padding bound * craft-raw: Pentax PEF and Samsung SRW Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module. * craft-raw: Canon CR2, with sRAW and mRAW Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs; the temperature and tint arithmetic stays with the developer). * craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview). * WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests) * craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level * craft-raw: Panasonic RW2 / RWL / RAW, every raw format * craft-raw: RW2 format 8 refuses strips stating fewer bits than samples Ported from storytold/lightcraft#3910. * craft-raw: RW2 format 8 refuses samples above the declared bit depth Ported from storytold/photocraft#11838. * craft-raw: the RW2 format 8 robustness seed carries a valid stream Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes. * craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections * craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits Ported from storytold/lightcraft#3910. * craft-raw: test RAF orientation and camera from the embedded JPEG's Exif * craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview) --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: wnanbei <wnanbei@gmail.com> |
||
|
Dylan M. Taylor
|
e3e31693c7 |
Add craft-docx: WordCraft's DOCX reader and writer on craft-wordmodel, craft-opc and craft-xml (#1015)
* Add craft-wordmodel: WordCraft's document model without its editing code WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX, RTF, ODT and other text-format crates can read into and write from one model. Module paths, type names and serde attributes are unchanged: an app that re-exports the crate keeps its native file format. Left in the app: range editing (edit.rs), IME text and equation editing. Not model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs. * craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs The DOCX reader and writer walk an equation's runs. The walk and the child accessors it uses were in WordCraft's equation editor (math_edit.rs), which stays in the app; they are structure, not editing. * craft-wordmodel: keep the document operations (edit.rs) with the model The DOCX and RTF readers use them: alternative-format chunks are merged into a document with adopt_blocks, and a reader formats ranges with format_range and inserts objects. They operate on the document, not on a view, so they belong beside it. The tests that drove them come back with them; the paragraph walks stay in walk.rs and edit re-exports them. base64 0.23, as WordCraft and craft-offcrypto use. * craft-docx: WordCraft's crates/docx as it is at wordcraft 07f97737 The source, unchanged, so the next commit shows only what the move changes. This commit does not build on its own (it still names wordcraft-doc and wordcraft-geom, and is not a workspace member yet). * craft-docx: on craft-wordmodel, craft-opc, craft-xml and craft-offcrypto - The package (zip reading, part names, relationships, content types, zip writing) is craft-opc's; the module keeps WordCraft's bookkeeping (passthrough keys, the embedded-part manifest) and its own Rel/Rels shape. - Parts are parsed with craft-xml's canonical names (the Strict drawing names and mc:Choice/@Requires through its hooks), then given the tree shape the reader was written against. - Encryption and Restrict Editing hashes are craft-offcrypto's; this crate maps its errors and the w:documentProtection attributes. - wordcraft-doc is craft-wordmodel; wordcraft-geom's helpers are its geom. Tests that covered what now lives in craft-opc and craft-offcrypto are left to those crates; the DIFAT regression is craft-cfb's. Test-only zip use follows zip 9's fallible names. The attribute-cap test times the read only. |
||
|
Dylan M. Taylor
|
4586109726 |
craft-psd: Photoshop's layered TIFF conventions, from PhotoCraft (#1014)
The format parts of PhotoCraft's layered TIFF (crates/io/src/tiff_layers.rs at abeb16c96) that sit beside craft-psd's tiff module: the two tag numbers; the layer data Photoshop writes (empty Patt, default FMsk, MTrn, a full LMsk record, flag bit 3 and the per-layer blocks) and its removal on open; the channel resources (1006, 1022, 1045, 1053, 1077) matched to the samples the IFD stores; foreign-order block stripping; and the interleaved to planar sample conversions with CMYK inversion. |
||
|
Dylan M. Taylorandshuff57
|
85b4dabf0f |
Add craft-math: equations as one tree with OMML, MathML, LaTeX and UnicodeMath (#1013)
* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath
The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (storytold/craft-libs#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.
mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.
* craft-math: read MathLive's and other spellings of LaTeX math commands
\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (storytold/craft-libs#29).
* craft-math: equaSHion's equation corpus as test data
corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
storytold/craft-libs#29 (
|
||
|
Dylan M. Taylor
|
dc14024a19 |
craft-sketch: map the layers onto the REST file JSON (#1012)
* Add craft-figma-rest: a typed model of the Figma REST file JSON
The neutral model for design files: the .fig and .sketch readers map onto
it and an application's importer takes it. Typed records for the document
tree, paints, effects, text styles, layout grids, constraints, export
settings and the component and style metadata; every record keeps the
fields it has no name for, so a file read and written loses nothing.
The vectorNetwork extension is typed and documented with the others.
The layer tree is converted level by level in both directions, and the
depth and node limits are checked on the JSON before any typed value is
built.
* craft-figma-rest: Warnings and style_name for the readers that build this shape
Moved from FigCraft's REST importer and .fig reader (crates/format/src/rest.rs
Warnings, fig.rs style_name), which the .fig and Sketch mappings share.
* craft-sketch: map the layers onto the REST file JSON
FigCraft's Sketch mapping (crates/format/src/sketch/{tree,style,text,
symbols,path,shared,stack,proto}.rs at e8b53e33) moved onto
craft-figma-rest: rest::to_rest turns an Archive's pages and layers into
the REST API file JSON, with the vector networks of its paths handed over
beside it as craft_figma_rest::ext::VectorNetwork.
The star and polygon point generators it compares a path with are copied
from FigCraft's geometry; comments and two warnings no longer name the
app.
* craft-sketch: reach the nesting limit with nested groups, not 210 masters
The same limit and warning, in 1 second instead of 58 in a debug build.
* craft-sketch: a nesting limit that fits a 1 MiB stack, in small frames
The mapping recurses once per level of layer nesting, symbol expansion
included. Nested symbol instances reach its limit with shallow page JSON,
and a debug build overflowed a 2 MiB stack at about 140 levels: a crash
on hostile input.
The work on one layer, master or instance now happens in functions that
return before its children are mapped, which takes the stack per level
from about 4.5 KB to about 2 KB for an instance and its group in a debug
build. That maps about 140 levels of nested instances on 512 KiB, so the
limit is now 128 (was 200), leaving twice the room on the 1 MiB stack of
a WebAssembly build. The test at the limit runs on a 1 MiB thread.
* Cargo.lock: craft-sketch's craft-figma-rest and kurbo after the merge with main
|
||
|
Dylan M. Taylorandjayzed80
|
227cfee7e0 |
craft-video: DeckCraft decoder fixes (HEVC queued-output budget, VP9 failed superframes, AV1 tile metadata cap, H.264 POC overflow) (#1010)
* craft-video: budget HEVC queued output, drop a failed VP9 superframe's repeats, serial AV1 tiles past a metadata cap Ported from storytold/deckcraft#1778 and #1841, which landed in DeckCraft's decoder copies after craft-video took them. * craft-video: H.264 picture order count no longer overflows on extreme stream values Wrapping arithmetic through the POC type 0/1/2 paths, frame_num_offset and the MMCO5 prev_ref_lsb update. Ported from storytold/deckcraft#1913. --------- Co-authored-by: jayzed80 <jzinkant@gmail.com> |
||
|
Dylan M. Taylorandljkx
|
c6ded901fb |
craft-idml: story model and ICML (InCopy Markup) (#1008)
* craft-idml: a typed model of the layout (spreads, pages, guides, page items, graphics, geometry) The first step of the typed IDML model. Spreads and parent spreads, pages, guides, page items (nested), placed graphics and their links are typed; each keeps its attributes and Properties as written and reads them through accessors, path geometry among them. Elements the model does not type are kept where they stood, so a node read and written again reads the same. * craft-idml: the story model (ranges, content, tables, notes, anchored items) Stories as IDML writes them: paragraph and character ranges, Content with the special characters InDesign writes as <?ACE n?>, breaks, the containers text sits in (hyperlink sources, tracked changes, XML elements, cross-reference sources, footnotes, notes), tables with their cells, and page items anchored in the text. Each node keeps its values as written; text is gathered without recursion. * craft-idml: read and write ICML (InCopy Markup) The format layer of storytold/designcraft#2248: an ICML file told by its InCopyInterchange instruction, read within the size of one package part, its references to package parts left out with a warning, a file without a story refused, and written with the two instructions that mark it. Choosing the story to place and mapping it stay with the application. --------- Co-authored-by: ljkx <177671075+ljkx@users.noreply.github.com> |
||
|
|
06aee09c5c |
craft-metadata: MPF index reader and writer for gain map JPEGs; XmpData::toolkit (#1007)
* craft-metadata: EXIF, IPTC-IIM, XMP and GPX from LightCraft lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif (read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse, write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers. They share Metadata, DateTime, Gps and Orientation. Changed in the move: - TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own. - The app namespace is a parameter: parse_xmp_with, write_xmp_with and MergeRules::namespaces take it, and XmpData no longer has lc_settings. - The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and craft-raw read those containers; from_embedded takes the blocks they find. * craft-metadata: a RW2 reads ISO and the rest of its Exif from the JpgFromRaw A RW2's own Exif IFD is a short subset. What it lacks is filled from the Exif of the embedded JpgFromRaw, then from the ISO tags of the Panasonic IFD0 (0x0017, 0x0037), each only in its recorded field type. 65535 in a SHORT ISOSpeedRatings is the overflow marker and states no ISO. The private tags are read only in a RW2 stream, under the caller's parse options. Ported from storytold/lightcraft#3518. * craft-metadata: the Exif rating is written, and 65535 reads as rejected write_exif writes Rating (0x4746), with 0xFFFF for a rejected photo, and from_tiff reads 65535 as -1 instead of clamping it to 5. Ported from storytold/lightcraft#1995. * craft-metadata: one shutter-speed formatter, exact to four digits format_exposure_time shows a duration within 0.1% of 1/n second as 1/n and anything else to four significant digits, so 0.3 s, 1/2.8 s and 0.625 s are no longer rounded to a reciprocal or one decimal. Metadata::exposure_display and the ExposureTime tag row both use it. Ported from storytold/lightcraft#1692. * craft-metadata: JPEG XL containers give their Exif and xml boxes The first Exif and xml box of a JPEG XL container, plain or Brotli-wrapped in brob, honouring the Exif box's TIFF header offset. A brob payload is capped at 32 MiB. A bare codestream has no metadata. Ported from storytold/lightcraft#3756. * craft-metadata: XMP Resource structs flatten their attribute fields A property with rdf:parseType="Resource" may give its fields as attributes as well as child elements. Only the children were flattened into XmpData::properties; the attributes now are too, as for a nested rdf:Description. Ported from storytold/lightcraft#1226 (the XMP part). * craft-metadata: a capture date of just a year or a month stays as stated XMP dates may stop at the year (1983) or the month (1983-04). DateTime now has a DatePrecision; such a date writes back to XMP as stated and only in photoshop:DateCreated, and write_exif leaves it out, since an Exif date cannot say it. Ported from storytold/lightcraft#3502. * craft-metadata: Fujifilm in-camera recipes from the maker note fuji module: the film simulation and the rest of the in-camera recipe (grain, colour chrome, tone, dynamic range, white balance and its shift, sharpness, noise reduction, clarity) from a Fujifilm maker note. recipe_from_exif, recipe_from_bytes (TIFF, JPEG, RAF) and read_recipe_file, which reads only as far as the metadata. from_embedded fills Metadata::fuji_recipe and file_tag_rows lists the recipe under Fujifilm. Ported from storytold/lightcraft#1209. * craft-metadata: the linear-time scan test compares times, not a fixed bound The test asserted that 200 000 attributes scan in under 5 s, which failed on a loaded machine. It now scans 25 000 and 100 000 attributes, takes the best of three runs of each, and asserts that four times the attributes take less than ten times as long. Linear scans give about 4; a quadratic duplicate check gave 18.6 and fails. * craft-metadata: interleave the linear-time scan test's runs The best-of-three ratio still failed once under load. The test now alternates five runs of 50 000 and 200 000 attributes and compares the fastest of each, so a burst of load slows both sizes. Six runs at load average 55 gave ratios of 3.9 to 5.4 against the bound of 10. * craft-metadata: XmpData::toolkit, the x:xmptk of the packet The name of the program that wrote a packet, from x:xmptk on x:xmpmeta (or x:xaptk on the older x:xapmeta), whatever prefix the packet binds to adobe:ns:meta/. write_xmp_with writes XmpWrite::toolkit there, and the test reads it back, escaped characters included. * craft-metadata: mpf, the Multi-Picture index of a JPEG mpf::images lists the images an APP2 MPF index (CIPA DC-007) puts after the primary image, with their file offsets; the index is read with craft-tiff, in either byte order, up to 16 entries. mpf::write inserts an index after the primary's leading APPn segments and appends the other images, with the primary listed as the representative baseline image: the layout LightCraft's gain map JPEG writer uses, for any number of images. mpf::markers lists the marker segments before the first scan, for APPn payloads such as ISO 21496-1 gain map metadata. --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: Sebastian Berschneider <berschi@gmail.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Michael Anderson <1427851+endthestart@users.noreply.github.com> Co-authored-by: Ilya Dorman <ilya@codeforhumans.dev> Co-authored-by: Clyde C Brown <clyde.c.brown@pm.me> Co-authored-by: MohammadTeimori <teimori.mohammad@gmail.com> |
||
|
Dylan M. TaylorandOmar Saidahmed
|
417680b824 |
craft-psd: float pixels, Lab composites and salvaging a damaged layer section (#1006)
* craft-psd: Photoshop PSD/PSB reader and writer, moved from PhotoCraft PhotoCraft's `photocraft-psd` crate (crates/psd at storytold/photocraft 63b641b55e54bf8f5c83efd977fdb16a2c255431) as `craft-psd`: the lossless PSD/PSB model, reader, writer and builder, descriptors, the Photoshop resource files that share its primitives (.abr, .asl, .aco, .act, .ase, .grd, .pat, .atn, .csh, .shc) and the Photoshop blocks of a layered TIFF. The crate had no dependency on any other PhotoCraft crate. Code is unchanged apart from the crate name, craft-libs' rustfmt width, comments that named other PhotoCraft crates, issue references (now `photocraft#N`), and the corpus tests' directory (`CRAFT_PSD_CORPUS` or `corpus/` at the workspace root). Tests, proptests, fuzz targets and fixtures move with it. Part of #778, #786. * craft-psd: engine_data, the text engine document of type layers The EngineData reader and writer from PhotoCraft's photocraft-text (crates/text/src/engine_data.rs at storytold/photocraft 63b641b5): a tree of values that is written back in Photoshop's layout, with strings stored as plain bytes kept as plain bytes. From EffectCraft's effectcraft-psd (crates/psd/src/engine_data.rs at storytold/effectcraft 072e1af9), ported onto that tree: `first_style`, the first style run as a typed `TextStyle` with bounded sizes and spacings, and `minimal`, a one-run document for writers and fixtures. `Value::index` is new. Both apps had their own parser for the same syntax. * craft-psd: engine_data::parse_lossy reads what it can of a damaged document EffectCraft's EngineData reader was best effort: a truncated or damaged document still gave the text and style it held. `parse_lossy` does that on the same parser: a truncated dictionary, array or string keeps what it has, a byte that starts no key is skipped, a token that is no number becomes a name, and garbage gives an empty dictionary. `parse` stays strict and is unchanged in what it accepts. The parser's `value` is split into one function per kind of value, and reads with `get` instead of indexing. * craft-psd: typed views of type layers, smart objects and adjustments Read-only views EffectCraft's effectcraft-psd had over the blocks this crate keeps raw (storytold/effectcraft f334bcf3, crates/psd/src/parse.rs and placed.rs). The blocks stay raw and write back unchanged. - `text`: `TypeLayer` from `TySh`: text, transform (the identity when not finite), bounds, the first style run (read with `engine_data::parse_lossy`) and the text descriptor. `LayerRecord::type_layer`. - `smart`: `SmartObject` from `SoLd`/`SoLE` (id, quad, non-affine transform, size, warp) and `PlLd`; `Warp` with the named styles and the custom quilt mesh; the quad geometry (`map`, `placed_bounds`, `placed_size`, `bake_scale`, `placement`). `LinkedFile` and `linked_files` for `lnk2`/`lnkD`/`lnk3`/`lnkE`, `PsdFile::linked_files` and `PsdFile::linked_data`. `LayerRecord::smart_object`. - `adjust`: `Adjustment` from `nvrt`, `brit`, `CgEd`, `hue2`/`hue `, `levl`, `expA`, and the other adjustment keys by name; `LayerRecord::adjustment`, `solid_color` (`SoCo`) and `effects` (`lfx2`). - `Descriptor`: `object`, `number`, `numbers`, `boolean`, `text`, `enumerated`, `raw_data`, `color`; `Value::as_number`. Linked-file data is a range of the block's data, checked against the record, instead of a range of the whole file. * craft-psd: float pixels, Lab composites and salvaging a damaged layer section From EffectCraft's effectcraft-psd (storytold/effectcraft f334bcf3, crates/psd/src/pixels.rs, limits.rs and placed.rs): - `float`: `PsdFile::composite_f32` and `layer_f32` decode to straight RGBA f32 in every colour mode and depth: CMYK naively, indexed through the palette, Lab through D50 to sRGB, 32-bit encoded to sRGB, a layer's raster mask applied and a solid colour fill layer filled. One image is at most 2^28 pixels and a decode refuses before allocating when what it holds at once is over its budget (4 GiB; `_within` takes a limit). `SmartObject::render_placed` bakes a warped or perspective placement, bounded in pixels and in rasterising work. - `composite_rgba8` and `Layer::rgba8` read Lab. - `PsdFile::from_bytes_salvaging`: a file whose layer and mask section cannot be read opens as its merged image, with the error it skipped. * craft-psd: shared descriptor lookups Descriptor::get_bool, get_number, get_text, get_enum and get_object, used by asl, grd and slices in place of their own copies. Ported from storytold/photocraft#9552 (its crates/psd part). * craft-psd: the typed views use the shared descriptor lookups One set of Descriptor accessors: the get_* lookups from photocraft#9552. numbers, raw_data and color move next to them as get_numbers, get_raw_data and get_color. --------- Co-authored-by: Omar Saidahmed <osaidahm@uwaterloo.ca> |
||
|
Dylan M. Taylor
|
8693b9181c |
craft-exr: bounded band-by-band reader for flat scan-line parts from PhotoCraft (stacked on #1001) (#1004)
* craft-exr: bounded part listing, flat decoding and writing (feature decode) PhotoCraft's exr-crate wrapper (preview check, per-part and total size limits, file-size plausibility, part ranking, one-part decode) and EffectCraft's streaming channel filter, as one reader that returns a neutral model: part headers with text attributes, and the kept channels in their stored type. * craft-exr: deep data and Cryptomatte samples (feature decode) PhotoCraft's deep OpenEXR reader (NONE, RLE and ZIPS deep scan lines and tiles, every allocation admitted against the byte budget) and its flattening, and its Cryptomatte layer listing and rank decoding, on the decode module. * craft-exr: bounded band-by-band reader for flat scan-line parts PhotoCraft's exr_stream decoder: bounded metadata, the offset table, then one compression block at a time as interleaved f32. Custom chromaticities are reported instead of refused, so the colour policy stays with the caller. |
||
|
Dylan M. Taylor
|
6cc226e454 |
craft-timeline: OpenTimelineIO on the neutral model (#1002)
* craft-timeline: a neutral timeline model and the import builder model: a Project of bins and items (media, sequences, subclips, and items no interchange format carries), sequences of tracks with clips, transitions, markers and effects with keyframes. Cut down from FilmCraft's project model to what Final Cut Pro 7 XML, FCPXML and OpenTimelineIO describe; EffectCraft already uses that shape for its interchange. Three things stay the application's: effects (looked up through a Catalog the caller supplies), generators beyond the common ones (Generator::Other with an opaque payload), and per-object extension JSON (ext) that a format can carry unread. fx: the effect and parameter ids the formats name themselves, and Standard, a catalog of just those. build: FilmCraft's import Builder and shared helpers (filmcraft-interchange common.rs at 4bec22bd) on the model, with the effect registry replaced by the catalog, and the bounds every import ends with (Imported::bound, from lib.rs). * craft-timeline: Final Cut Pro 7 XML (xmeml) on the neutral model fcp7: FilmCraft's xmeml reader and writer (filmcraft-interchange fcp7.rs at 4bec22bd) moved onto craft_timeline::model, behind feature xml. - XML is read with craft-xml, strictly: at most 256 levels of nesting (the limit EffectCraft checked with its own depth scan before handing documents over), 2 000 000 nodes, 256 MiB; a document type declaration with an internal subset and entities XML does not define are refused. It is written with craft-xml's Writer. - Effects other than motion, opacity, time remap, audio levels and pan are looked up through the caller's Catalog, by id and by name, and written with the catalog's stable name and category. - An application generator round-trips as <namespace>:<payload> in effectid (FilmCraft wrote filmcraft:<json>); transparent video, counting leader and bars and tone are written by name, which the reader already took. - Finding the sequence a clip item is in is one walk of the document instead of an ancestors() lookup per clip item. tests/it: FilmCraft's FCP7 integration tests (the hand-written Premiere-style document, the rich round trip, the 48-case property round trip, split mono and stereo channels, sub-frame spans, absurd timebases and start timecodes, the link-group bound) on the model, with a test catalog for the four application effects they use. * craft-timeline: FCPXML on the neutral model fcpxml: FilmCraft's FCPXML 1.9-1.11 reader and writer (filmcraft-interchange fcpxml.rs at 4bec22bd) moved onto craft_timeline::model, behind feature xml. XML is read with craft-xml as fcp7 is; filters other than the transform, blend and volume adjustments go through the caller's Catalog by name, and an effect resource's uid is <namespace>.<effect id> (FilmCraft wrote filmcraft.<id>). ExportOptions gains fcpxml_version. tests/it/fcpxml.rs: FilmCraft's FCPXML integration tests on the model (the hand-written connected-clips document, hostile lane numbers, standalone video stills, self-referencing compound media, round trips at 1.9, 1.10 and 1.11, the property round trip with its regression seed, percent-decoded relative URLs). The FCP7 tests export to FCPXML again where FilmCraft's did. * craft-timeline: OpenTimelineIO on the neutral model otio: FilmCraft's OTIO reader and writer (filmcraft-interchange otio.rs at 4bec22bd) moved onto craft_timeline::model, behind feature otio (serde_json). FilmCraft wrote its own serde types into metadata.filmcraft. Here the model's fields are written as JSON by hand under metadata.<namespace>, with the same keys and value shapes FilmCraft's serde produced (labels, settings, media info, generators, effects with externally tagged parameter values, alignment, marker kinds, channel layouts), so files FilmCraft wrote read back. Every key the reader does not know comes back in the object's ext, and ext's keys are written next to the model's: an application's palette, track volume, masks and the like round-trip without the crate knowing them. ParamValue::Opaque and Generator::Other carry values only the application reads. The model's Sequence loses its ext: a sequence's is its item's, so there is one place for it. ImportOptions gains name (for an unnamed timeline), as FilmCraft's had. tests: FilmCraft's OTIO integration tests on the model (the golden document, the lossless round trip now through ext, collections and bare tracks, items at an audio sample rate, generator labels and identity, the property round trip, marker URLs, percent-decoded relative URLs); its palette tests stay with FilmCraft, whose palette merge they test, and are replaced by one ext round trip. The OTIO unit tests (exact rational times, the nested-copies limit) and a JSON nesting test. |
||
|
Dylan M. Taylor
|
99b5141700 |
craft-exr: deep data and Cryptomatte samples from PhotoCraft (feature decode, stacked on #992) (#1001)
* craft-exr: bounded part listing, flat decoding and writing (feature decode) PhotoCraft's exr-crate wrapper (preview check, per-part and total size limits, file-size plausibility, part ranking, one-part decode) and EffectCraft's streaming channel filter, as one reader that returns a neutral model: part headers with text attributes, and the kept channels in their stored type. * craft-exr: deep data and Cryptomatte samples (feature decode) PhotoCraft's deep OpenEXR reader (NONE, RLE and ZIPS deep scan lines and tiles, every allocation admitted against the byte budget) and its flattening, and its Cryptomatte layer listing and rank decoding, on the decode module. |
||
|
|
0da3928eee |
craft-raw: Panasonic RW2, RWL and RAW (#1000)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW A new crate for #587 and #778: bytes in, the sensor data out as a neutral RawImage (samples, CFA, levels, active area, crop, orientation, colour tags, opcode lists, profile tables), and back for DNG. One Cargo feature per format: dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw). The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw, unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile and gaintable, the model and probe. Demosaicing and everything after it stay with the develop pipeline. Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the default crop's per-tag defaults, an unusable crop falling back to the active area, non-finite white levels, the white default of a linearized image, the AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their part of the image, and the CinemaDNG frame tags. * craft-raw: Hasselblad 3FR and FFF (feature hasselblad) RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add. Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds, and the 8-bit RGB preview handed on as a baseline TIFF. Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests. Ported from PhotoCraft's reader: four BlackLevel values without a BlackLevelRepeatDim are one per CFA site in a file that is not a DNG. * craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak), moved from its vendor modules at 5ff567a4 with their tests; the MRW block container comes from lightcraft-meta. src/shared.rs holds the level helpers the maker decoders share. ERF and MEF state no white balance: the estimate LightCraft made from the preview is develop-side work, so wb_multipliers is None for them here. * craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal size is mostly padding outside the image is refused before it is inflated. * craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an image whose pixel bytes exceed the file and still takes a smaller complete one. * craft-raw: test for the Deflate padding bound * craft-raw: Pentax PEF and Samsung SRW Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module. * craft-raw: Canon CR2, with sRAW and mRAW Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs; the temperature and tint arithmetic stays with the developer). * craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview). * WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests) * craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level * craft-raw: Panasonic RW2 / RWL / RAW, every raw format * craft-raw: RW2 format 8 refuses strips stating fewer bits than samples Ported from storytold/lightcraft#3910. * craft-raw: RW2 format 8 refuses samples above the declared bit depth Ported from storytold/photocraft#11838. * craft-raw: the RW2 format 8 robustness seed carries a valid stream Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes. --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: wnanbei <wnanbei@gmail.com> |
||
|
Dylan M. Taylor
|
3826abcddf |
craft-drawingml: preset and system colours, Strict percentages, chart series formulas (from WordCraft and GridCraft) (#998)
* Add craft-drawingml: units, colours, preset shapes, custom geometry and text warps The parts of DrawingML that need no document model, moved from DeckCraft (60f91f78): crates/color, crates/geom (preset, warp, Xfrm and the EMU conversions) and crates/pptx/src/custgeom.rs with CustomPath from crates/model. The code is unchanged except that custgeom reads craft-xml trees and writes to a craft-xml Writer instead of DeckCraft's own XML helper, and matches elements by local name. The 59 tests of the moved code come with it; six more cover the custom geometry limits. * drawingml: evaluate a custom geometry's guide lists once per coordinate space read_cust_geom rebuilt Guides and evaluated avLst and gdLst again for every path: up to 1,000 paths times up to 10,000 guides, each name copied and hashed. Guides are now cached per path coordinate space (w, h). At most 16 spaces and 4 MiB of guide names are kept per geometry; paths in further spaces are left out. The first space is always kept, and a geometry without guides is not capped. Ported from storytold/deckcraft#1789. A list counts as having guides only when it has gd children, as that PR's review asked. * drawingml: the model types for fills, lines, text, tables, charts and diagrams Moved from DeckCraft (60f91f78) crates/model/src: style.rs, text.rs, table.rs, chart.rs, numfmt.rs and trendline.rs, plus Diagram, DiagramNode and the MediaId and SlideId newtypes from lib.rs. The files are unchanged except that deckcraft_color:: is crate::color:: and Fill and TableBorder allow clippy::large_enum_variant. Their 60 tests come with them. Readers and writers follow. * drawingml: the theme model (colour, font and format schemes) Moved from DeckCraft crates/model/src/theme.rs at 60f91f78: FontSet, ScriptFont, FontScheme, FormatScheme, Theme and default_format. The app's theme gallery, its variants and its own default theme stay in the app. Theme::default() is a plain theme built in place, so nothing here depends on a gallery. * drawingml: readers and writers for fills, lines, text, charts, themes, table styles and diagrams Moved from DeckCraft crates/pptx/src at 60f91f78: read/dml.rs, write/dml.rs, read/chart.rs, read/chartex.rs, write/chart.rs, read/theme.rs, write/theme.rs, tables.rs and dgm.rs, onto craft-xml. Package lookups (Imp, Part, Exp, Out) are the caller's part::Source and part::Sink. The chart frame budget and kept-part capture stay with the presentation reader. * drawingml: what WordCraft and GridCraft read that DeckCraft's copy did not Every ST_PresetColorVal name has its colour, in any case and with the dk/lt/med abbreviations; a sysClr without lastClr takes the usual value of its name; a Strict percentage (50%) reads anywhere a percentage does; a chart series keeps its c:f formulas and the writer uses them. * drawingml xmlx: copy only used, bounded namespace declarations onto kept fragments and mc branches From storytold/deckcraft#1740 (988a4920), which landed after the extraction commit 60f91f78. |
||
|
Dylan M. Taylor
|
a4b57d47b9 |
Add craft-crs: Camera Raw crs: settings in XMP, read and written, from PhotoCraft (#996)
* Add craft-crs: Camera Raw crs: settings in XMP, read and written, from PhotoCraft The format half of PhotoCraft's Camera Raw settings files (crates/io/src/camera_raw_xmp.rs at abeb16c96): the crs: settings of an XMP packet as written, Camera Raw's number and curve point text, the crop and its orientation, a writer, and the toolkit marker. Which setting drives which control stays in the app. * craft-crs: structures carry their element and are written back Local corrections and other structured settings were dropped, and an rdf:Seq of descriptions read as empty strings. A structure now holds its crs: element, a sequence is only plain text items, descriptions inside a structure are not read as top-level settings, and to_xmp writes structures back. * craft-crs: read packets through craft-metadata's XMP reader One XMP reader for every crate: Settings::parse goes through craft_metadata::parse_xmp and Settings::from_xmp takes its XmpData. Settings are kept by name (the reader's map has no file order), structures are craft-metadata's XmpValue and are written back as rdf:parseType Resource elements and rdf:Seq arrays. toolkit stays a bounded first-start-tag scan, as the reader does not return x:xmptk. |
||
|
Dylan M. Taylor
|
c516452b47 |
craft-codepage: one table per legacy code page, for craft-rtf, craft-dxf and craft-mime (#991)
craft-rtf and craft-dxf each carried tables for DOS 437, 737, 850, 852 and 857 (and craft-rtf one for 866). They are now one set in craft-codepage, which takes craft-rtf's codepage module (Windows and Mac tables, OEM glyphs, font charsets) and craft-dxf's DOS tables. craft_rtf::codepage re-exports it unchanged; craft-dxf keeps its $DWGCODEPAGE names and Decoder and looks the DOS tables up there. Every byte of every code page either crate decodes is unchanged: the public decoders of both crates were dumped over all 256 bytes of each page (and the East Asian pairs) before and after, and the dumps are identical. craft-mime now reads the DOS pages by their IANA names (IBM437, cp850, ibm-852, 857...), which it read as UTF-8 before. |
||
|
Dylan M. Taylor
|
0d68632c9b |
Add craft-ods: OpenDocument spreadsheet tables on craft-odf (#990)
* Add craft-sheet: the cell positions, values and tables the spreadsheet formats share craft-xls carried its own CellRef, Range, Value, CellError, Visibility, sheet-name quoting, built-in number format table and indexed palette. craft-xlsx and craft-ods need the same ones, so they move to a crate with no dependencies and craft-xls re-exports them: an application gets one Value and one CellRef whichever file it read. Added on the way: A1 parsing (CellRef::parse, Range::parse), Range::count and a1, CellError::parse and the newer error values SpreadsheetML stores as text, builtin_format_id for writers. * Add craft-ods: OpenDocument spreadsheet tables on craft-odf read() checks the spreadsheet mimetype, refuses an encrypted package, and reads the tables of content.xml into sheets of cells that keep the file's terms: value type, the raw value, displayed text, formula text, style name, covered cells, with repeats expanded and merged ranges. Cell::value() gives the shown value, dates and durations as serial numbers. From GridCraft crates/xlsx/src/ods.rs and its tests (7e0d96d7), split from GridCraft's document model and moved onto craft-odf and craft-xml's pull reader. Options::only_sheet covers WordCraft's reader of one chosen sheet. |
||
|
|
fdbb9c2471 |
craft-captions: TTML import keeps inherited styles; literal < and & stay text (#983)
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0, U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a leading or trailing <br/> was trimmed away. * craft-captions: TTML/DFXP export drops characters XML 1.0 forbids Ported from storytold/filmcraft#4467. A control character in cue text made the exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the review of that pull request asked for. * craft-captions: WebVTT cue-text arrow escape is reversible Ported from storytold/filmcraft#4437. A literal --> in cue text came back as --> after a write and parse; the writer now adds one amp; level and the reader removes exactly one. * craft-captions: a WebVTT speaker name with > no longer spills into the cue text Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in place of > inside the voice tag. Also from that pull request's review: a line break in a name is written as a space, and a name ending in - is kept apart from the tag's closing >. * craft-captions: an SRT text line that looks like a timing line survives a round trip Ported from storytold/filmcraft#4448. The writer marks such a line with a leading zero-width space and the reader removes it, so the cue is not split. * craft-captions: back-to-back SCC captions with the same text stay separate cues Ported from storytold/filmcraft#3808. Every End of Caption now closes the open cue. MCC's CEA-608 path uses the same decoder. * craft-captions: an SCC pop-on load may run across the previous caption's erase Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely follows another popped on about a second late, because its load had to be one unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed between them. * craft-captions: CEA-708 export starts each caption at its window's top left Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now sends DeleteWindows before DefineWindow in place of the ClearWindows after it: a window made anew is empty with the pen at its top left in any decoder, for the same number of bytes, so cue timing is unchanged. * craft-captions: TTML/DFXP export keeps literal < and > text Ported from storytold/filmcraft#2830. The writer treated every < with a later > as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its middle. Only a well-formed tag is markup now; a > inside a quoted attribute value does not end the tag. * craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could collide with the style ID, and a generated c1 could take the ID of a later cue. Valid explicit IDs are reserved first; missing, invalid or repeated ones get a replacement that is in no one's way. IDs are checked as XML NCNames. * craft-captions: compare the craft-xml and roxmltree views of TTML documents The TTML reader moved from roxmltree to craft-xml. This dev-only test checks that both hand it the same elements, attributes and text, by namespace, on TTML documents and on the writer's output. roxmltree becomes a dev-dependency, which the cue ID tests use too. * craft-captions: use craft-timeline's exact time instead of a private copy craft-captions carried a 290-line subset of the tick, frame rate and timecode field code because craft-libs had none when it was added. craft-timeline now has the whole module with the same tick unit. craft_captions::time is a re-export of craft_timeline::time, so every public path still resolves and the types are now the same ones craft-timeline's callers hold. * craft-captions: SCC and MCC time codes count from the document's start timecode Ported from the caption-format part of storytold/filmcraft#4131 (filmcraft#4130). Document::start_timecode is the label of time zero; the SCC and MCC writers count from the 29.97 frame with the same label, so the label carries across rates. Readers note the rate and drop-frame mode of a file's labels in Document::labels, and Document::count_from counts cues back from a start time code. charset_warnings names the time codes the file has, which that pull request's review found missing. * craft-captions: MCC export can write a packet on every frame Ported from the caption-format part of storytold/filmcraft#4037 (filmcraft#4024). WriteOptions::every_frame_until fills every 29.97 fps frame before that time that has no caption data with a packet of CEA-608 nulls and CEA-708 padding, as SDI and frame-wrapped MXF carry one per frame. Off by default, at most 24 hours (mcc::MAX_PADDED_FRAMES), and write_to streams the file line by line. * craft-captions: README has its MCC writer section, with the every-frame option The section's text was cut off when the crate was added; only a stray heading marker was left. * craft-captions: export warnings name STL's ? and text cut to fit Ported from storytold/filmcraft#4157 (filmcraft#4155, filmcraft#4156). charset_warnings now names the characters EBU STL writes as ? (from the writer's own encoder, at the STL file's time code) and each caption that CEA-608's 15 rows or the CEA-708 window's 12 rows cut, with how much it lost. The CEA-708 writer loads only the 12 rows its window holds. The time codes count from the document's start time code, as the files do. * craft-captions: TTML import keeps inherited and reset text styles Ported from the TTML style part of storytold/filmcraft#1723 (filmcraft#1721). Italic, bold and underline set on <body> or on a timed <span> cue root now reach the cue text; normal, noUnderline and inherit reset or restore the parent value; styling is read from the TTML styling namespaces (modern and 2006 drafts) with unqualified attributes as a fallback, and foreign namespaces are ignored. The inline text collector is iterative. Line breaks at the start or end of a cue are kept, as #3145's port does, so this differs from the source pull request there. * craft-captions: literal < and & stay text beside cue style tags Ported from the literal-text part of storytold/filmcraft#1723. The TTML and EBU STL readers keep a literal < as < (and an & that would read as an entity as &), as a WebVTT import already keeps its entities, so text like "<b>" in a TTML file never becomes a bold tag. The TTML and STL writers decode those entities, so a cue read from WebVTT with "1 < 2" is no longer written as "1 &lt; 2". entity_at and decode_entities are shared with plain_text. --------- Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com> Co-authored-by: SeeRay11 <searay112023@gmail.com> Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com> Co-authored-by: JonRC <jonarc06@gmail.com> Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com> Co-authored-by: nuclearspike <liquidmonkey@gmail.com> |
||
|
|
6f51ec7f8d |
craft-fig: map the .fig message onto the REST file JSON (#980)
* Add craft-figma-rest: a typed model of the Figma REST file JSON The neutral model for design files: the .fig and .sketch readers map onto it and an application's importer takes it. Typed records for the document tree, paints, effects, text styles, layout grids, constraints, export settings and the component and style metadata; every record keeps the fields it has no name for, so a file read and written loses nothing. The vectorNetwork extension is typed and documented with the others. The layer tree is converted level by level in both directions, and the depth and node limits are checked on the JSON before any typed value is built. * craft-figma-rest: Warnings and style_name for the readers that build this shape Moved from FigCraft's REST importer and .fig reader (crates/format/src/rest.rs Warnings, fig.rs style_name), which the .fig and Sketch mappings share. * craft-fig: map the message onto the REST file JSON FigCraft's .fig mapping (crates/format/src/fig/map and fig/text_path.rs at e8b53e33) moved onto craft-figma-rest: rest::to_rest turns a decoded message into the REST API file JSON, with the vector networks layers name by vectorNetworkRef handed over beside it. Connector routing and the text path measure check are the caller's geometry (rest::Geometry; rest::Straight when it has none). The plugin data key under which a writer keeps an outlined stroke is an option. The geometry blob is read with craft_fig::path instead of a second decoder. * craft-fig: an empty overriddenSymbolID is no swap The empty GUID (4294967295:4294967295) in overriddenSymbolID means "no swap". The mapping took it as the main, found no node and made the instance an empty frame; it now keeps the main its symbolData names, in the instance mapping and in the list of mains a file's instances use. Ported from storytold/figcraft#2208. * craft-fig: a swapped nested instance takes its new main's look A nested instance swapped to another main stores the previous main's look on its own node, so the mapping drew that. Each group of the look (fills, strokes, stroke settings, effects, corners) whose stored values are a copy of the previous main's now takes the new main's; a group the node or a row above it changes is kept, and a style link is decided by the topmost layer that names it. Ported from storytold/figcraft#2204 (the fig/map part; the warning's locale entry stays in the app). * craft-fig: a vector point without a corner radius of its own takes the layer's A point whose style sets no cornerRadius, or that has no style, takes the layer's cornerRadius, as the file draws it; a point's own value (an explicit 0 included) still wins. The layer's radius is part of the network cache key, so an instance overriding the radius doesn't share its main's network. Ported from storytold/figcraft#2115 (the fig/map part; its export change stays in the app). * craft-fig: map deep files within a 1 MiB stack The mapping recursed once per level of layer nesting in a frame of about 2 KB (an unoptimised build): 300 nested frames, or a chain of instances of mains holding each other, overflowed a 1 MiB stack before the 256 level limit could stop them, a crash on hostile input. The work on one layer now happens in layer_head and layer_tail, which return before and after its children are mapped, and what they hand over is boxed, so the recursive frame is about 800 bytes. An unoptimised build now maps past the limit on 512 KiB; the limit stays 256. The test at the limit runs on a 1 MiB thread, for frames and for instances, and overflowed before this change. --------- Co-authored-by: Trevor Brown <trevorbrown.dev@gmail.com> Co-authored-by: Giovanni Almeida <gioalmeida07@gmail.com> |
||
|
Dylan M. Taylor
|
00efe17d1a |
craft-drawingml: readers and writers on craft-xml (#979)
* Add craft-drawingml: units, colours, preset shapes, custom geometry and text warps The parts of DrawingML that need no document model, moved from DeckCraft (60f91f78): crates/color, crates/geom (preset, warp, Xfrm and the EMU conversions) and crates/pptx/src/custgeom.rs with CustomPath from crates/model. The code is unchanged except that custgeom reads craft-xml trees and writes to a craft-xml Writer instead of DeckCraft's own XML helper, and matches elements by local name. The 59 tests of the moved code come with it; six more cover the custom geometry limits. * drawingml: evaluate a custom geometry's guide lists once per coordinate space read_cust_geom rebuilt Guides and evaluated avLst and gdLst again for every path: up to 1,000 paths times up to 10,000 guides, each name copied and hashed. Guides are now cached per path coordinate space (w, h). At most 16 spaces and 4 MiB of guide names are kept per geometry; paths in further spaces are left out. The first space is always kept, and a geometry without guides is not capped. Ported from storytold/deckcraft#1789. A list counts as having guides only when it has gd children, as that PR's review asked. * drawingml: the model types for fills, lines, text, tables, charts and diagrams Moved from DeckCraft (60f91f78) crates/model/src: style.rs, text.rs, table.rs, chart.rs, numfmt.rs and trendline.rs, plus Diagram, DiagramNode and the MediaId and SlideId newtypes from lib.rs. The files are unchanged except that deckcraft_color:: is crate::color:: and Fill and TableBorder allow clippy::large_enum_variant. Their 60 tests come with them. Readers and writers follow. * drawingml: the theme model (colour, font and format schemes) Moved from DeckCraft crates/model/src/theme.rs at 60f91f78: FontSet, ScriptFont, FontScheme, FormatScheme, Theme and default_format. The app's theme gallery, its variants and its own default theme stay in the app. Theme::default() is a plain theme built in place, so nothing here depends on a gallery. * drawingml: readers and writers for fills, lines, text, charts, themes, table styles and diagrams Moved from DeckCraft crates/pptx/src at 60f91f78: read/dml.rs, write/dml.rs, read/chart.rs, read/chartex.rs, write/chart.rs, read/theme.rs, write/theme.rs, tables.rs and dgm.rs, onto craft-xml. Package lookups (Imp, Part, Exp, Out) are the caller's part::Source and part::Sink. The chart frame budget and kept-part capture stay with the presentation reader. * drawingml xmlx: copy only used, bounded namespace declarations onto kept fragments and mc branches From storytold/deckcraft#1740 (988a4920), which landed after the extraction commit 60f91f78. |
||
|
Dylan M. Taylor
|
384da3e951 |
craft-timeline: Final Cut Pro 7 XML (xmeml) on the neutral model (#978)
* craft-timeline: a neutral timeline model and the import builder model: a Project of bins and items (media, sequences, subclips, and items no interchange format carries), sequences of tracks with clips, transitions, markers and effects with keyframes. Cut down from FilmCraft's project model to what Final Cut Pro 7 XML, FCPXML and OpenTimelineIO describe; EffectCraft already uses that shape for its interchange. Three things stay the application's: effects (looked up through a Catalog the caller supplies), generators beyond the common ones (Generator::Other with an opaque payload), and per-object extension JSON (ext) that a format can carry unread. fx: the effect and parameter ids the formats name themselves, and Standard, a catalog of just those. build: FilmCraft's import Builder and shared helpers (filmcraft-interchange common.rs at 4bec22bd) on the model, with the effect registry replaced by the catalog, and the bounds every import ends with (Imported::bound, from lib.rs). * craft-timeline: Final Cut Pro 7 XML (xmeml) on the neutral model fcp7: FilmCraft's xmeml reader and writer (filmcraft-interchange fcp7.rs at 4bec22bd) moved onto craft_timeline::model, behind feature xml. - XML is read with craft-xml, strictly: at most 256 levels of nesting (the limit EffectCraft checked with its own depth scan before handing documents over), 2 000 000 nodes, 256 MiB; a document type declaration with an internal subset and entities XML does not define are refused. It is written with craft-xml's Writer. - Effects other than motion, opacity, time remap, audio levels and pan are looked up through the caller's Catalog, by id and by name, and written with the catalog's stable name and category. - An application generator round-trips as <namespace>:<payload> in effectid (FilmCraft wrote filmcraft:<json>); transparent video, counting leader and bars and tone are written by name, which the reader already took. - Finding the sequence a clip item is in is one walk of the document instead of an ancestors() lookup per clip item. tests/it: FilmCraft's FCP7 integration tests (the hand-written Premiere-style document, the rich round trip, the 48-case property round trip, split mono and stereo channels, sub-frame spans, absurd timebases and start timecodes, the link-group bound) on the model, with a test catalog for the four application effects they use. |
||
|
|
59fc18b262 |
craft-metadata: Fujifilm in-camera recipes, from LightCraft (#974)
* craft-metadata: EXIF, IPTC-IIM, XMP and GPX from LightCraft lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif (read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse, write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers. They share Metadata, DateTime, Gps and Orientation. Changed in the move: - TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own. - The app namespace is a parameter: parse_xmp_with, write_xmp_with and MergeRules::namespaces take it, and XmpData no longer has lc_settings. - The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and craft-raw read those containers; from_embedded takes the blocks they find. * craft-metadata: a RW2 reads ISO and the rest of its Exif from the JpgFromRaw A RW2's own Exif IFD is a short subset. What it lacks is filled from the Exif of the embedded JpgFromRaw, then from the ISO tags of the Panasonic IFD0 (0x0017, 0x0037), each only in its recorded field type. 65535 in a SHORT ISOSpeedRatings is the overflow marker and states no ISO. The private tags are read only in a RW2 stream, under the caller's parse options. Ported from storytold/lightcraft#3518. * craft-metadata: the Exif rating is written, and 65535 reads as rejected write_exif writes Rating (0x4746), with 0xFFFF for a rejected photo, and from_tiff reads 65535 as -1 instead of clamping it to 5. Ported from storytold/lightcraft#1995. * craft-metadata: one shutter-speed formatter, exact to four digits format_exposure_time shows a duration within 0.1% of 1/n second as 1/n and anything else to four significant digits, so 0.3 s, 1/2.8 s and 0.625 s are no longer rounded to a reciprocal or one decimal. Metadata::exposure_display and the ExposureTime tag row both use it. Ported from storytold/lightcraft#1692. * craft-metadata: JPEG XL containers give their Exif and xml boxes The first Exif and xml box of a JPEG XL container, plain or Brotli-wrapped in brob, honouring the Exif box's TIFF header offset. A brob payload is capped at 32 MiB. A bare codestream has no metadata. Ported from storytold/lightcraft#3756. * craft-metadata: XMP Resource structs flatten their attribute fields A property with rdf:parseType="Resource" may give its fields as attributes as well as child elements. Only the children were flattened into XmpData::properties; the attributes now are too, as for a nested rdf:Description. Ported from storytold/lightcraft#1226 (the XMP part). * craft-metadata: a capture date of just a year or a month stays as stated XMP dates may stop at the year (1983) or the month (1983-04). DateTime now has a DatePrecision; such a date writes back to XMP as stated and only in photoshop:DateCreated, and write_exif leaves it out, since an Exif date cannot say it. Ported from storytold/lightcraft#3502. * craft-metadata: Fujifilm in-camera recipes from the maker note fuji module: the film simulation and the rest of the in-camera recipe (grain, colour chrome, tone, dynamic range, white balance and its shift, sharpness, noise reduction, clarity) from a Fujifilm maker note. recipe_from_exif, recipe_from_bytes (TIFF, JPEG, RAF) and read_recipe_file, which reads only as far as the metadata. from_embedded fills Metadata::fuji_recipe and file_tag_rows lists the recipe under Fujifilm. Ported from storytold/lightcraft#1209. * craft-metadata: the linear-time scan test compares times, not a fixed bound The test asserted that 200 000 attributes scan in under 5 s, which failed on a loaded machine. It now scans 25 000 and 100 000 attributes, takes the best of three runs of each, and asserts that four times the attributes take less than ten times as long. Linear scans give about 4; a quadratic duplicate check gave 18.6 and fails. * craft-metadata: interleave the linear-time scan test's runs The best-of-three ratio still failed once under load. The test now alternates five runs of 50 000 and 200 000 attributes and compares the fastest of each, so a burst of load slows both sizes. Six runs at load average 55 gave ratios of 3.9 to 5.4 against the bound of 10. --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: Sebastian Berschneider <berschi@gmail.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Michael Anderson <1427851+endthestart@users.noreply.github.com> Co-authored-by: Ilya Dorman <ilya@codeforhumans.dev> Co-authored-by: Clyde C Brown <clyde.c.brown@pm.me> Co-authored-by: MohammadTeimori <teimori.mohammad@gmail.com> |
||
|
Dylan M. Taylor
|
026256d8cc |
craft-mime: detect, for bytes that name no character set (#973)
is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding the bytes look like), legacy and legacy_label, behind a detect feature that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a caller can tell a label decode would read as UTF-8 from one it knows. |
||
|
Dylan M. Taylor
|
2a24c10817 |
Support existing app identities and repository-prefixed update tags (#970)
* fix(update): support existing bundle identities and repository-prefixed tags * fix(update): recognize established Tauri installer asset names |
||
|
Justin Wilson
|
8cdcdce326 |
craft-video: VP8 key-frame encoder (vp8-encode feature) (#969)
Moves effectcraft's effectcraft-vp8enc here: the lossy picture of a WebP image or animation frame, written from RFC 6386. Rate-distortion choice between the 16x16 and B_PRED intra modes, per-frame token probability updates, loop filter level from the quantizer. `vp8` now turns on `vp8-decode` and `vp8-encode`, as `vp9` does; the decoder's tests and examples require `vp8-decode`. The tests decode the encoder's frames with this crate's decoder: with the loop filter off it reproduces the encoder's reconstruction exactly. |
||
|
|
75471d19bd |
craft-metadata: fixes ported from open LightCraft pull requests (#967)
* craft-metadata: EXIF, IPTC-IIM, XMP and GPX from LightCraft lightcraft-meta at 5b61ce1e, as one crate with a module per format: exif (read, write, maker-note lens, focus point, tag listing), iptc, xmp (parse, write, merge), gpx, and the JPEG, PNG, WebP, TIFF, RAF and CR3 containers. They share Metadata, DateTime, Gps and Orientation. Changed in the move: - TIFF comes from craft-tiff; Point, Rect and Orientation are the crate's own. - The app namespace is a parameter: parse_xmp_with, write_xmp_with and MergeRules::namespaces take it, and XmpData no longer has lc_settings. - The HEIF, PSD and MRW readers stay out, since craft-heif, craft-psd and craft-raw read those containers; from_embedded takes the blocks they find. * craft-metadata: a RW2 reads ISO and the rest of its Exif from the JpgFromRaw A RW2's own Exif IFD is a short subset. What it lacks is filled from the Exif of the embedded JpgFromRaw, then from the ISO tags of the Panasonic IFD0 (0x0017, 0x0037), each only in its recorded field type. 65535 in a SHORT ISOSpeedRatings is the overflow marker and states no ISO. The private tags are read only in a RW2 stream, under the caller's parse options. Ported from storytold/lightcraft#3518. * craft-metadata: the Exif rating is written, and 65535 reads as rejected write_exif writes Rating (0x4746), with 0xFFFF for a rejected photo, and from_tiff reads 65535 as -1 instead of clamping it to 5. Ported from storytold/lightcraft#1995. * craft-metadata: one shutter-speed formatter, exact to four digits format_exposure_time shows a duration within 0.1% of 1/n second as 1/n and anything else to four significant digits, so 0.3 s, 1/2.8 s and 0.625 s are no longer rounded to a reciprocal or one decimal. Metadata::exposure_display and the ExposureTime tag row both use it. Ported from storytold/lightcraft#1692. * craft-metadata: JPEG XL containers give their Exif and xml boxes The first Exif and xml box of a JPEG XL container, plain or Brotli-wrapped in brob, honouring the Exif box's TIFF header offset. A brob payload is capped at 32 MiB. A bare codestream has no metadata. Ported from storytold/lightcraft#3756. * craft-metadata: XMP Resource structs flatten their attribute fields A property with rdf:parseType="Resource" may give its fields as attributes as well as child elements. Only the children were flattened into XmpData::properties; the attributes now are too, as for a nested rdf:Description. Ported from storytold/lightcraft#1226 (the XMP part). * craft-metadata: a capture date of just a year or a month stays as stated XMP dates may stop at the year (1983) or the month (1983-04). DateTime now has a DatePrecision; such a date writes back to XMP as stated and only in photoshop:DateCreated, and write_exif leaves it out, since an Exif date cannot say it. Ported from storytold/lightcraft#3502. * craft-metadata: the linear-time scan test compares times, not a fixed bound The test asserted that 200 000 attributes scan in under 5 s, which failed on a loaded machine. It now scans 25 000 and 100 000 attributes, takes the best of three runs of each, and asserts that four times the attributes take less than ten times as long. Linear scans give about 4; a quadratic duplicate check gave 18.6 and fails. * craft-metadata: interleave the linear-time scan test's runs The best-of-three ratio still failed once under load. The test now alternates five runs of 50 000 and 200 000 attributes and compares the fastest of each, so a burst of load slows both sizes. Six runs at load average 55 gave ratios of 3.9 to 5.4 against the bound of 10. --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: Sebastian Berschneider <berschi@gmail.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Michael Anderson <1427851+endthestart@users.noreply.github.com> Co-authored-by: Ilya Dorman <ilya@codeforhumans.dev> Co-authored-by: Clyde C Brown <clyde.c.brown@pm.me> |
||
|
Dylan M. Taylorandskylar taylor-barrick
|
c88555e4a5 |
craft-epub: table of contents reading, EPUB 2 NCX writing, encryption.xml (#962)
* craft-zip: ZIP preflight, bounded entry read and Budget (from craft-archive, #129) The ZIP half of the craft-archive proposal (#126, #129): an allocation-free check of the end records and central directory that runs before a ZIP reader indexes the archive, a bounded entry read that distrusts the declared size, and the named Budget both charge. The XML half of #129 is not carried: craft-xml bounds XML now. * craft-zip: Archive (entry-count guard, duplicate names, budgeted reads) and Writer check_entry_count is craft-opc's guard: the declared entry count, ZIP64 included, checked on the raw bytes before zip parses the central directory. Archive reads entries within a per-entry limit and a total Budget through read_entry. Duplicate names are found by counting the central directory headers against the entries zip kept. Writer is craft-opc's ZipOut: stored or deflated entries, ZIP64 for large ones, one fixed timestamp. * craft-idml, craft-epub: read and write through craft-zip Both crates drop their own bounded ZIP reads and zip writers. They gain the entry-count check before the archive is indexed (ZIP64 included) and archives whose bytes depend only on their content (one fixed timestamp). * craft-epub: table of contents (navigation document, NCX), EPUB 2 NCX writing, encryption.xml Epub::toc reads the EPUB 3 navigation document's toc, else the EPUB 2 NCX, as entries with a label, target and level, walked without recursion and capped. Book::ncx also writes toc.ncx for older reading systems. Epub::encryption lists what META-INF/encryption.xml says is encrypted and tells font obfuscation from DRM. --------- Co-authored-by: skylar taylor-barrick <hi@skylar.cc> |
||
|
|
035fc4f642 |
craft-captions: export warnings name STL's ? and captions cut to fit (#959)
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0, U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a leading or trailing <br/> was trimmed away. * craft-captions: TTML/DFXP export drops characters XML 1.0 forbids Ported from storytold/filmcraft#4467. A control character in cue text made the exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the review of that pull request asked for. * craft-captions: WebVTT cue-text arrow escape is reversible Ported from storytold/filmcraft#4437. A literal --> in cue text came back as --> after a write and parse; the writer now adds one amp; level and the reader removes exactly one. * craft-captions: a WebVTT speaker name with > no longer spills into the cue text Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in place of > inside the voice tag. Also from that pull request's review: a line break in a name is written as a space, and a name ending in - is kept apart from the tag's closing >. * craft-captions: an SRT text line that looks like a timing line survives a round trip Ported from storytold/filmcraft#4448. The writer marks such a line with a leading zero-width space and the reader removes it, so the cue is not split. * craft-captions: back-to-back SCC captions with the same text stay separate cues Ported from storytold/filmcraft#3808. Every End of Caption now closes the open cue. MCC's CEA-608 path uses the same decoder. * craft-captions: an SCC pop-on load may run across the previous caption's erase Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely follows another popped on about a second late, because its load had to be one unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed between them. * craft-captions: CEA-708 export starts each caption at its window's top left Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now sends DeleteWindows before DefineWindow in place of the ClearWindows after it: a window made anew is empty with the pen at its top left in any decoder, for the same number of bytes, so cue timing is unchanged. * craft-captions: TTML/DFXP export keeps literal < and > text Ported from storytold/filmcraft#2830. The writer treated every < with a later > as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its middle. Only a well-formed tag is markup now; a > inside a quoted attribute value does not end the tag. * craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could collide with the style ID, and a generated c1 could take the ID of a later cue. Valid explicit IDs are reserved first; missing, invalid or repeated ones get a replacement that is in no one's way. IDs are checked as XML NCNames. * craft-captions: compare the craft-xml and roxmltree views of TTML documents The TTML reader moved from roxmltree to craft-xml. This dev-only test checks that both hand it the same elements, attributes and text, by namespace, on TTML documents and on the writer's output. roxmltree becomes a dev-dependency, which the cue ID tests use too. * craft-captions: use craft-timeline's exact time instead of a private copy craft-captions carried a 290-line subset of the tick, frame rate and timecode field code because craft-libs had none when it was added. craft-timeline now has the whole module with the same tick unit. craft_captions::time is a re-export of craft_timeline::time, so every public path still resolves and the types are now the same ones craft-timeline's callers hold. * craft-captions: SCC and MCC time codes count from the document's start timecode Ported from the caption-format part of storytold/filmcraft#4131 (filmcraft#4130). Document::start_timecode is the label of time zero; the SCC and MCC writers count from the 29.97 frame with the same label, so the label carries across rates. Readers note the rate and drop-frame mode of a file's labels in Document::labels, and Document::count_from counts cues back from a start time code. charset_warnings names the time codes the file has, which that pull request's review found missing. * craft-captions: MCC export can write a packet on every frame Ported from the caption-format part of storytold/filmcraft#4037 (filmcraft#4024). WriteOptions::every_frame_until fills every 29.97 fps frame before that time that has no caption data with a packet of CEA-608 nulls and CEA-708 padding, as SDI and frame-wrapped MXF carry one per frame. Off by default, at most 24 hours (mcc::MAX_PADDED_FRAMES), and write_to streams the file line by line. * craft-captions: README has its MCC writer section, with the every-frame option The section's text was cut off when the crate was added; only a stray heading marker was left. * craft-captions: export warnings name STL's ? and text cut to fit Ported from storytold/filmcraft#4157 (filmcraft#4155, filmcraft#4156). charset_warnings now names the characters EBU STL writes as ? (from the writer's own encoder, at the STL file's time code) and each caption that CEA-608's 15 rows or the CEA-708 window's 12 rows cut, with how much it lost. The CEA-708 writer loads only the 12 rows its window holds. The time codes count from the document's start time code, as the files do. --------- Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com> Co-authored-by: SeeRay11 <searay112023@gmail.com> Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com> Co-authored-by: JonRC <jonarc06@gmail.com> Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com> |
||
|
Dylan M. TaylorandOmar Saidahmed
|
430a908f68 |
craft-psd: typed views of type layers, smart objects and adjustments (#957)
* craft-psd: Photoshop PSD/PSB reader and writer, moved from PhotoCraft PhotoCraft's `photocraft-psd` crate (crates/psd at storytold/photocraft 63b641b55e54bf8f5c83efd977fdb16a2c255431) as `craft-psd`: the lossless PSD/PSB model, reader, writer and builder, descriptors, the Photoshop resource files that share its primitives (.abr, .asl, .aco, .act, .ase, .grd, .pat, .atn, .csh, .shc) and the Photoshop blocks of a layered TIFF. The crate had no dependency on any other PhotoCraft crate. Code is unchanged apart from the crate name, craft-libs' rustfmt width, comments that named other PhotoCraft crates, issue references (now `photocraft#N`), and the corpus tests' directory (`CRAFT_PSD_CORPUS` or `corpus/` at the workspace root). Tests, proptests, fuzz targets and fixtures move with it. Part of #778, #786. * craft-psd: engine_data, the text engine document of type layers The EngineData reader and writer from PhotoCraft's photocraft-text (crates/text/src/engine_data.rs at storytold/photocraft 63b641b5): a tree of values that is written back in Photoshop's layout, with strings stored as plain bytes kept as plain bytes. From EffectCraft's effectcraft-psd (crates/psd/src/engine_data.rs at storytold/effectcraft 072e1af9), ported onto that tree: `first_style`, the first style run as a typed `TextStyle` with bounded sizes and spacings, and `minimal`, a one-run document for writers and fixtures. `Value::index` is new. Both apps had their own parser for the same syntax. * craft-psd: engine_data::parse_lossy reads what it can of a damaged document EffectCraft's EngineData reader was best effort: a truncated or damaged document still gave the text and style it held. `parse_lossy` does that on the same parser: a truncated dictionary, array or string keeps what it has, a byte that starts no key is skipped, a token that is no number becomes a name, and garbage gives an empty dictionary. `parse` stays strict and is unchanged in what it accepts. The parser's `value` is split into one function per kind of value, and reads with `get` instead of indexing. * craft-psd: typed views of type layers, smart objects and adjustments Read-only views EffectCraft's effectcraft-psd had over the blocks this crate keeps raw (storytold/effectcraft f334bcf3, crates/psd/src/parse.rs and placed.rs). The blocks stay raw and write back unchanged. - `text`: `TypeLayer` from `TySh`: text, transform (the identity when not finite), bounds, the first style run (read with `engine_data::parse_lossy`) and the text descriptor. `LayerRecord::type_layer`. - `smart`: `SmartObject` from `SoLd`/`SoLE` (id, quad, non-affine transform, size, warp) and `PlLd`; `Warp` with the named styles and the custom quilt mesh; the quad geometry (`map`, `placed_bounds`, `placed_size`, `bake_scale`, `placement`). `LinkedFile` and `linked_files` for `lnk2`/`lnkD`/`lnk3`/`lnkE`, `PsdFile::linked_files` and `PsdFile::linked_data`. `LayerRecord::smart_object`. - `adjust`: `Adjustment` from `nvrt`, `brit`, `CgEd`, `hue2`/`hue `, `levl`, `expA`, and the other adjustment keys by name; `LayerRecord::adjustment`, `solid_color` (`SoCo`) and `effects` (`lfx2`). - `Descriptor`: `object`, `number`, `numbers`, `boolean`, `text`, `enumerated`, `raw_data`, `color`; `Value::as_number`. Linked-file data is a range of the block's data, checked against the record, instead of a range of the whole file. * craft-psd: shared descriptor lookups Descriptor::get_bool, get_number, get_text, get_enum and get_object, used by asl, grd and slices in place of their own copies. Ported from storytold/photocraft#9552 (its crates/psd part). * craft-psd: the typed views use the shared descriptor lookups One set of Descriptor accessors: the get_* lookups from photocraft#9552. numbers, raw_data and color move next to them as get_numbers, get_raw_data and get_color. --------- Co-authored-by: Omar Saidahmed <osaidahm@uwaterloo.ca> |
||
|
Dylan M. Taylor
|
55d120f201 |
Add craft-figma-rest: a typed model of the Figma REST file JSON (#954)
* Add craft-figma-rest: a typed model of the Figma REST file JSON The neutral model for design files: the .fig and .sketch readers map onto it and an application's importer takes it. Typed records for the document tree, paints, effects, text styles, layout grids, constraints, export settings and the component and style metadata; every record keeps the fields it has no name for, so a file read and written loses nothing. The vectorNetwork extension is typed and documented with the others. The layer tree is converted level by level in both directions, and the depth and node limits are checked on the JSON before any typed value is built. * craft-figma-rest: Warnings and style_name for the readers that build this shape Moved from FigCraft's REST importer and .fig reader (crates/format/src/rest.rs Warnings, fig.rs style_name), which the .fig and Sketch mappings share. |
||
|
|
8b7f64e2e0 |
craft-pdf: crypt ports of two open PdfCraft changes (revision 2 reserved bits, attachments-only encryption) (#952)
* craft-pdf: the standard security handler from PdfCraft (`crypt`) Adds the `crypt` feature: PdfCraft's pdfcraft-crypt crate (revision 2b65ab000bfe0997dec5fbed2769d3da3799355b) as craft_pdf::crypt, with its unit, integration and property tests. The code is unchanged; doc comments and test imports name the new module. It decrypts and encrypts strings and streams for revisions 2 to 6 and creates encryption dictionaries. The reader keeps its own decryptor. * craft-pdf: the crypt unit tests build for wasm The property test in src/crypt/tests.rs needs proptest, which is a dev-dependency of the non-wasm targets only. It is now compiled for those targets only; the other unit tests still build for wasm. * craft-pdf: crypt create sets every bit revision 2 reserves Revision 2 reserves bits 7 to 32 of /P, later revisions bits 7-8 and 13-32. create left bits 9 to 12 as the caller gave them for RC4-40. Ported from storytold/pdfcraft#2789 (the create half; the accessors already read /P by revision). * craft-pdf: crypt can encrypt only embedded files create_attachments_only builds AES encryption whose /StmF and /StrF are /Identity and whose /EFF is the standard filter (ISO 32000-2 §7.6.6). RC4-40 and RC4-128 have no crypt filters and are refused. Ported from storytold/pdfcraft#5071 (the crypt half). The test is new: the source change tests it through PdfCraft's object layer. --------- Co-authored-by: Ahmad Mujahid <me@ahmadmu.com> Co-authored-by: Harrison Booth <73434116+Harrison-Booth@users.noreply.github.com> |
||
|
|
9e78e96c16 |
craft-captions: MCC export can write a packet on every frame (#951)
* craft-captions: TTML cue text collapses only XML whitespace and keeps edge line breaks Ported from storytold/filmcraft#3145. char::is_whitespace also matched U+00A0, U+2009 and U+3000, so no-break spaces became ordinary spaces on import, and a leading or trailing <br/> was trimmed away. * craft-captions: TTML/DFXP export drops characters XML 1.0 forbids Ported from storytold/filmcraft#4467. A control character in cue text made the exported file unreadable. The filter also covers U+FFFE and U+FFFF, which the review of that pull request asked for. * craft-captions: WebVTT cue-text arrow escape is reversible Ported from storytold/filmcraft#4437. A literal --> in cue text came back as --> after a write and parse; the writer now adds one amp; level and the reader removes exactly one. * craft-captions: a WebVTT speaker name with > no longer spills into the cue text Ported from storytold/filmcraft#4470. The writer puts the lookalike U+203A in place of > inside the voice tag. Also from that pull request's review: a line break in a name is written as a space, and a name ending in - is kept apart from the tag's closing >. * craft-captions: an SRT text line that looks like a timing line survives a round trip Ported from storytold/filmcraft#4448. The writer marks such a line with a leading zero-width space and the reader removes it, so the cue is not split. * craft-captions: back-to-back SCC captions with the same text stay separate cues Ported from storytold/filmcraft#3808. Every End of Caption now closes the open cue. MCC's CEA-608 path uses the same decoder. * craft-captions: an SCC pop-on load may run across the previous caption's erase Ported from storytold/filmcraft#1837 (filmcraft#1791). A caption that closely follows another popped on about a second late, because its load had to be one unbroken run of free frames. Doubled codes stay adjacent; pauses are allowed between them. * craft-captions: CEA-708 export starts each caption at its window's top left Ported from storytold/filmcraft#4043 (filmcraft#4042). Each window load now sends DeleteWindows before DefineWindow in place of the ClearWindows after it: a window made anew is empty with the pen at its top left in any decoder, for the same number of bytes, so cue timing is unchanged. * craft-captions: TTML/DFXP export keeps literal < and > text Ported from storytold/filmcraft#2830. The writer treated every < with a later > as a tag and wrote nothing for unknown names, so "1 < 2 and 3 > 2" lost its middle. Only a well-formed tag is markup now; a > inside a quoted attribute value does not end the tag. * craft-captions: TTML/DFXP export keeps cue IDs unique and valid Unicode IDs as written Ported from the TTML commit of storytold/filmcraft#1839. A cue named s1 could collide with the style ID, and a generated c1 could take the ID of a later cue. Valid explicit IDs are reserved first; missing, invalid or repeated ones get a replacement that is in no one's way. IDs are checked as XML NCNames. * craft-captions: compare the craft-xml and roxmltree views of TTML documents The TTML reader moved from roxmltree to craft-xml. This dev-only test checks that both hand it the same elements, attributes and text, by namespace, on TTML documents and on the writer's output. roxmltree becomes a dev-dependency, which the cue ID tests use too. * craft-captions: use craft-timeline's exact time instead of a private copy craft-captions carried a 290-line subset of the tick, frame rate and timecode field code because craft-libs had none when it was added. craft-timeline now has the whole module with the same tick unit. craft_captions::time is a re-export of craft_timeline::time, so every public path still resolves and the types are now the same ones craft-timeline's callers hold. * craft-captions: SCC and MCC time codes count from the document's start timecode Ported from the caption-format part of storytold/filmcraft#4131 (filmcraft#4130). Document::start_timecode is the label of time zero; the SCC and MCC writers count from the 29.97 frame with the same label, so the label carries across rates. Readers note the rate and drop-frame mode of a file's labels in Document::labels, and Document::count_from counts cues back from a start time code. charset_warnings names the time codes the file has, which that pull request's review found missing. * craft-captions: MCC export can write a packet on every frame Ported from the caption-format part of storytold/filmcraft#4037 (filmcraft#4024). WriteOptions::every_frame_until fills every 29.97 fps frame before that time that has no caption data with a packet of CEA-608 nulls and CEA-708 padding, as SDI and frame-wrapped MXF carry one per frame. Off by default, at most 24 hours (mcc::MAX_PADDED_FRAMES), and write_to streams the file line by line. * craft-captions: README has its MCC writer section, with the every-frame option The section's text was cut off when the crate was added; only a stray heading marker was left. --------- Co-authored-by: NetterSilver <133535268+Silvertree2010@users.noreply.github.com> Co-authored-by: SeeRay11 <searay112023@gmail.com> Co-authored-by: jnohclee-rgb <336857373+jnohclee-rgb@users.noreply.github.com> Co-authored-by: JonRC <jonarc06@gmail.com> Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com> |
||
|
Dylan M. Taylor
|
491fb0095f |
craft-dxf: model::write, a drawing as tags (R12 to 2018) (#947)
* craft-dxf: model, the drawing above the tags (header, tables, blocks, layouts) Document::read indexes a tag list into header variables, table records, blocks with their entities, the ENTITIES records (a POLYLINE with its vertices, an INSERT with its attributes) and the objects, each a slice of the tag list. The typed table records, LAYOUT and DICTIONARY are made on request. Also the colour index palette, colour, lineweight and transparency values, extended data lookup and the object coordinate system. Merged from CADCraft's cadcraft-io dxf_read (5a6aa9b6) and VectorCraft's vectorcraft-cad import (566f7208). Neither app's document types appear. * craft-dxf: model entities (geometry, text, MTEXT columns, hatches, dimensions) Entity::read types an entity record: the properties every entity has, and LINE, POINT, CIRCLE, ARC, ELLIPSE, LWPOLYLINE, POLYLINE with its vertices (2D, 3D, mesh, polyface), SPLINE, RAY, XLINE, TEXT, MTEXT with columns, ATTDEF, INSERT with its attributes (multiline ones included), DIMENSION, LEADER, SOLID, TRACE, 3DFACE, HATCH (polyline and edge loops, pattern lines, seeds, gradient), VIEWPORT, IMAGE, WIPEOUT and TOLERANCE. Other types stay readable from the record's tags. model::text has the caret encoding of strings, both ways. Merged from CADCraft's cadcraft-io dxf_read (5a6aa9b6) and VectorCraft's vectorcraft-cad import (566f7208). * craft-dxf: a hatch spline edge without fit data leaves the loop's source count Before R2010 a spline edge has no 97 of its own, so the 97 after it counts the loop's source entities. It was read as the edge's fit point count and the sources were lost. * craft-dxf: model::write, a drawing as tags (R12 to 2018) Drawing::tags writes the header, classes, the nine symbol tables, blocks, entities and (R2000 and later) the dictionaries, plot style placeholder and layouts around the layers, linetypes, styles, dimension styles, blocks and entities it is given, in the model's own types. entity_tags writes one entity for a caller with its own file. Handles given with the content are kept; the rest are numbered after them. The file skeleton is VectorCraft's vectorcraft-cad writer.rs (566f7208); the MTEXT column groups are CADCraft's dxf_read/mtext_dxf.rs column_tags (5a6aa9b6). * README: keep main's crate list after the merge |