mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-12 08:52:32 +00:00
craft-math-function-scripts
1396
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dylan M. Taylor
|
6a73cd9c6b | dxf: escape and cut a code page name before logging it (#855) | ||
|
Dylan M. Taylor
|
cc6106c56a |
craft-raw: the file-format layer of camera raw files, with DNG and CRW (#862)
A new crate for #587 and #778: bytes in, the sensor data out as a neutral RawImage (samples, CFA, levels, active area, crop, orientation, colour tags, opcode lists, profile tables), and back for DNG. One Cargo feature per format: dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw). The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw, unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile and gaintable, the model and probe. Demosaicing and everything after it stay with the develop pipeline. Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the default crop's per-tag defaults, an unusable crop falling back to the active area, non-finite white levels, the white default of a linearized image, the AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their part of the image, and the CinemaDNG frame tags. |
||
|
Dylan M. Taylor
|
de1daefe8c |
craft-media: AVI and OpenDML demuxer from FilmCraft (feature avi) (#856)
Moves filmcraft-avi (storytold/filmcraft@4bec22b) into craft_media::avi. ByteSource is the crate's trait, Error::Io holds the io::Error, and Stream::codec_params() maps the FourCC and the WAVEFORMATEX tag to a CodecId. Adds Container::Avi and tests for the stream, header-size, nesting and OpenDML index limits. |
||
|
|
01cbc50269 |
metafile: dual EMF+ files draw their EMF+ records (stacked on #843) (#845)
* metafile: hold a placed pen width to what covers the frame Picture::place scaled a pen width by the fit with no upper bound, so a pen of 1e30 units made a renderer flatten round caps of that size. A placed pen is now at most twice the diagonal of the frame and the path together, which already covers the whole frame. Ported from storytold/wordcraft#5427. * metafile: one byte budget for items, EMF+ objects and clips; count clip regions when placing - Canvas::shade_shape returns when the canvas is full or the clip is empty, instead of sampling a tile that is then dropped. - EMF+ objects, the pieces of a continued object and the current and saved EMF+ clips count against the canvas's 256 MiB budget, in both directions: an object is not stored past what the canvas holds, and the canvas stops at what the objects hold. - A nested metafile held by an EMF+ image object is charged for what its items own (segments, pixels, text, clip rectangles; a shared tile or clip path once) rather than 128 bytes an item. - Picture::place charges each clip region it expands to the 500,000 element budget, for every kind of item, and expands a region once for the consecutive items that share it. Ported from storytold/deckcraft#1777, with its review comments addressed (the canvas checks include the held bytes; text and bitmap items are checked too; regions are matched by content, not by address). * metafile: a negative crop side pads the picture crop_window took each side as 0..1, so a negative side (a negative a:srcRect side in an Office file, or a picture fitted whole into a frame of another shape) was dropped and the picture was stretched over its frame. A side is now taken down to -100: the window starts before the source or ends past it, and Picture::fit places the source inside the frame at its own proportions. Ported from storytold/wordcraft#2027 (the -100 bound and the invariant test) and storytold/wordcraft#5206 (the padding test). * metafile: decode 8-bit text in the font's code page 8-bit text (WMF TEXTOUT and EXTTEXTOUT, EMF EXTTEXTOUTA and SMALLTEXTOUT) was decoded in Windows-1250, 1251 or 1252 only, and a double-byte character became a blank. - Greek, Turkish, Vietnamese, Hebrew, Arabic, Baltic and Thai character sets decode in their code pages (encoding_rs, already a workspace dependency). - Shift_JIS, UHC, GBK and Big5 pairs decode to their character, with the lead and trail ranges of each set; the character keeps the advances of both bytes. A lead byte without a trail byte is one undefined byte. - A WMF face name is read in the font's code page. - The Symbol font selected with DEFAULT_CHARSET keeps its bytes as symbol codes (U+F000 + code), as SYMBOL_CHARSET fonts do. Ported from storytold/wordcraft#5187 (the code pages, the double-byte ranges, the face name and the Symbol font) and storytold/deckcraft#792 (the Hebrew and Arabic tests). * metafile: dual EMF+ files draw their EMF+ records, as GDI+ does For a dual EMF+ file the reader played the plain EMF records and ignored the EMF+ ones. Office draws metafiles with GDI+, which does the opposite, and a file's GDI fallback can be incomplete (the EMF+ part draws a nested metafile the fallback lacks). EMF+ records now always play. In a dual file the plain EMF records are skipped, except between an EmfPlusGetDC record and the next EMF+ record. EMF+ only files and plain EMF files are read as before. Ported from storytold/deckcraft#1464. --------- Co-authored-by: Dmitrii Khlapov <19991457+dkhlapov@users.noreply.github.com> Co-authored-by: David Lin <woei66@gmail.com> Co-authored-by: Harrison Booth <73434116+Harrison-Booth@users.noreply.github.com> |
||
|
|
f19f131247 |
craft-audio: a damaged FLAC keeps its timing; AAC uses BitReader::peek (#846)
* craft-audio: AAC Huffman lookahead uses BitReader::peek `BitReader::peek` reads from one 8-byte load since the bit reader was made faster, which is what the AAC decoder's private `peek32` did. The private copy goes; the lookahead is the same zero-padded 32 bits. Ported from storytold/filmcraft#4232. * craft-audio: a damaged FLAC keeps its length and timing symphonia's FLAC demuxer drops a frame whose checksum fails and resyncs on the next one, so the decoder never saw an error: the file decoded shorter and all later audio came out one block early, with `damaged` at 0. The jump in the packet timestamps is the audio that was lost. It is returned as silence, in blocks of at most 65 536 frames, and counted once in `AudioInfo::damaged`. It never passes the length the file declares; with no declared length a jump of more than 2^20 frames is not believed. A seek into the lost frames lands on silence at the right position. Ported from storytold/soundcraft#556, onto the streaming decoder. --------- Co-authored-by: vangel76 <vangel1976@googlemail.com> Co-authored-by: Gert Pellin <gert@pellin.be> |
||
|
Dylan M. Taylor
|
ba3b3a892b | craft-audio: document that decode and decode_with keep every sample, and what to pass for untrusted files (#826) | ||
|
Dylan M. Taylor
|
0226f1d03e |
craft-video: H.264 decoder from FilmCraft, with DeckCraft's hardening (h264-decode) (#834)
* craft-video: H.264 decoder from FilmCraft (`h264-decode` feature) FilmCraft's crates/h264 at 4c0d2ad7 becomes craft_video::h264::decode. The encoder moves to h264/encode and is still exported from craft_video::h264. Feature h264 now turns on h264-decode and h264-encode, so either half can be built without the other. Without the threads feature a reference row that was never published read through expect(); it now reads as a gray row. craft_video::budget (the picture memory budget from DeckCraft's bitstream crate) is added for the next commit. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its codecs. * craft-video: H.264 decoder hardening from DeckCraft's copy - The picture store (DPB, current picture, pictures in flight or queued for output) is checked against a memory budget before a picture is allocated; Decoder::set_memory_budget. - configure_avcc checks every entry before handling any: SPS NAL units in the first list, PPS in the second. A picture slice in the record starts no picture. - The inverse transforms clamp coefficients to the spec's r range instead of overflowing. - Temporal direct motion vectors are scaled in i32 and clipped to the spec range. - slice_qp_delta is added with a checked add. - A picture job sets up its picture state inside the panic guard. |
||
|
|
d1f13fbea2 |
craft-heif: crop to the declared size, read_metadata, stereo test data (ports of open LightCraft PRs) (#841)
* craft-heif: stills decode at the size the container declares A coded HEVC picture is a whole number of coding blocks and can be larger than the image the container's ispe declares (Sony codes 1920x1080 as 1920x1088 without a conformance window). decode returned the coded size, so probe and decode disagreed and the padding rows showed as a dark band. decode_item now crops to ispe before the alpha merge and the transforms, measures the clap parity on the declared size, and compares the alpha item's declared size with the picture's instead of the coded planes. A declared size larger than the coded picture is Malformed. Ported from storytold/lightcraft#3977. * craft-heif: read_metadata reads the primary image's EXIF and XMP without decoding Cataloguing a photo needs its EXIF and XMP, not its pixels. read_metadata opens the container under the same panic guard and box census as probe and returns what decode would attach. Ported from storytold/lightcraft#2087. * craft-heif testdata: Spec::stereo builds a spatial photo (grpl/ster entity group) A second image of the primary's size and a ster entity group naming both, so callers can test stereo detection on a file this crate decodes. Ported from storytold/lightcraft#1684. * craft-heif: put_alpha uses is_multiple_of --------- Co-authored-by: Mason <mason276752@gmail.com> Co-authored-by: jkalkstein <76634771+jkalkstein@users.noreply.github.com> Co-authored-by: Mitch Lindgren <mitch@lindg.ren> |
||
|
Dylan M. Taylor
|
562327b58c | craft-video: Vp9Encoder::encode_frame refuses planes shorter than the configured size (port of effectcraft #2995) (#832) | ||
|
Dylan M. Taylor
|
4e92a95e41 | craft-video: the DNx decoder refuses repeated or decreasing scan-line offsets (port of filmcraft #4101) (#833) | ||
|
Dylan M. Taylor
|
b16a2584fe | craft-media tests: scratch files live in a directory only the test created (#839) | ||
|
Dylan M. TaylorandLloyd Nicholson
|
b72cb4d68c |
craft-mesh OBJ: map options by argument count, material libraries read once (from open EffectCraft PRs) (#854)
* craft-mesh OBJ: map_* options are passed over by their argument counts A map line with options named its image by its final token, so 'map_Kd -o 0.5 0.5 my checker.png' named 'checker.png'. Options are now skipped by the arguments they take (-o, -s and -t take one to three numbers), so a name with spaces after options is one name. After an option the reader does not know, the name is still the final token. Ported from storytold/effectcraft#1324. * craft-mesh OBJ: a material library named again is read once Exporters repeat the mtllib line for every object. Each repeat asked the resolver again, read the library again, added its materials and images a second time, and counted its bytes against max_data_bytes once per line, so a legitimate file with many objects could reach the limit. Ported from storytold/effectcraft#2965 (its OBJ part: 'a library already loaded is skipped'). --------- Co-authored-by: Lloyd Nicholson <nichlloyd@gmail.com> |
||
|
|
8f1f35aa19 |
craft-xml: whole-tree attribute limit, node_count, parse_streaming_partial, strict pull reader (from open app PRs) (#850)
* craft-xml: a limit on the attributes a tree holds over all its elements max_attrs bounds one start tag and max_nodes counts elements and text, so attributes were the one thing nothing counted across the tree: 1,025 elements of 1,024 attributes each are 1,025 nodes and over a million (String, String) pairs, ten times the input in headers alone. Limits::max_attrs_total (default 4,000,000) bounds the attributes kept at once; elements handed to a streaming callback take theirs out of the count. Error::TooManyAttrsTotal. Ported from storytold/wordcraft#4320, which added the same budget to WordCraft's DOCX reader. * craft-xml tests: an internal DTD subset is refused wherever it hides, in linear time The cases of storytold/filmcraft#4139 (a marker inside a comment or a processing instruction, a '>' inside the system literal, 100,000 markers that never close) against Doctype::RejectInternalSubset. The reader already handles them; the tests keep it so. * craft-xml: El::node_count, for one node budget over the parts of a package max_nodes bounds one parse. A package of many parts, each inside the limit, still costs a tree per part the reader keeps. node_count is what max_nodes counted for an element, so a reader charges each tree against one budget and reads the next part with what is left as its max_nodes: a part over the budget stops before its tree is built. Ported from storytold/wordcraft#5665, which adds a package-wide element budget to WordCraft's DOCX reader (parse_counted with a max). * craft-xml: parse_streaming_partial returns what was read when a streaming read fails A worksheet whose rows cannot all be read (one row over the node limit, a callback error, markup the tokenizer cannot read) lost everything else in the part with it. parse_streaming_partial returns the error together with the root as far as it was read: the children complete before the failure and the elements open at that point. parse_streaming is unchanged. Ported from storytold/gridcraft#3317 (xml::parse_streaming_partial). * craft-xml: a strict pull reader refuses a second root and text outside the root Options::events builds no tree, so the checks the tree reader makes for a strict read (a second root element, character data outside the root) were not made at all: a strict pull over '<a/><b/>' or '<a/>text' ended without an error. A reader that walks events would take a damaged file's first element for the whole document. The reader now makes both checks itself, so the pull reader and the tree agree. Ported from storytold/wordcraft#1862, which made WordCraft's ODT event reader report malformed XML instead of stopping silently. --------- Co-authored-by: Oliwer Urbaniak <oliwerx12@gmail.com> Co-authored-by: Bra1nZ <95569966+Bra1nZ@users.noreply.github.com> Co-authored-by: Neil Cawse <neilcawse@geotab.com> Co-authored-by: midasmatterhorn <248507632+midasmatterhorn@users.noreply.github.com> |
||
|
|
ebb40fa0c9 |
craft-dxf: R12 binary, more code pages, borrowing sections, exponent reals (ports of open CadCraft PRs) (#847)
* craft-dxf: the ASCII reader unescapes only text that holds a backslash Only a string with a backslash can hold a \U+ or \M+ escape; every other string moves into the tag as it is instead of being copied. The EOF check reads the value before it moves. Ported from storytold/cadcraft#1833. * craft-dxf: write_ascii_in_page writes the code page without the UTF-8 fallback write_ascii_in falls back to pure ASCII when the code page bytes happen to be valid UTF-8, for readers that guess UTF-8. A reader that decodes a pre-2007 DXF strictly by its $DWGCODEPAGE needs the code page bytes. Ported from storytold/cadcraft#2851. * craft-dxf: behaviour-preserving cleanups Exact group-code mappings in one place, documented errors on the fallible public parsers, and simpler option, result and formatting idioms. No change to parsing, writing, numeric conversion, encoding or the public API. Ported from storytold/cadcraft#2571. * craft-dxf: read R12 and earlier binary DXF (1-byte group codes) Binary DXF written before R13 stores each group code in one byte, with 255 escaping a 2-byte code. The reader only knew the R13+ layout, where codes are two bytes, and also treated 255 as an escape there. The layout is now told from the first tag; 255 is an escape only in the 1-byte layout and an ordinary code in R13+. Ported from storytold/cadcraft#1499. * craft-dxf: section_slices, record_slices and Tag::str_ref borrow instead of copying sections() and records() copy every tag out of the parsed list, so a reader that keeps the list holds a large file's tags two or three times. The new functions return slices of the list. sections() and records() now build their copies from them, so there is one implementation of each split. Ported from storytold/cadcraft#1708. storytold/cadcraft#1588 solves the same problem by changing Section itself; this keeps the existing API. * craft-dxf: reals beyond 1e16 or below 1e-10 are written in exponent form; string_groups format_real wrote f64::MAX as 309 digits. Values outside 1e-10..1e16 are now written as AutoCAD writes them (1.0E+20) and read back exactly. string_groups splits text into the 250-character groups long strings are stored in (MAX_STRING_CHARS). Ported from storytold/cadcraft#1790. * craft-dxf: DOS 720/737/775/855/857/860/861/863/864/865/869 and ISO 8859-15 code pages; cheaper unescape and hex $DWGCODEPAGE values DOS720, 737, 775, 855, 857, 860, 861, 863, 864, 865, 869 and ISO8859_15 had no decoder and fell back to Windows-1252. ISO 8859-15 decodes through encoding_rs, the DOS pages through tables checked against the Unicode Consortium mapping files (dos_tables.rs). DOS 864's byte 0x25 is U+066A, so a writer escapes '%' there. U+FFFD is never written as a table byte. A string without an escape moves into its tag without a copy, cstr finds its terminator with one scan, and binary chunks are hex-encoded from a table. Ported from storytold/cadcraft#1271. Its group-code-255 change is carried by the R12 binary port: the 1-byte layout is detected only when the escaped code is one that needs the escape, so an R13+ file starting with code 255 still reads. * craft-dxf: Korean Johab (ANSI_1361) text and \M+4 escapes decode Johab had no decoder: files declaring ANSI_1361 were read as Windows-1252 and \M+4 escapes stayed literal. Hangul syllables are composed from the three 5-bit jamo indices (KS X 1001 annex 3); symbols and Hanja map onto KS X 1001 rows decoded as EUC-KR. Read only: a writer escapes instead. Ported from storytold/cadcraft#1911 (its DOS code pages are covered by the tables from storytold/cadcraft#1271). --------- Co-authored-by: mo <131036780+mbolling@users.noreply.github.com> Co-authored-by: ffishman <20211677+ffishman@users.noreply.github.com> Co-authored-by: Cédric Legrand <cedric@legrand-tech.fr> Co-authored-by: valentinevg812-eng <valentinevg812@gmail.com> Co-authored-by: Mads Holten Rasmussen <mads.h.rasmussen@gmail.com> Co-authored-by: LastSkywalker <maksim@sky-tehnol.uk> Co-authored-by: vangel76 <vangel1976@googlemail.com> Co-authored-by: Francesco BV <francesco.esposito@bvitalia.net> |
||
|
|
9adca4a8e5 |
craft-tiff: shared child IFDs, multi-string ASCII, parse_with_status (ports of open LightCraft PRs) (#842)
* craft-tiff: keep shared child IFDs and every string of a multi-string ASCII value Two pointers to the same child IFD (two SubIFDs entries, or an Exif IFD shared by two directories) were treated as a loop and the second was dropped. Loops are now detected on the current parent path only; a shared IFD is parsed once and copied for later pointers, and each copy is charged its IFDs, entries and value bytes so max_ifds, max_entries_total and the value budget still hold. ASCII values keep every NUL-separated string (Value::as_strings); as_str and as_bytes return the first. Ported from storytold/lightcraft#3002. * craft-tiff: parse_with_status reports whether the whole IFD structure was read The reader is best effort: truncated tables, over-budget values, dangling pointers and unfollowed children are skipped silently. A caller that must tell a clean absence from something it could not inspect (a privacy scan for GPS data) gets the same tree plus a completeness flag. Also adds the PHOTOSHOP_IMAGE_RESOURCES and GPS destination tag constants. Ported from storytold/lightcraft#3766. On top of the shared-child change, a second pointer to an IFD already read counts as read, and a next pointer that loops back to the chain is incomplete. --------- Co-authored-by: Andrew Frahn <andrew@emmertex.com> Co-authored-by: ronibalthazar <61232335+ronibalthazar@users.noreply.github.com> |
||
|
Dylan M. Taylor
|
12e24c1029 |
svg-paths: CSS colour parser and a small SVG writer (#857)
color: parses the CSS colours SVG paint uses (hex, rgb()/hsl(), the 148 named colours) into an opaque 8-bit sRGB Color, and writes the canonical #rrggbb form. Shape::stroke is still returned as written; parse_color reads it. write: SvgWriter (XML declaration, root element with the SVG namespace, elements and attributes escaped through craft-xml, caller-controlled line breaks), path data in shortest round-trip form or with fixed decimals, and fixed-decimal numbers that are never NaN or infinite. |
||
|
Dylan M. Taylorandvangel76
|
e7814b1727 |
craft-media: Opus timing treats an out-of-range Ogg granule as missing (#859)
A granule outside 0..=MAX_OPUS_GRANULE (negative other than -1, or absurdly large) no longer moves the packet positions or the total: the page's packets continue from the previous position. Ported from storytold/filmcraft#4049. Co-authored-by: vangel76 <vangel1976@googlemail.com> |
||
|
Dylan M. Taylor
|
725ce2bdf2 | craft-cms: read mAB/mBA elements through one checked slice (#823) | ||
|
|
75895c0321 |
metafile: pen width cap, shared byte budget, negative crops, code pages (#843)
* metafile: hold a placed pen width to what covers the frame Picture::place scaled a pen width by the fit with no upper bound, so a pen of 1e30 units made a renderer flatten round caps of that size. A placed pen is now at most twice the diagonal of the frame and the path together, which already covers the whole frame. Ported from storytold/wordcraft#5427. * metafile: one byte budget for items, EMF+ objects and clips; count clip regions when placing - Canvas::shade_shape returns when the canvas is full or the clip is empty, instead of sampling a tile that is then dropped. - EMF+ objects, the pieces of a continued object and the current and saved EMF+ clips count against the canvas's 256 MiB budget, in both directions: an object is not stored past what the canvas holds, and the canvas stops at what the objects hold. - A nested metafile held by an EMF+ image object is charged for what its items own (segments, pixels, text, clip rectangles; a shared tile or clip path once) rather than 128 bytes an item. - Picture::place charges each clip region it expands to the 500,000 element budget, for every kind of item, and expands a region once for the consecutive items that share it. Ported from storytold/deckcraft#1777, with its review comments addressed (the canvas checks include the held bytes; text and bitmap items are checked too; regions are matched by content, not by address). * metafile: a negative crop side pads the picture crop_window took each side as 0..1, so a negative side (a negative a:srcRect side in an Office file, or a picture fitted whole into a frame of another shape) was dropped and the picture was stretched over its frame. A side is now taken down to -100: the window starts before the source or ends past it, and Picture::fit places the source inside the frame at its own proportions. Ported from storytold/wordcraft#2027 (the -100 bound and the invariant test) and storytold/wordcraft#5206 (the padding test). * metafile: decode 8-bit text in the font's code page 8-bit text (WMF TEXTOUT and EXTTEXTOUT, EMF EXTTEXTOUTA and SMALLTEXTOUT) was decoded in Windows-1250, 1251 or 1252 only, and a double-byte character became a blank. - Greek, Turkish, Vietnamese, Hebrew, Arabic, Baltic and Thai character sets decode in their code pages (encoding_rs, already a workspace dependency). - Shift_JIS, UHC, GBK and Big5 pairs decode to their character, with the lead and trail ranges of each set; the character keeps the advances of both bytes. A lead byte without a trail byte is one undefined byte. - A WMF face name is read in the font's code page. - The Symbol font selected with DEFAULT_CHARSET keeps its bytes as symbol codes (U+F000 + code), as SYMBOL_CHARSET fonts do. Ported from storytold/wordcraft#5187 (the code pages, the double-byte ranges, the face name and the Symbol font) and storytold/deckcraft#792 (the Hebrew and Arabic tests). --------- Co-authored-by: Dmitrii Khlapov <19991457+dkhlapov@users.noreply.github.com> Co-authored-by: David Lin <woei66@gmail.com> |
||
|
|
ce31f42e76 |
craft-media: QuickTime logs box, Matroska Colour in the muxer, laced remux test (#840)
* craft-media: read and write the QuickTime logs box A video sample entry's `logs` child holds the CoreMedia log transfer-function identifier (`com.apple.rec2020.apple-log` for Apple Log). It is kept as `VideoParams::log_transfer_function`, whatever curve it names. The reader takes non-empty UTF-8 without NUL of at most 255 bytes; anything else, or a second `logs` box, reads as unknown. The writer refuses an identifier the reader would not accept. Ported from storytold/filmcraft#1764. * craft-media: the Matroska muxer writes a video track's Colour element `TrackSpec::color` takes the reader's `Color` and the muxer writes the fields that are set: matrix, range, transfer, primaries, chroma layout, light levels and mastering metadata (RFC 9559). VP9's own colour syntax carries only the matrix and range, so without the element a Rec. 2020, P3, PQ or HLG WebM reads as BT.709. Ported from storytold/effectcraft#2926, which wrote the element by hand in EffectCraft's WebM export. * craft-media: the laced remux test laces ffmpeg's Vorbis fixture again ffmpeg's Vorbis files declare a short DefaultDuration and store longer durations per block, so with the track's own default nothing laced and the test failed ("vorbis.mkv Xiph: nothing was laced", ffmpeg 9.0.2). Audio tracks now take their median packet duration as the default, and frames are written with their explicit durations, so a short last packet ends its lace. Ported from storytold/filmcraft#3622 (test-only part). --------- Co-authored-by: 20Coder <51251284+BitL8-ByteShort@users.noreply.github.com> Co-authored-by: JonRC <jonarc06@gmail.com> Co-authored-by: Michael Butler <michaelbutler75@gmail.com> |
||
|
Dylan M. Taylor
|
c814fdfe7c | craft-audio: bound MP4 audio stts entries and frames x channels per packet (port of soundcraft #1332) (#825) | ||
|
Dylan M. Taylor
|
0e60ef3b0a |
craft-audio: extension and media-type tables, linear WAV chunk scan, SoundCraft's later tests (#824)
DECODE_EXTENSIONS lists the file extensions of the formats decode reads. EncodeFormat gains ALL, extension() and mime_type(). SoundCraft kept these tables in its own FileFormat type; they are format knowledge, so they live here. A WAV with many empty data chunks parsed in quadratic time: whether an empty data chunk is followed by chunks alone was decided for every one of them, by walking the rest of the file. Only the first data chunk is used, so it is decided once. Tests SoundCraft added to audio-io after craft-audio was cut from it: Matroska element sizes beyond the file, an AIFF channel count with the sign bit set, probe and decode agreeing on the length of AAC in MP4, and one small file per decodable extension (tests/fixtures). |
||
|
Dylan M. Taylor
|
1175abec2e |
craft-video: HEVC decoder from FilmCraft, with DeckCraft's hardening (hevc-decode) (#828)
* craft-video: HEVC decoder from FilmCraft (`hevc-decode` feature) FilmCraft's crates/hevc at 4c0d2ad7 becomes craft_video::hevc::decode. The encoder moves to hevc/encode and is still exported from craft_video::hevc. Feature hevc now turns on hevc-decode and hevc-encode, so either half can be built without the other. craft_video::budget (the picture memory budget from DeckCraft's bitstream crate) is added for the next commit. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its codecs. * craft-video: HEVC decoder hardening from DeckCraft's copy - The picture store (DPB, current picture, pictures in flight) is checked against a memory budget before a picture is allocated; Decoder::set_memory_budget. - configure_hvcc checks every array before handling any NAL: only VPS, SPS, PPS and SEI arrays, each NAL header agreeing with its array type. A picture slice in the record starts no picture. - init_qp_minus26, slice_qp_delta and delta_chroma_log2_weight_denom are range checked before the additions that overflowed on Exp-Golomb values near the i32 limits. - A picture job sets up its picture state inside the panic guard. |
||
|
Dylan M. Taylor
|
4c6fe7c864 | craft-cms: confirm ICC identity by bytes, not the 64-bit hash alone (#822) | ||
|
Dylan M. Taylor
|
4e309f8458 |
craft-heif: cap hvcC parameter sets and tiles, count alpha-grid tiles (port of lightcraft #3645) (#812)
* craft-heif: cap hvcC parameter sets and tiles, count alpha-grid tiles in the visit budget * craft-heif: charge every tile reference of a thumbnail candidate to the visit budget |
||
|
Dylan M. Taylor
|
6ac2a6a34a |
craft-video: VP9 decoder from FilmCraft, with DeckCraft's memory budget (vp9-decode) (#821)
* craft-video: VP9 decoder from FilmCraft (`vp9-decode` feature) FilmCraft's crates/vp9 at 4c0d2ad7 becomes craft_video::vp9::decode. The encoder moves to vp9/encode and is still exported from craft_video::vp9. Feature vp9 now turns on vp9-decode and vp9-encode, so either half can be built without the other. The decoder's tests find ffmpeg and the fixture directory through craft-media-testkit; a fixture that cannot be generated fails under CRAFT_MEDIA_REQUIRE_ORACLES=1. craft-video is built at opt-level 3 in the dev profile, as FilmCraft built its codecs. * craft-video: VP9 decoder checks its picture store against a memory budget DeckCraft's copy of the decoder refuses a frame size whose pictures (eight reference slots, the new frame and those in flight) would not fit a byte budget. The budget type comes with it as craft_video::budget; FilmCraft's 2^26 luma sample cap stays. |
||
|
Dylan M. Taylor
|
9dfc35965c |
Add craft-metafile: WMF and EMF reader and writer (#830)
* craft-metafile: DeckCraft's WMF, EMF and EMF+ reader with WordCraft's changes merged The base is DeckCraft's crates/metafile at cf0d7fbb. WordCraft's changes since the copy was taken (WordCraft 05679497, the ancestor) are merged three ways from WordCraft 639b8133: the cropped, depth-once DIB decoder, the bitmap side cap, the header-only size(), crop_window, the free-slot object table, capacity-based budget charging and the exact reservation in subpaths. WordCraft's own text records lose to DeckCraft's richer text model. All 120 DeckCraft tests and all 69 WordCraft tests are kept (132 distinct). * craft-metafile: line styles, clip paths and the records behind them, from VectorCraft's reader The model gains what VectorCraft's reader kept and the other two dropped: LineStyle (cap, join, miter limit, dashes) on paths, and clip paths (SELECTCLIPPATH) shared between the items they clip. The reader now plays EXTCREATEPEN style entries and the pen's cap and join bits, SETMITERLIMIT, SETARCDIRECTION, SCALEVIEWPORTEXTEX, SCALEWINDOWEXTEX, SELECTCLIPPATH and ABORTPATH; STROKEANDFILLPATH closes open figures and CLOSEFIGURE moves the current position; the header's micrometre device size is used when present; BITMAPCOREHEADER, BI_JPEG and BI_PNG bitmaps decode; records of unknown types are counted in Picture::skipped_records. Each has a test. * craft-metafile: EMF and WMF writers behind the write feature, bitmap transforms, a decode budget The writers are VectorCraft's, taking a list of drawing operations (fills, strokes with pens, images, clips) instead of its document; the document walk stays in the application. Bitmap items gain a transform, so an image drawn under a turning world transform keeps its turn (the writer places images that way). JPEG and PNG bitmaps share one per-file pixel budget. The tests cover the writers record by record, round-trip both formats through the reader, and play VectorCraft's hand-built files into the picture model. * craft-metafile: a test per limit, README, crate docs Adds tests for the limits no test held to their value (string lengths, clip region rectangles, placed elements, tile side, palette size, nesting, the EMF+ object table, save stack, point lists, gradient stops, region depth, continued objects, bitmap masking work) and a truncation and mutation sweep over the new records. Records that change what is drawn but are not played now count as skipped instead of ignored. The README has the coverage tables, the limits, what stays in the apps, the origin of each part and a migration table per app. * craft-metafile: a region node cap test, the box of a turned bitmap The region node cap gets a test of its own (a balanced tree of intersections), and a turned bitmap's rectangle is the box around its four corners, so a caller that draws rectangles still has an area to draw into. |
||
|
Dylan M. Taylor
|
b59aeeb10a | craft-cms: per-profile budgets for decoded curve samples and CLUT values (#815) | ||
|
Dylan M. Taylor
|
2408292033 | craft-heif: cap zero-width iloc extents, apply the census on the thumbnail path (#813) | ||
|
Dylan M. Taylor
|
cf14284af1 | craft-tiff: cap chunk descriptors at the file size and saturate the grid product (#827) | ||
|
Dylan M. Taylor
|
8f3e83aaee |
Add craft-mesh: STL, OBJ/MTL, PLY and glTF/GLB behind one neutral mesh model (#819)
* Add craft-mesh: neutral mesh model with STL, OBJ/MTL and PLY * craft-mesh: glTF 2.0 and GLB * craft-mesh: README, crate list entry, a test for the UV set bound |
||
|
Dylan M. Taylor
|
06af3f2d46 |
Add craft-cfb: Compound File Binary reader and writer (#818)
The base is FilmCraft's filmcraft-cfb (reader, writer, tests) at filmcraft 6d5269ed. Ported on top of it, each with a test: From GridCraft's and DeckCraft's readers: only sector shifts 9 and 12; a file that ends inside its last sector still reads; a chain loop shorter than the stream is an error instead of repeated data; a damaged mini stream no longer fails the open; links to missing directory entries are ignored; a stream size cap (Limits, default 1 GiB, and a per-call cap); a lenient directory scan (directory_names, has_entry_named) for files too damaged to open. From WordCraft's guard: a FAT sector listed twice is refused; spare FAT sectors and zero-padded DIFAT sectors are accepted. New: Limits::max_dir_entries, walk() and path() to list entries, has_stream, Writer::stream_at and write_streams, and only storages the root reaches are walked. |
||
|
Dylan M. Taylor
|
8152fc0be0 |
Add craft-xml: bounded XML tree, streaming and writer (#816)
Almost every Craft app carries its own small XML helper: a bounded element tree on a pull parser and an escaping string writer. This is the shared one, so each app can drop its copy. - An owned tree (El, Node) read from bytes with limits on input size, depth, node count and attributes per element. A limit exceeded is an error. No DTD is processed and no entity is expanded beyond the five predefined ones and numeric character references. UTF-8 and UTF-16. - Names as written, by local name, or rewritten to the caller's canonical prefixes by namespace URI, with hooks for a prefix that depends on the parent and for attribute values that hold prefixes. - Lenient and strict reading; streaming of repeated elements to a callback; first_start; a pull reader with byte ranges. - Accessors, typed attribute getters, mutation and a builder. Where the apps disagreed (whole or local names, own or deep text, exact or lossy numbers) there are two spellings. - esc, esc_attr, compact and pretty serialising, and a Writer. A name that is not an XML name is never written. Built on quick-xml 0.41's tokenizer (MIT), the version the apps use. The apps' own tests for their helpers are ported in tests/apps.rs. No app is switched here. |
||
|
Dylan M. Taylor
|
d4e3017c34 |
Add craft-mime: Internet message and MIME reader and writer (#814)
MailCraft and WordCraft each parse MIME: MailCraft for mail (crates/mime, crates/charset), WordCraft for MHT web archives (formats/src/mht.rs). This adds one shared crate for the format, taken from MailCraft's code where it is format code, with WordCraft's multipart/related lookups and its fix for comments in Content-Type. Reader: parse() gives a tree of parts (header fields, content type and parameters, decoded body, charset-decoded text). It never fails or panics, is linear in the input, and is bounded by Limits (input size, header count and sizes, depth, part count, decoded size); Message::truncated reports a limit that left something out. Writer: Entity::text / file / multipart and write_message, with folded headers, RFC 2047 and RFC 2231 encoding, quoted-printable and base64 with line limits, and boundaries checked against the content. Every header value loses CR, LF and NUL, and a name that is no field name is not written. Also: address lists, mbox splitting and writing on raw message bytes, and Date parsing and formatting on a plain struct (chrono is a private dependency; no date type is in the public API). No app is switched here. The README has the migration tables and says what stays in the apps. |
||
|
Dylan M. Taylor
|
27f2204603 |
craft-media: DeckCraft's two Ogg fixes (#810)
Packet byte ranges are checked against the source before a buffer is sized for them (deckcraft 51d19baa), and packets after the last known granule count towards the Opus duration (deckcraft 9d36b593). Each test failed before its fix. |
||
|
Dylan M. Taylor
|
56d088bf04 |
craft-audio: flac::Encoder, the FLAC packet encoder, is public (#809)
* craft-audio: FilmCraft's FLAC encoder and MP1/MP2 decoder speed-ups FLAC: per-order predictor kernels, merged Rice partition sums, the LPC window kept per block length, and batches consumed by offset (filmcraft 5eaff794, d9dad4a5). The output is unchanged: 64 golden cases hash the same before and after. MP1/MP2: the faster synthesis filterbank (filmcraft 51c755e5), with its bit-exact test. * craft-audio: FilmCraft's AAC and Opus decoder speed-ups and refactors AAC: table-driven Huffman decoding, per-stage FFT twiddles and fused stages, reused per-channel work buffers, and the encoder's memoised per-band work (filmcraft f9627280, d587ceed, 6121635c, dc124ab7, 7a532eca). Opus: radix-2/3/4 FFT kernels in the CELT IMDCT, the PVQ table built once, one de-emphasis loop for both channels, the SILK resampler without per-sample divisions and with eight partial sums, concealment and FEC capped at 120 ms (filmcraft bf7ac420, 12e32026, c4dc64d1, eb352ec8, 87b8e22d, 014b5e56, 78d77b72). The per-frame mid-only flag this crate fixed in the SILK decoder is kept in the refactored decode. * craft-media: Ogg CRC-32 by slice-by-16 FilmCraft's 6763f9bf: sixteen independent lookups per 16 input bytes, so opening a file is several times faster. Checked against a bit-at-a-time reference on every short length and alignment, and across every split. * craft-audio: SoundCraft's CAF cookie-size and FLAC vendor-length checks A CAF magic cookie size above the signed range is refused instead of passed to symphonia (soundcraft 3bd3965), and a Vorbis comment block whose vendor string claims more than the block holds is refused (soundcraft ddfe991). Each with its regression test. * craft-audio: AC-3 reads bits through the shared BitReader FilmCraft's 1c9c2e82: the AC-3 decoder's private bit-at-a-time reader is replaced by craft-media's BitReader, with its two tests: the decoded output of 2,000 synthetic and 12 real syncframes (and their damaged variants) hashes the same as before, and the reader agrees with the old one on random reads, skips and overruns. The three AC-3 features now turn on craft-media's bitstream module. * craft-audio: the quantiser rounds without a libm call SoundCraft's d293336, the audio-io part: round-half-away by integer arithmetic in the bounce's inner loop, with its test against f64::round over the quantiser's range. * craft-audio: flac::Encoder, the FLAC packet encoder, is public FilmCraft muxes FLAC into its own outputs from integer samples, so it needs the encoder below FileEncoder: planar samples in, frames out, and the STREAMINFO to patch in at the end. The module is behind flac-encode, its errors convert to AudioError::Encode, and pending_samples says how long the final frame will be. |
||
|
Dylan M. Taylor
|
44b4a23137 |
craft-audio, craft-media: the changes FilmCraft and SoundCraft made to their copies since extraction (#807)
* craft-audio: FilmCraft's FLAC encoder and MP1/MP2 decoder speed-ups FLAC: per-order predictor kernels, merged Rice partition sums, the LPC window kept per block length, and batches consumed by offset (filmcraft 5eaff794, d9dad4a5). The output is unchanged: 64 golden cases hash the same before and after. MP1/MP2: the faster synthesis filterbank (filmcraft 51c755e5), with its bit-exact test. * craft-audio: FilmCraft's AAC and Opus decoder speed-ups and refactors AAC: table-driven Huffman decoding, per-stage FFT twiddles and fused stages, reused per-channel work buffers, and the encoder's memoised per-band work (filmcraft f9627280, d587ceed, 6121635c, dc124ab7, 7a532eca). Opus: radix-2/3/4 FFT kernels in the CELT IMDCT, the PVQ table built once, one de-emphasis loop for both channels, the SILK resampler without per-sample divisions and with eight partial sums, concealment and FEC capped at 120 ms (filmcraft bf7ac420, 12e32026, c4dc64d1, eb352ec8, 87b8e22d, 014b5e56, 78d77b72). The per-frame mid-only flag this crate fixed in the SILK decoder is kept in the refactored decode. * craft-media: Ogg CRC-32 by slice-by-16 FilmCraft's 6763f9bf: sixteen independent lookups per 16 input bytes, so opening a file is several times faster. Checked against a bit-at-a-time reference on every short length and alignment, and across every split. * craft-audio: SoundCraft's CAF cookie-size and FLAC vendor-length checks A CAF magic cookie size above the signed range is refused instead of passed to symphonia (soundcraft 3bd3965), and a Vorbis comment block whose vendor string claims more than the block holds is refused (soundcraft ddfe991). Each with its regression test. * craft-audio: AC-3 reads bits through the shared BitReader FilmCraft's 1c9c2e82: the AC-3 decoder's private bit-at-a-time reader is replaced by craft-media's BitReader, with its two tests: the decoded output of 2,000 synthetic and 12 real syncframes (and their damaged variants) hashes the same as before, and the reader agrees with the old one on random reads, skips and overruns. The three AC-3 features now turn on craft-media's bitstream module. * craft-audio: the quantiser rounds without a libm call SoundCraft's d293336, the audio-io part: round-half-away by integer arithmetic in the bounce's inner loop, with its test against f64::round over the quantiser's range. |
||
|
.CooCoo
|
6e03f65bf4 |
craft-update: a failed check for a running attach is reported on macOS (#803)
detach_unmounted leaves a disk image's attachments alone when the check for a running `hdiutil attach` of the image gets no answer, and it returned Ok in that case. A failed install then ended with no note that the image may still be attached. It now returns the check's reason as an error, so the install's error says "the disk image may still be attached" with that reason, and craft-update logs it at warn level. |
||
|
Dylan M. Taylorandwhatever-industries
|
2765c84bda |
craft-audio: AC-3 and E-AC-3 packet encoders (#806)
* feat(audio): add AC-3 and E-AC-3 packet encoders * craft-audio: AC-3 encoder errors convert to AudioError::Encode --------- Co-authored-by: whatever-industries <77seventy77@users.noreply.github.com> |
||
|
el-j
|
9f10d434d9 |
craft-video: clean-room video codecs, one feature per codec (#702–#708)
The eight-part craft-video series landed as one commit, because each part's branch contains the parts before it. - #702 dnx: DNxHD/DNxHR decoder and encoder - #703 vp8: VP8 decoder - #704 prores: ProRes decoder and encoder - #705 h264: H.264 encoder - #706 hevc: HEVC encoder - #707 vp9: VP9 encoder - #708 av1: AV1 encoder |
||
|
Aleksander HelgakerandDylan M. Taylor
|
7e40e12989 |
craft-audio: ALAC decoder and encoder (#731)
* craft-audio: ALAC decoder and encoder, ported from Apple's reference Add `craft_audio::alac`, a Rust port of Apple's ALAC reference implementation (macosforge/alac@c38887c, Apache-2.0): - `alac::Decoder`: packets to planar i32, bit depths 1-32, 1-8 channels, cookies in MP4/Matroska (24-byte config), CAF and old QuickTime form. Hostile input returns errors; arithmetic wraps where C would overflow. - `alac::Encoder`: 16/20/24/32 bits, 1-8 channels, fast mode, any frame length; same packets as the reference for the same input. - `SpecificConfig` (cookie parse/write), `wav_order`/`alac_order`, `channel_mask`. - Features `alac-decode` (in `decode-native`) and `alac-encode` (in `encode`). `TrackDecoder` decodes ALAC natively, so ALAC in MP4 and Matroska no longer goes through symphonia (CAF still does). Apple's reference writes the cookie's channel layout tag in host byte order; this writes it big-endian as its cookie description specifies and reads either. Tests: unit tests for each part, an MP4 decode-and-seek test without symphonia, and ffmpeg oracles both ways (bit-exact, 1-8 channels). NOTICE and each file's header record the Apache-2.0 origin. * craft-audio: ALAC review fixes (#731) - Decode the full-width residual escape 0xffffffff as i32::MIN; the reference's `nd + 1` wraps it to 0. - Require a whole element tag after the last channel (the end tag, after any fill or data stream elements), so a packet that lost its last byte is an error instead of decoding without damage. Other elements there end the packet, as in the reference. - Refuse an ALAC track with no sample rate in its cookie or container. - Conceal a damaged packet with the length its header gives (`alac::Decoder::packet_frames`), falling back to the last packet's. - Matroska: place ALAC packet starts at multiples of the frame length instead of the rounded block timestamps, so seeks land on the exact frame. MP4 keeps its exact sample-table timing. * craft-audio: exact Matroska ALAC seeks after a short packet (#731) Packet starts were multiples of the frame length, so a short packet before the end moved every later packet and seeks after it returned the wrong frames. When a packet's timestamp is further from that multiple than its rounding allows, the starts are now counted from the earlier packets' headers (read once and kept). Other codecs with a fixed packet length fall back to the timestamp, as before. --------- Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com> |
||
|
|
ca53e90b18 |
craft-audio: MP1/MP2 decoder (#448)
FilmCraft's MPEG audio Layer I/II decoder as `craft_audio::mpa` behind `mpa-decode`, kept over symphonia's: the same quality where both decode, and it also decodes the MP2 files ffmpeg writes by default (mono at 384 kbit/s), which symphonia rejects. `TrackDecoder` and the native read path take MP1/MP2 in MP4, Matroska and raw streams; symphonia keeps MP3 only, so its `mp1` and `mp2` features are dropped. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com> |
||
|
|
b1c64766e6 |
craft-audio: AC-3 and E-AC-3 decoder (9/10) (#447)
* craft-audio: AC-3 and E-AC-3 decoder FilmCraft's AC-3 decoder as `craft_audio::ac3` behind `ac3-decode`: every AC-3 channel mode and frame size, and E-AC-3 independent substream 0 with spectral extension, in WAV order without downmix. `TrackDecoder` and the native read path take AC-3 and E-AC-3 in MP4, Matroska and raw streams. Not bit-exact: zero-bit mantissas are filled with decoder-specific dither (A/52 §7.3.4), so tests compare at 65-94 dB SNR on tones and 27-48 dB on white noise. 7.1 E-AC-3 decodes as its 5.1 core. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * craft-audio: review fixes, round 1 (#447) An AC-3 / E-AC-3 packet is as long as all the primary syncframes in it. Before the first decode, TrackDecoder::frames_per_packet gave 1536 for any AC-3 track, and conceal used the first syncframe's header only. An E-AC-3 stream of one-, two- or three-block syncframes then got the wrong MP4 length and seek positions and the wrong Matroska positions. A packet that groups several such syncframes concealed at a fraction of its length. Seeks in AC-3 and E-AC-3 now have to match the whole-file decode bit for bit, as every other codec's do. The dither is seeded from each syncframe, so the old 25 dB bound was looser than the decoder needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * craft-audio: review fixes, round 2 (#447) - AC-3 / E-AC-3 packets that decode to different lengths (in the packets read ahead, or any decoded since) have no frames_per_packet, so a Matroska seek finds its packet by timestamp rather than by index times the first packet's length. - E-AC-3 syncframes of fewer than six blocks may reuse exponents from any syncframe before them: once one is seen, a seek decodes from the start of the stream instead of 1536 frames before the target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * craft-audio: review fixes, round 3 (#447) - Matroska E-AC-3: the first seek reads every packet's syncframe headers (only the headers, header stripping undone) and places packets by the frames their primary syncframes hold. Packets that started to differ in length after the eight read ahead were placed as the first packet's length times their index. - A packet whose later syncframe fails no longer sets the length lost packets are concealed at, nor marks the packets' lengths as differing: it decoded only in part. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Merge main; E-AC-3 in Matroska: read packet headers only up to the seek target, and count a packet of dependent syncframes as zero frames --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com> |
||
|
|
97c83dc9e8 |
craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time (#743)
* craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time Bands reads the headers of a sequential Huffman-coded (SOF0, SOF1) 8-bit file whose first scan holds every component, and a Reader decodes it one line of MCUs at a time: Huffman decoding, dequantization, inverse transform, upsampling and colour conversion, holding at most three MCU lines of samples. Nothing the size of the image is allocated, so a frame up to the format's 65,535 px sides can be decoded straight into the caller's storage. The samples equal decode()'s for the same rows: both go through the same block transform (idct_block, factored out of reconstruct), the same upsampling maps and the same colour conversion (to_color, to_u8). A component subsampled down reads the MCU line above and below its band, as the whole-image upsampling does. A band that begins a restart interval is a seek point once find_seek_points has found every restart marker, in sequence; readers on other threads can decode from there. Damaged data leaves the rest of its restart interval at zero and decoding resumes at the next marker, so the result is the same however the bands were read. Bits::restart now also clears the exhausted flag: decode() never restarts after running out of data, the band decoder does. parse_frame is split into parse_header (validation and geometry) and the whole-image allocation; parse_scan takes the component identifiers. decode() is unchanged. Tests: bands against decode() sample for sample over jpeg-encoder streams (gray, YCbCr, CMYK, YCCK; 4:4:4, 4:2:2, 4:2:0, 4:4:0; four restart intervals) and a test-only encoder's interleaved layouts with factors of 3 and 4, in order and by seeking; zune-jpeg within the existing tolerance; damage with and without restarts; limits; unsupported files; the libjpeg test images; the mutation run with a seek-equals-in-order check; and a `bands` fuzz target. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: faster inverse transform, rounding, Huffman codes and band rows Each change keeps every sample as it was: decode() and the bands give bit-identical output before and after on the test images, 12-bit and arithmetic-coded files, 48 generated streams and a 48,000 × 34,200 file decoded in bands. - idct_block works on rows of eight: each sum keeps its order (over u, then over v), eight values of x at a time, and rows of zero coefficients are skipped (a zero term changes only the sign of a zero, which the level shift removes). The transposed basis is built once. - to_u8 rounds half away from zero as truncation plus one from .5 up, exact on 0..=255, instead of the `round` library call targets without SSE4.1 make. - Huffman::decode resolves codes of up to 9 bits from a table of the next 9 bits, built by running the bit-by-bit procedure on each, so both give the same symbol. - The band rows are colour converted in loops written out for gray, YCbCr and RGB; YCbCr to RGB is one helper shared with the whole-image decode. On the 48,000 × 34,200 4:4:4 file with a restart interval per line of MCUs (Ryzen 7 7800X3D, idle), one thread decodes a band of 8 rows in about 3.0 ms (5.8 ms before), and 16 threads decode the whole image in 2.05 s (3.8 s before). Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: bands report damage, never fail on it, and see data left before a restart Two cases a mutation run through PhotoCraft's import turned up. - Data that fails in the first MCU made a band read fail, as decode() refuses a stream with no image data. The baseline decoder PhotoCraft uses for other files opens such files (grey, with restarts the rest of the image), so a band read now marks it damaged and goes on: with restart intervals everything after the first decodes. Refusing a file with nothing decodable is the caller's, from the size of its scan data. - An interval that decodes without an invalid code but leaves whole bytes before its restart marker was decoded from cut, shifted or added data. It now counts as damage (libjpeg warns "extraneous bytes before marker"). The check runs after an interval's last MCU, so readers that seek and readers that read in order agree. Over 1,074 sequential 8-bit JPEGs on one machine the damage flag agrees with djpeg's warnings on every file, reading from seek points equals reading in order on all 151 with restart intervals, and the samples are within mean 0.57, max 8 of zune-jpeg's. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: seek points follow the restart markers in the data, as reading in order does find_seek_points gave up on a file whose markers were not all there in sequence, so a damaged or cut-short file with a restart interval at every line of MCUs decoded in order on one core (about 13 s for an undamaged 48,000 × 34,200 file on an idle 8-core machine, against 2 s on 16 threads). With these seek points the same file with one marker renumbered, cut at 60%, or its first interval overwritten decodes in 2.8 to 4.4 s on 16 threads. A reader that reads in order pairs the k-th restart marker in the data with interval k, whatever its number, and resumes after each, so seek points built the same way give the same samples for any data: missing, misnumbered or added markers only change which bands are seek points. Tested: misnumbered, missing (file cut), added and removed markers read in order and by seeking; the mutation run's seek-equals-in-order check (more of its cases now seek); 600,000 runs of the bands fuzz target, which asserts the same; 169 local files with restarts. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: Bands::halo, the MCU lines a reader decodes around its bands A reader that seeks decodes the MCU line above and below its bands when a component is subsampled down. A caller splitting the work across threads needs to know it: reading one band from each seek point of a 4:2:0 file decodes three lines per band (4.97 s for a 48,000 × 34,200 file on 16 threads, against 3.09 s reading four bands from each), while for 4:4:4 the run length makes no difference. Tested against the sampling factors of every interleaved layout. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: band decoder review fixes - Data left over after the last MCU, before EOI, is damage, as it already was before a restart marker. - A scan that names a component twice is Malformed (T.81 B.2.3), not Unsupported. - The upsampling maps are sized from the header and checked against max_alloc before they are made. - The bands fuzz target and the mutation test fail on a read or seek error once Bands::new has accepted a stream; damage never surfaces as an error. - Known-sample CMYK and YCCK fixtures check the ink polarity directly; the bands-equal-decode tests cannot, since both share the colour conversion. - Docs: a single-component frame's bands are 8 rows; Reader::damaged and the scan's size cannot tell a failure at the first MCU from later damage; with a halo a seek decodes from the previous seek point; each fuzz target writes to its own corpus; doc comments on the remaining new functions. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> * craft-jpeg: test YCCK chroma and trailing-data samples - The ink fixture gains a YCCK case with Cb and Cr away from neutral, with the T.871 values worked out by hand: a conversion that drops or swaps the chroma now fails it. - Bytes added before EOI leave the samples as the clean stream's; the test now checks that, not only that reading in order and seeking agree. Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> --------- Co-authored-by: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is> Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com> |
||
|
|
861c5e172d |
craft-audio: Opus decoder (7/10) (#445)
* craft-audio: Opus decoder FilmCraft's `filmcraft-opus` decoder as `craft_audio::opus` behind `opus-decode`: SILK, hybrid and CELT, every frame size, PLC and FEC, multistream (`Decoder`) and single stream (`StreamDecoder`), 8-48 kHz output. - `TrackDecoder` decodes Opus; mono and stereo without an `OpusHead` (RTP, MPEG-TS) get a default one. Lost packets use Opus's own concealment. - The native read path takes Ogg Opus (pre-skip and end trimming per RFC 7845), and Opus in Matroska/WebM and MP4, where packet starts after a seek come from the packets' own lengths. The RFC 8251 conformance test runs all 36 vectors. `scripts/fetch-opus-vectors.sh` downloads them (SHA-256 checked) and the CI test job caches them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * craft-audio: review fixes, round 1 (#445) - CELT concealment keeps its pitch in its own field. It reused postfilter_period_old, which the next good frame's post-filter cross-fades from. - Matroska Opus seeks: a failed packet read no longer caches a wrong start; a one-byte packet is read as one byte. - A malformed OpusHead in Ogg is reported as malformed instead of falling back. - The libopus oracle tests skip through the testkit, so CRAFT_MEDIA_REQUIRE_ORACLES fails them. - Decoder::reset documents that it re-arms pre-skip trimming. - fetch-opus-vectors.sh retries its size probe. - CI: the vectors are fetched after ffmpeg is installed, and apt's timeout runs as root so a stalled apt-get is killed and frees the dpkg lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * craft-audio: review fixes, round 2 (#445) - CELT: the noise-based concealment sets skip_plc, as libopus does, so pitch-based concealment waits for two good frames in a row instead of extending history that holds noise. - SILK: decode_only_middle is per call, as in libopus. A lost or mono frame no longer reuses the mid-only flag from an earlier frame. - A damaged Opus packet is concealed at its own length (from its TOC and framing) rather than the previous packet's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com> |
||
|
Gerrit TimmerandDylan M. Taylor
|
5b58973bc6 |
craft-media: index a Matroska file progressively (#430)
* craft-media: index a Matroska file progressively (OpenOptions::index_limit, MkvFile::extend_index) * craft-media: Matroska element headers never read past their parent --------- Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com> |
||
|
Aleksander HelgakerandClaude Opus 5.5
|
ca99f91a18 |
craft-audio: AAC encoder (6/10) (#444)
* craft-audio: AAC decoder
FilmCraft's `filmcraft-aac` decoder as `craft_audio::aac` behind
`aac-decode`, kept over symphonia's: the same quality (about 135 dB
SNR against ffmpeg), and it decodes 5.1, which symphonia refuses.
HE-AAC decodes as its AAC-LC core, upsampled 2x to the stream's rate.
- `TrackDecoder` decodes AAC, including ADTS-framed packets without
extradata (MPEG-TS, raw `.aac`).
- The native read path takes AAC in MP4 and Matroska and raw ADTS
streams (frame sync over the file, with resync after damage).
- AAC errors convert into `AudioError`.
The AAC module's shared parts (Huffman, MDCT, TNS, ICS) arrive whole
as ported; their encoder half is unused until the encoder PR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 1 (#443)
Raw ADTS and TrackDecoder:
- Channel configuration 0 (a PCE in the stream, as ffmpeg's -aac_pce and
AudioToolbox's 7.x write) decodes; it was refused.
- Frames with several raw data blocks are refused; only their first block
was decoded, silently losing the rest.
- A frame that changes the sample rate, profile or channel configuration is
concealed; it played at the wrong speed.
- A source that fails during the frame scan is AudioError::Io; it looked
like the end of the file.
- A stream after an ID3 tag longer than the file head is found.
- conceal counts all the ADTS frames of a packet, and a packet of more than
1024 frames is refused before it is decoded into memory.
AAC decoder:
- Indexed channel configurations place elements by arrival order, not tag,
as ffmpeg does; tagged differently, channels were swapped.
- A PCE in an access unit that fails no longer changes the layout, which
left every later packet failing.
- Non-finite samples are zeroed before the output and the overlap.
MP4: probe and decode agree: the samples' durations end the track when no
single edit does, and an edit cannot outlast the media.
The routing oracle fails on a fixture ffmpeg cannot write when oracles are
required. The README's Limits names HE-AAC v2 as mono, multi-block ADTS,
iTunSMPB, Matroska's tick precision and AAC in CAF.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 2 (#443)
Corrects round 1:
- A multi-block ADTS frame after the stream has started is concealed at its
whole length; round 1's filter skipped it, shifting the audio after it.
- Sample durations only trim an MP4's last packet: wrong ones (some count a
HE-AAC packet's core frames) were cutting audio. Durations of 0 with an edit
use the edit.
Raw ADTS packets now keep their headers, so TrackDecoder alone handles format
changes, multi-block frames and concealment, for apps as for files, and nested
ADTS payloads are not split again. Headers come from the frame scan's reads,
not one read per frame. The stream's format is the one most of its first frames
share, so one damaged header does not silence the file. The ID3 walk and a
packet's ADTS split are bounded.
AAC:
- HE-AAC is delayed by the SBR filterbanks' 481 core samples; it played 19.5 ms
early against ffmpeg in every container.
- A failed access unit leaves the decoder's state as it was, so the next packet
decodes as if it had not been there.
- reset restarts the noise generator, so a seek into PNS repeats.
- A signalled rate that is not twice the core's plays the core; it reported the
signalled rate with the core's samples.
The decoder oracle's tolerance tightens from 1e-3 to 1e-5. Tests cover HE-AAC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 3 (#443)
Corrects round 2:
- ADTS headers are compared with the stream's format only when the first header
gave it. With extradata the format comes from there, so ADTS in MP4 whose
headers say something else decodes again; every packet had been concealed.
- A stream's only program config element is found in its first frames even when
that frame's header is damaged.
- The test of wrong sample durations patched the second run's count, not the
delta, so "durations of one" changed nothing. It now patches the delta, and
checks a file was changed and that seeks land on the same audio.
MP4 seeks place a packet by its index when every packet decodes to the same
length, as decoding does; they used the durations round 2 had stopped trusting
for the length, and returned the wrong audio. A last delta of 0 is overridden too.
Docs: what TrackInfo says for raw ADTS, that opening one reads the whole file, and
which codecs symphonia reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 4 (#443)
- TrackDecoder takes an ADTS stream's format from the one most of the lookahead
shares, as the file path did, so one damaged first header does not fail every
packet of an MPEG-TS stream. open_adts now just asks it.
- MP4 samples of several ADTS frames keep the sample table's timing; a packet
count of one frame each truncated them silently.
- Sample durations past what the packets decode to no longer over-report the
length.
- split_adts_upto is not behind aac-decode, as split_adts is not: an aac-encode
build in the next PR did not compile.
Removes an unreachable ID_END check and open_adts's unused core rate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 5 (#443)
No change in decoding. An aac-encode build in the next PR needs the
split_adts_upto re-export behind aac-decode, and ID_END only in tests.
Docs say the lookahead's headers give an ADTS stream's configuration, that AAC in
CAF still goes through symphonia, and that ProgramConfig is not #[non_exhaustive].
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 6 (#443)
Two tests from round 1 failed on ffmpeg before 9, including CI's 8.0. Neither was
a decoding fault:
- ffmpeg 9 ends an MP4 with no edit list at its sample durations and earlier
versions present the whole last packet, so the length is checked against the
file's known duration, not ffmpeg's decode.
- The order ffmpeg gives channels with -aac_pce differs by version, so each decoded
channel must match a different one of ffmpeg's, whatever the order.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 7 (#443)
Reading:
- an element of another type or a tag a program config does not list is
refused, not written to another channel; only an LFE coded as an SCE
is still allowed, as ffmpeg allows it
- an explicit sampling rate of 0 is refused
- a packet's later ADTS frames are searched for SBR
- a multi-frame ADTS packet that fails leaves the decoder as it was, and
one whose frames change the channel count is an error
- the whole ADTS frames before a damaged end of a packet decode
- a failed read of a raw stream's first frames is an I/O error
- an MP4 sample of several ADTS frames past the lookahead keeps its length
- an Ogg file with leading bytes keeps its track order and mask comments
Writing:
- ProgramConfig::write, AudioSpecificConfig::to_bytes and
AdtsHeader::write refuse what their fields cannot hold
Tests:
- the routing test checks one alignment for the ffmpeg version under test
- a default decode past an Ogg stream symphonia has no track for
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 8 (#443)
From wanikhawar's review:
- AudioSpecificConfig::parse reads backward-compatible SBR and PS signalling
(a 0x2b7 sync extension after the config), which craft-media's Matroska
demuxer writes for legacy HE-AAC IDs. Without lookahead those streams decoded
as their core.
- An MP4 media edit after one empty edit is applied: its priming skip and its
end. The empty edit's delay is not added as silence.
- A packet lost whole is concealed as long as the last one decoded, so a stream
of multi-frame ADTS packets keeps its timing.
- ADTS-framed MP4 samples keep their sample table's timing, instead of every
sample being read at open to look for several frames in one.
- AdtsHeader::write refuses an object type, sampling frequency index or channel
configuration its fields cannot hold; it masked them into other values.
- AudioSpecificConfig::lc no longer panics for a sampling frequency index with no
rate, and the decoder refuses such a config.
- AudioSpecificConfig::to_bytes refuses channel configuration 0 without a program
config element, which would not parse back.
- Upsample2x starts each channel's history afresh when the channel count
changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 9 (#443)
From CodeRabbit's review of round 8:
- AudioSpecificConfig::parse consumes extensionFlag3 after the config when
extensionFlag is set, so the 0x2b7 sync extension after it is found.
- AudioSpecificConfig::to_bytes refuses a sampling rate of 0, which would not
parse back.
- ADTS-framed HE-AAC in MP4 whose sample durations count core frames is not cut
to the durations' half length: durations short of one frame per sample no
longer end an ADTS-framed track, and its length is unknown until decoded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 10 (#443)
ADTS-framed AAC in MP4: durations short of the frames in the samples read
ahead no longer end the track. With HE-AAC samples of two ADTS frames each,
timed as two core frames, the durations matched one output frame per sample,
so the undercount check passed and the track stopped at half its length,
which probe reported as known. The frames each read-ahead sample holds are
now counted, and durations short of them leave the length unknown until
decoded, as durations short of one frame per sample already did. Correctly
timed AAC-LC samples of several frames keep their end trim.
Reported by FredApps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: review fixes, round 11 (#443)
Corrects round 10. A short ADTS-framed MP4 track, whose last sample is in
the read-ahead, lost that sample's trim when it cut more than one frame:
the trim counted as durations falling short, so the length was left to
decoding and its padding was presented. The track's last sample is now left
out of the read-ahead comparison.
Reported by CodeRabbit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: document the ADTS-in-MP4 seek and edit limits (#443)
AAC in MP4 stored as ADTS frames, with sample durations that count fewer
frames than the samples hold, decodes in full from the start, but seeks
place samples by those durations and land later than asked, and an edit
list measured in them can end the track early. The README's Limits section
now says so, and the seek example points to it.
Reported by FredApps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-audio: say why the ADTS-in-MP4 seek limit is kept (#443)
The Limits entry from
|
||
|
|
139a39e94c |
cms: write the profile ID into encoded profiles (#749)
encode() stamps the header profile ID: the MD5 of the profile with the flags, rendering-intent and ID fields zeroed (ICC.1:2022 7.2.18). profile_id(bytes) computes it for any profile. Adds md-5 (RustCrypto, MIT OR Apache-2.0, no default features); digest was already in the lock. Closes #748. Co-authored-by: KenM76 <KenM76> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a9608afe43 |
cms: read named-colour (ncl2) profiles and convert their colours (#747)
* cms: read named-colour (ncl2) profiles and convert their colours A named-colour profile's ncl2 tag (ICC.1:2022 §10.17) was never read, so a spot-colour library could not be used. Profile::named_colors() returns its entries: the name joined from prefix, root and suffix, the PCS value and any device coordinates. NamedColors::to_device(name, dst) converts one into a destination profile, relative colorimetric, through the Lab identity profile. Lab in ncl2 uses the legacy 16-bit encoding (0xFF00 = L* 100) in every profile version; reading it as v4 puts L* about 0.4 low. Counts are checked against the tag size before allocating, and more than 15 device coordinates is refused. Adapted from icccer's named_color module (MIT, same author), rewritten onto craft-cms types. Refs #746 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * cms: cap the ncl2 tag-table walk; doc fixes Cap the tag count at 1024, as Profile::parse does, so the table offsets cannot overflow on a 32-bit target. Correct the named_colors doc and add one for is_empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * cms: README notes named-colour tables are readable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: KenM76 <KenM76> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Tex JerniganandClaude Opus 5.5
|
7480fc2ac4 |
craft-media: write and read MP4/MOV chapter lists (#526)
* craft-media: write and read MP4/MOV chapter lists
WriterOptions::chapters (Chapter { start, title }, starts in the movie
timescale) makes Mp4Writer write a chapter list in the two forms players
read:
- a QuickTime chapter track (a disabled text track referenced by
tref/chap from the first video or audio track, one UTF-8 sample with an
encd atom per chapter) for QuickTime / AVFoundation, and
- a Nero udta/chpl list (version 1, 100 ns units) for VLC and
ffprobe -show_chapters.
Chapters are normalized (sorted, one per start, inside the movie, at most
255 chapters with 255-byte titles cut at a character boundary, control
characters replaced) and a late first chapter is placed with an empty
edit. FragmentedWriter writes chpl only. The demuxer reads both forms
back: Mp4File::chpl_chapters and Mp4File::read_chapters(src).
Ported from FilmCraft's own isobmff crate (storytold/filmcraft#2821, for
storytold/filmcraft#2747), now that FilmCraft uses craft-media.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: read chapters only from enabled tracks, inside the edit, with bounded titles
read_chapters took a tref/chap from any track, though QuickTime requires the referencing track to be
enabled. It ignored the chapter track's edit list, so a chapter before the edit start became a second
chapter at zero and samples past its end were returned. It kept titles of any length.
Now a disabled host is skipped; chapters outside the first non-empty edit are dropped, and one spanning
its start begins there; each sample read is capped at 4098 bytes and titles are cut to
MAX_CHAPTER_TITLE_BYTES, which bounds what is kept. Later edits are still ignored.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: a chapter title longer than the sample read keeps its start
A sample declaring more title than the 4096 bytes read parsed to nothing, so the
chapter lost its whole title rather than being cut to MAX_CHAPTER_TITLE_BYTES.
A sample cut by the read now decodes what it holds; one read whole but declaring
more than it has is still no title.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|