Commit Graph
1375 Commits
Author SHA1 Message Date
Dylan M. Taylor d4e3017c34 Add craft-mime: Internet message and MIME reader and writer (#814)
MailCraft and WordCraft each parse MIME: MailCraft for mail (crates/mime,
crates/charset), WordCraft for MHT web archives (formats/src/mht.rs). This
adds one shared crate for the format, taken from MailCraft's code where it
is format code, with WordCraft's multipart/related lookups and its fix for
comments in Content-Type.

Reader: parse() gives a tree of parts (header fields, content type and
parameters, decoded body, charset-decoded text). It never fails or panics,
is linear in the input, and is bounded by Limits (input size, header count
and sizes, depth, part count, decoded size); Message::truncated reports a
limit that left something out.

Writer: Entity::text / file / multipart and write_message, with folded
headers, RFC 2047 and RFC 2231 encoding, quoted-printable and base64 with
line limits, and boundaries checked against the content. Every header
value loses CR, LF and NUL, and a name that is no field name is not
written.

Also: address lists, mbox splitting and writing on raw message bytes, and
Date parsing and formatting on a plain struct (chrono is a private
dependency; no date type is in the public API).

No app is switched here. The README has the migration tables and says what
stays in the apps.
2026-11-11 12:48:08 -05:00
Dylan M. Taylor 27f2204603 craft-media: DeckCraft's two Ogg fixes (#810)
Packet byte ranges are checked against the source before a buffer is
sized for them (deckcraft 51d19baa), and packets after the last known
granule count towards the Opus duration (deckcraft 9d36b593). Each test
failed before its fix.
2026-11-11 12:48:03 -05:00
Dylan M. Taylor ac5f998274 ci: run Harden-Runner on every job (#811)
Adds step-security/harden-runner (audit, pinned to v2.22.4 like
craft-common-actions) as the first step of each job that did not have it,
so every workflow in the org's public repos reports its egress to
StepSecurity. Audit, not block: block-mode allow lists come from each
job's observed traffic.
2026-11-11 12:10:08 -05:00
Dylan M. Taylor 56d088bf04 craft-audio: flac::Encoder, the FLAC packet encoder, is public (#809)
* craft-audio: FilmCraft's FLAC encoder and MP1/MP2 decoder speed-ups

FLAC: per-order predictor kernels, merged Rice partition sums, the LPC
window kept per block length, and batches consumed by offset
(filmcraft 5eaff794, d9dad4a5). The output is unchanged: 64 golden cases
hash the same before and after.

MP1/MP2: the faster synthesis filterbank (filmcraft 51c755e5), with its
bit-exact test.

* craft-audio: FilmCraft's AAC and Opus decoder speed-ups and refactors

AAC: table-driven Huffman decoding, per-stage FFT twiddles and fused
stages, reused per-channel work buffers, and the encoder's memoised
per-band work (filmcraft f9627280, d587ceed, 6121635c, dc124ab7,
7a532eca).

Opus: radix-2/3/4 FFT kernels in the CELT IMDCT, the PVQ table built
once, one de-emphasis loop for both channels, the SILK resampler
without per-sample divisions and with eight partial sums, concealment
and FEC capped at 120 ms (filmcraft bf7ac420, 12e32026, c4dc64d1,
eb352ec8, 87b8e22d, 014b5e56, 78d77b72).

The per-frame mid-only flag this crate fixed in the SILK decoder is
kept in the refactored decode.

* craft-media: Ogg CRC-32 by slice-by-16

FilmCraft's 6763f9bf: sixteen independent lookups per 16 input bytes, so
opening a file is several times faster. Checked against a bit-at-a-time
reference on every short length and alignment, and across every split.

* craft-audio: SoundCraft's CAF cookie-size and FLAC vendor-length checks

A CAF magic cookie size above the signed range is refused instead of
passed to symphonia (soundcraft 3bd3965), and a Vorbis comment block
whose vendor string claims more than the block holds is refused
(soundcraft ddfe991). Each with its regression test.

* craft-audio: AC-3 reads bits through the shared BitReader

FilmCraft's 1c9c2e82: the AC-3 decoder's private bit-at-a-time reader is
replaced by craft-media's BitReader, with its two tests: the decoded
output of 2,000 synthetic and 12 real syncframes (and their damaged
variants) hashes the same as before, and the reader agrees with the old
one on random reads, skips and overruns. The three AC-3 features now
turn on craft-media's bitstream module.

* craft-audio: the quantiser rounds without a libm call

SoundCraft's d293336, the audio-io part: round-half-away by integer
arithmetic in the bounce's inner loop, with its test against f64::round
over the quantiser's range.

* craft-audio: flac::Encoder, the FLAC packet encoder, is public

FilmCraft muxes FLAC into its own outputs from integer samples, so it
needs the encoder below FileEncoder: planar samples in, frames out, and
the STREAMINFO to patch in at the end. The module is behind flac-encode,
its errors convert to AudioError::Encode, and pending_samples says how
long the final frame will be.
2026-11-11 11:48:37 -05:00
Dylan M. Taylor 44b4a23137 craft-audio, craft-media: the changes FilmCraft and SoundCraft made to their copies since extraction (#807)
* craft-audio: FilmCraft's FLAC encoder and MP1/MP2 decoder speed-ups

FLAC: per-order predictor kernels, merged Rice partition sums, the LPC
window kept per block length, and batches consumed by offset
(filmcraft 5eaff794, d9dad4a5). The output is unchanged: 64 golden cases
hash the same before and after.

MP1/MP2: the faster synthesis filterbank (filmcraft 51c755e5), with its
bit-exact test.

* craft-audio: FilmCraft's AAC and Opus decoder speed-ups and refactors

AAC: table-driven Huffman decoding, per-stage FFT twiddles and fused
stages, reused per-channel work buffers, and the encoder's memoised
per-band work (filmcraft f9627280, d587ceed, 6121635c, dc124ab7,
7a532eca).

Opus: radix-2/3/4 FFT kernels in the CELT IMDCT, the PVQ table built
once, one de-emphasis loop for both channels, the SILK resampler
without per-sample divisions and with eight partial sums, concealment
and FEC capped at 120 ms (filmcraft bf7ac420, 12e32026, c4dc64d1,
eb352ec8, 87b8e22d, 014b5e56, 78d77b72).

The per-frame mid-only flag this crate fixed in the SILK decoder is
kept in the refactored decode.

* craft-media: Ogg CRC-32 by slice-by-16

FilmCraft's 6763f9bf: sixteen independent lookups per 16 input bytes, so
opening a file is several times faster. Checked against a bit-at-a-time
reference on every short length and alignment, and across every split.

* craft-audio: SoundCraft's CAF cookie-size and FLAC vendor-length checks

A CAF magic cookie size above the signed range is refused instead of
passed to symphonia (soundcraft 3bd3965), and a Vorbis comment block
whose vendor string claims more than the block holds is refused
(soundcraft ddfe991). Each with its regression test.

* craft-audio: AC-3 reads bits through the shared BitReader

FilmCraft's 1c9c2e82: the AC-3 decoder's private bit-at-a-time reader is
replaced by craft-media's BitReader, with its two tests: the decoded
output of 2,000 synthetic and 12 real syncframes (and their damaged
variants) hashes the same as before, and the reader agrees with the old
one on random reads, skips and overruns. The three AC-3 features now
turn on craft-media's bitstream module.

* craft-audio: the quantiser rounds without a libm call

SoundCraft's d293336, the audio-io part: round-half-away by integer
arithmetic in the bounce's inner loop, with its test against f64::round
over the quantiser's range.
2026-11-11 11:32:33 -05:00
.CooCoo 6e03f65bf4 craft-update: a failed check for a running attach is reported on macOS (#803)
detach_unmounted leaves a disk image's attachments alone when the check
for a running `hdiutil attach` of the image gets no answer, and it
returned Ok in that case. A failed install then ended with no note that
the image may still be attached. It now returns the check's reason as
an error, so the install's error says "the disk image may still be
attached" with that reason, and craft-update logs it at warn level.
2026-11-11 11:22:26 -05:00
Dylan M. Taylorandwhatever-industries 2765c84bda craft-audio: AC-3 and E-AC-3 packet encoders (#806)
* feat(audio): add AC-3 and E-AC-3 packet encoders

* craft-audio: AC-3 encoder errors convert to AudioError::Encode

---------

Co-authored-by: whatever-industries <77seventy77@users.noreply.github.com>
2026-11-11 10:36:06 -05:00
el-j 9f10d434d9 craft-video: clean-room video codecs, one feature per codec (#702–#708)
The eight-part craft-video series landed as one commit, because each part's branch contains the parts before it.

- #702 dnx: DNxHD/DNxHR decoder and encoder
- #703 vp8: VP8 decoder
- #704 prores: ProRes decoder and encoder
- #705 h264: H.264 encoder
- #706 hevc: HEVC encoder
- #707 vp9: VP9 encoder
- #708 av1: AV1 encoder
2026-11-11 08:25:23 -05:00
Aleksander HelgakerandDylan M. Taylor 7e40e12989 craft-audio: ALAC decoder and encoder (#731)
* craft-audio: ALAC decoder and encoder, ported from Apple's reference

Add `craft_audio::alac`, a Rust port of Apple's ALAC reference
implementation (macosforge/alac@c38887c, Apache-2.0):

- `alac::Decoder`: packets to planar i32, bit depths 1-32, 1-8 channels,
  cookies in MP4/Matroska (24-byte config), CAF and old QuickTime form.
  Hostile input returns errors; arithmetic wraps where C would overflow.
- `alac::Encoder`: 16/20/24/32 bits, 1-8 channels, fast mode, any frame
  length; same packets as the reference for the same input.
- `SpecificConfig` (cookie parse/write), `wav_order`/`alac_order`,
  `channel_mask`.
- Features `alac-decode` (in `decode-native`) and `alac-encode` (in
  `encode`).

`TrackDecoder` decodes ALAC natively, so ALAC in MP4 and Matroska no
longer goes through symphonia (CAF still does).

Apple's reference writes the cookie's channel layout tag in host byte
order; this writes it big-endian as its cookie description specifies and
reads either.

Tests: unit tests for each part, an MP4 decode-and-seek test without
symphonia, and ffmpeg oracles both ways (bit-exact, 1-8 channels).
NOTICE and each file's header record the Apache-2.0 origin.

* craft-audio: ALAC review fixes (#731)

- Decode the full-width residual escape 0xffffffff as i32::MIN; the
  reference's `nd + 1` wraps it to 0.
- Require a whole element tag after the last channel (the end tag, after
  any fill or data stream elements), so a packet that lost its last byte
  is an error instead of decoding without damage. Other elements there
  end the packet, as in the reference.
- Refuse an ALAC track with no sample rate in its cookie or container.
- Conceal a damaged packet with the length its header gives
  (`alac::Decoder::packet_frames`), falling back to the last packet's.
- Matroska: place ALAC packet starts at multiples of the frame length
  instead of the rounded block timestamps, so seeks land on the exact
  frame. MP4 keeps its exact sample-table timing.

* craft-audio: exact Matroska ALAC seeks after a short packet (#731)

Packet starts were multiples of the frame length, so a short packet
before the end moved every later packet and seeks after it returned
the wrong frames. When a packet's timestamp is further from that
multiple than its rounding allows, the starts are now counted from the
earlier packets' headers (read once and kept). Other codecs with a
fixed packet length fall back to the timestamp, as before.

---------

Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 08:18:09 -05:00
ca53e90b18 craft-audio: MP1/MP2 decoder (#448)
FilmCraft's MPEG audio Layer I/II decoder as `craft_audio::mpa` behind
`mpa-decode`, kept over symphonia's: the same quality where both
decode, and it also decodes the MP2 files ffmpeg writes by default
(mono at 384 kbit/s), which symphonia rejects. `TrackDecoder` and the
native read path take MP1/MP2 in MP4, Matroska and raw streams;
symphonia keeps MP3 only, so its `mp1` and `mp2` features are dropped.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 08:02:37 -05:00
b1c64766e6 craft-audio: AC-3 and E-AC-3 decoder (9/10) (#447)
* craft-audio: AC-3 and E-AC-3 decoder

FilmCraft's AC-3 decoder as `craft_audio::ac3` behind `ac3-decode`:
every AC-3 channel mode and frame size, and E-AC-3 independent
substream 0 with spectral extension, in WAV order without downmix.
`TrackDecoder` and the native read path take AC-3 and E-AC-3 in MP4,
Matroska and raw streams.

Not bit-exact: zero-bit mantissas are filled with decoder-specific
dither (A/52 §7.3.4), so tests compare at 65-94 dB SNR on tones and
27-48 dB on white noise. 7.1 E-AC-3 decodes as its 5.1 core.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#447)

An AC-3 / E-AC-3 packet is as long as all the primary syncframes in it.
Before the first decode, TrackDecoder::frames_per_packet gave 1536 for any
AC-3 track, and conceal used the first syncframe's header only. An E-AC-3
stream of one-, two- or three-block syncframes then got the wrong MP4 length
and seek positions and the wrong Matroska positions. A packet that groups
several such syncframes concealed at a fraction of its length.

Seeks in AC-3 and E-AC-3 now have to match the whole-file decode bit for bit,
as every other codec's do. The dither is seeded from each syncframe, so the
old 25 dB bound was looser than the decoder needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#447)

- AC-3 / E-AC-3 packets that decode to different lengths (in the packets
  read ahead, or any decoded since) have no frames_per_packet, so a
  Matroska seek finds its packet by timestamp rather than by index times
  the first packet's length.
- E-AC-3 syncframes of fewer than six blocks may reuse exponents from any
  syncframe before them: once one is seen, a seek decodes from the start
  of the stream instead of 1536 frames before the target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 3 (#447)

- Matroska E-AC-3: the first seek reads every packet's syncframe headers (only the headers,
  header stripping undone) and places packets by the frames their primary syncframes hold.
  Packets that started to differ in length after the eight read ahead were placed as the first
  packet's length times their index.
- A packet whose later syncframe fails no longer sets the length lost packets are concealed at,
  nor marks the packets' lengths as differing: it decoded only in part.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Merge main; E-AC-3 in Matroska: read packet headers only up to the seek target, and count a packet of dependent syncframes as zero frames

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 07:52:06 -05:00
97c83dc9e8 craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time (#743)
* craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time

Bands reads the headers of a sequential Huffman-coded (SOF0, SOF1) 8-bit file whose first scan
holds every component, and a Reader decodes it one line of MCUs at a time: Huffman decoding,
dequantization, inverse transform, upsampling and colour conversion, holding at most three MCU
lines of samples. Nothing the size of the image is allocated, so a frame up to the format's
65,535 px sides can be decoded straight into the caller's storage.

The samples equal decode()'s for the same rows: both go through the same block transform
(idct_block, factored out of reconstruct), the same upsampling maps and the same colour
conversion (to_color, to_u8). A component subsampled down reads the MCU line above and below
its band, as the whole-image upsampling does.

A band that begins a restart interval is a seek point once find_seek_points has found every
restart marker, in sequence; readers on other threads can decode from there. Damaged data
leaves the rest of its restart interval at zero and decoding resumes at the next marker, so the
result is the same however the bands were read. Bits::restart now also clears the exhausted
flag: decode() never restarts after running out of data, the band decoder does.

parse_frame is split into parse_header (validation and geometry) and the whole-image
allocation; parse_scan takes the component identifiers. decode() is unchanged.

Tests: bands against decode() sample for sample over jpeg-encoder streams (gray, YCbCr, CMYK,
YCCK; 4:4:4, 4:2:2, 4:2:0, 4:4:0; four restart intervals) and a test-only encoder's interleaved
layouts with factors of 3 and 4, in order and by seeking; zune-jpeg within the existing
tolerance; damage with and without restarts; limits; unsupported files; the libjpeg test
images; the mutation run with a seek-equals-in-order check; and a `bands` fuzz target.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: faster inverse transform, rounding, Huffman codes and band rows

Each change keeps every sample as it was: decode() and the bands give bit-identical output
before and after on the test images, 12-bit and arithmetic-coded files, 48 generated streams
and a 48,000 × 34,200 file decoded in bands.

- idct_block works on rows of eight: each sum keeps its order (over u, then over v), eight
  values of x at a time, and rows of zero coefficients are skipped (a zero term changes only the
  sign of a zero, which the level shift removes). The transposed basis is built once.
- to_u8 rounds half away from zero as truncation plus one from .5 up, exact on 0..=255, instead
  of the `round` library call targets without SSE4.1 make.
- Huffman::decode resolves codes of up to 9 bits from a table of the next 9 bits, built by
  running the bit-by-bit procedure on each, so both give the same symbol.
- The band rows are colour converted in loops written out for gray, YCbCr and RGB; YCbCr to
  RGB is one helper shared with the whole-image decode.

On the 48,000 × 34,200 4:4:4 file with a restart interval per line of MCUs (Ryzen 7 7800X3D,
idle), one thread decodes a band of 8 rows in about 3.0 ms (5.8 ms before), and 16 threads
decode the whole image in 2.05 s (3.8 s before).

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: bands report damage, never fail on it, and see data left before a restart

Two cases a mutation run through PhotoCraft's import turned up.

- Data that fails in the first MCU made a band read fail, as decode() refuses a stream with no
  image data. The baseline decoder PhotoCraft uses for other files opens such files (grey, with
  restarts the rest of the image), so a band read now marks it damaged and goes on: with
  restart intervals everything after the first decodes. Refusing a file with nothing
  decodable is the caller's, from the size of its scan data.
- An interval that decodes without an invalid code but leaves whole bytes before its restart
  marker was decoded from cut, shifted or added data. It now counts as damage (libjpeg warns
  "extraneous bytes before marker"). The check runs after an interval's last MCU, so readers
  that seek and readers that read in order agree.

Over 1,074 sequential 8-bit JPEGs on one machine the damage flag agrees with djpeg's warnings on
every file, reading from seek points equals reading in order on all 151 with restart intervals,
and the samples are within mean 0.57, max 8 of zune-jpeg's.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: seek points follow the restart markers in the data, as reading in order does

find_seek_points gave up on a file whose markers were not all there in sequence, so a damaged
or cut-short file with a restart interval at every line of MCUs decoded in order on one core
(about 13 s for an undamaged 48,000 × 34,200 file on an idle 8-core machine, against 2 s on
16 threads). With these seek points the same file with one marker renumbered, cut at 60%, or
its first interval overwritten decodes in 2.8 to 4.4 s on 16 threads. A reader that reads in
order pairs the k-th restart marker in the data with interval k, whatever its number, and
resumes after each, so seek points built the same way give the same samples for any data:
missing, misnumbered or added markers only change which bands are seek points.

Tested: misnumbered, missing (file cut), added and removed markers read in order and by
seeking; the mutation run's seek-equals-in-order check (more of its cases now seek); 600,000
runs of the bands fuzz target, which asserts the same; 169 local files with restarts.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: Bands::halo, the MCU lines a reader decodes around its bands

A reader that seeks decodes the MCU line above and below its bands when a component is
subsampled down. A caller splitting the work across threads needs to know it: reading one band
from each seek point of a 4:2:0 file decodes three lines per band (4.97 s for a 48,000 × 34,200
file on 16 threads, against 3.09 s reading four bands from each), while for 4:4:4 the run length
makes no difference. Tested against the sampling factors of every interleaved layout.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: band decoder review fixes

- Data left over after the last MCU, before EOI, is damage, as it already
  was before a restart marker.
- A scan that names a component twice is Malformed (T.81 B.2.3), not
  Unsupported.
- The upsampling maps are sized from the header and checked against
  max_alloc before they are made.
- The bands fuzz target and the mutation test fail on a read or seek error
  once Bands::new has accepted a stream; damage never surfaces as an error.
- Known-sample CMYK and YCCK fixtures check the ink polarity directly; the
  bands-equal-decode tests cannot, since both share the colour conversion.
- Docs: a single-component frame's bands are 8 rows; Reader::damaged and
  the scan's size cannot tell a failure at the first MCU from later damage;
  with a halo a seek decodes from the previous seek point; each fuzz target
  writes to its own corpus; doc comments on the remaining new functions.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: test YCCK chroma and trailing-data samples

- The ink fixture gains a YCCK case with Cb and Cr away from neutral, with
  the T.871 values worked out by hand: a conversion that drops or swaps
  the chroma now fails it.
- Bytes added before EOI leave the samples as the clean stream's; the test
  now checks that, not only that reading in order and seeking agree.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

---------

Co-authored-by: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 07:37:06 -05:00
861c5e172d craft-audio: Opus decoder (7/10) (#445)
* craft-audio: Opus decoder

FilmCraft's `filmcraft-opus` decoder as `craft_audio::opus` behind
`opus-decode`: SILK, hybrid and CELT, every frame size, PLC and FEC,
multistream (`Decoder`) and single stream (`StreamDecoder`), 8-48 kHz
output.

- `TrackDecoder` decodes Opus; mono and stereo without an `OpusHead`
  (RTP, MPEG-TS) get a default one. Lost packets use Opus's own
  concealment.
- The native read path takes Ogg Opus (pre-skip and end trimming per
  RFC 7845), and Opus in Matroska/WebM and MP4, where packet starts
  after a seek come from the packets' own lengths.

The RFC 8251 conformance test runs all 36 vectors.
`scripts/fetch-opus-vectors.sh` downloads them (SHA-256 checked) and
the CI test job caches them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#445)

- CELT concealment keeps its pitch in its own field. It reused postfilter_period_old, which the
  next good frame's post-filter cross-fades from.
- Matroska Opus seeks: a failed packet read no longer caches a wrong start; a one-byte packet
  is read as one byte.
- A malformed OpusHead in Ogg is reported as malformed instead of falling back.
- The libopus oracle tests skip through the testkit, so CRAFT_MEDIA_REQUIRE_ORACLES fails them.
- Decoder::reset documents that it re-arms pre-skip trimming.
- fetch-opus-vectors.sh retries its size probe.
- CI: the vectors are fetched after ffmpeg is installed, and apt's timeout runs as root so a
  stalled apt-get is killed and frees the dpkg lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#445)

- CELT: the noise-based concealment sets skip_plc, as libopus does, so
  pitch-based concealment waits for two good frames in a row instead of
  extending history that holds noise.
- SILK: decode_only_middle is per call, as in libopus. A lost or mono
  frame no longer reuses the mid-only flag from an earlier frame.
- A damaged Opus packet is concealed at its own length (from its TOC and
  framing) rather than the previous packet's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 07:36:57 -05:00
Gerrit TimmerandDylan M. Taylor 5b58973bc6 craft-media: index a Matroska file progressively (#430)
* craft-media: index a Matroska file progressively (OpenOptions::index_limit, MkvFile::extend_index)

* craft-media: Matroska element headers never read past their parent

---------

Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 07:26:24 -05:00
Aleksander HelgakerandClaude Opus 5.5 ca99f91a18 craft-audio: AAC encoder (6/10) (#444)
* craft-audio: AAC decoder

FilmCraft's `filmcraft-aac` decoder as `craft_audio::aac` behind
`aac-decode`, kept over symphonia's: the same quality (about 135 dB
SNR against ffmpeg), and it decodes 5.1, which symphonia refuses.
HE-AAC decodes as its AAC-LC core, upsampled 2x to the stream's rate.

- `TrackDecoder` decodes AAC, including ADTS-framed packets without
  extradata (MPEG-TS, raw `.aac`).
- The native read path takes AAC in MP4 and Matroska and raw ADTS
  streams (frame sync over the file, with resync after damage).
- AAC errors convert into `AudioError`.

The AAC module's shared parts (Huffman, MDCT, TNS, ICS) arrive whole
as ported; their encoder half is unused until the encoder PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#443)

Raw ADTS and TrackDecoder:
- Channel configuration 0 (a PCE in the stream, as ffmpeg's -aac_pce and
  AudioToolbox's 7.x write) decodes; it was refused.
- Frames with several raw data blocks are refused; only their first block
  was decoded, silently losing the rest.
- A frame that changes the sample rate, profile or channel configuration is
  concealed; it played at the wrong speed.
- A source that fails during the frame scan is AudioError::Io; it looked
  like the end of the file.
- A stream after an ID3 tag longer than the file head is found.
- conceal counts all the ADTS frames of a packet, and a packet of more than
  1024 frames is refused before it is decoded into memory.

AAC decoder:
- Indexed channel configurations place elements by arrival order, not tag,
  as ffmpeg does; tagged differently, channels were swapped.
- A PCE in an access unit that fails no longer changes the layout, which
  left every later packet failing.
- Non-finite samples are zeroed before the output and the overlap.

MP4: probe and decode agree: the samples' durations end the track when no
single edit does, and an edit cannot outlast the media.

The routing oracle fails on a fixture ffmpeg cannot write when oracles are
required. The README's Limits names HE-AAC v2 as mono, multi-block ADTS,
iTunSMPB, Matroska's tick precision and AAC in CAF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#443)

Corrects round 1:
- A multi-block ADTS frame after the stream has started is concealed at its
  whole length; round 1's filter skipped it, shifting the audio after it.
- Sample durations only trim an MP4's last packet: wrong ones (some count a
  HE-AAC packet's core frames) were cutting audio. Durations of 0 with an edit
  use the edit.

Raw ADTS packets now keep their headers, so TrackDecoder alone handles format
changes, multi-block frames and concealment, for apps as for files, and nested
ADTS payloads are not split again. Headers come from the frame scan's reads,
not one read per frame. The stream's format is the one most of its first frames
share, so one damaged header does not silence the file. The ID3 walk and a
packet's ADTS split are bounded.

AAC:
- HE-AAC is delayed by the SBR filterbanks' 481 core samples; it played 19.5 ms
  early against ffmpeg in every container.
- A failed access unit leaves the decoder's state as it was, so the next packet
  decodes as if it had not been there.
- reset restarts the noise generator, so a seek into PNS repeats.
- A signalled rate that is not twice the core's plays the core; it reported the
  signalled rate with the core's samples.

The decoder oracle's tolerance tightens from 1e-3 to 1e-5. Tests cover HE-AAC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 3 (#443)

Corrects round 2:
- ADTS headers are compared with the stream's format only when the first header
  gave it. With extradata the format comes from there, so ADTS in MP4 whose
  headers say something else decodes again; every packet had been concealed.
- A stream's only program config element is found in its first frames even when
  that frame's header is damaged.
- The test of wrong sample durations patched the second run's count, not the
  delta, so "durations of one" changed nothing. It now patches the delta, and
  checks a file was changed and that seeks land on the same audio.

MP4 seeks place a packet by its index when every packet decodes to the same
length, as decoding does; they used the durations round 2 had stopped trusting
for the length, and returned the wrong audio. A last delta of 0 is overridden too.

Docs: what TrackInfo says for raw ADTS, that opening one reads the whole file, and
which codecs symphonia reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 4 (#443)

- TrackDecoder takes an ADTS stream's format from the one most of the lookahead
  shares, as the file path did, so one damaged first header does not fail every
  packet of an MPEG-TS stream. open_adts now just asks it.
- MP4 samples of several ADTS frames keep the sample table's timing; a packet
  count of one frame each truncated them silently.
- Sample durations past what the packets decode to no longer over-report the
  length.
- split_adts_upto is not behind aac-decode, as split_adts is not: an aac-encode
  build in the next PR did not compile.

Removes an unreachable ID_END check and open_adts's unused core rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 5 (#443)

No change in decoding. An aac-encode build in the next PR needs the
split_adts_upto re-export behind aac-decode, and ID_END only in tests.

Docs say the lookahead's headers give an ADTS stream's configuration, that AAC in
CAF still goes through symphonia, and that ProgramConfig is not #[non_exhaustive].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 6 (#443)

Two tests from round 1 failed on ffmpeg before 9, including CI's 8.0. Neither was
a decoding fault:
- ffmpeg 9 ends an MP4 with no edit list at its sample durations and earlier
  versions present the whole last packet, so the length is checked against the
  file's known duration, not ffmpeg's decode.
- The order ffmpeg gives channels with -aac_pce differs by version, so each decoded
  channel must match a different one of ffmpeg's, whatever the order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 7 (#443)

Reading:
- an element of another type or a tag a program config does not list is
  refused, not written to another channel; only an LFE coded as an SCE
  is still allowed, as ffmpeg allows it
- an explicit sampling rate of 0 is refused
- a packet's later ADTS frames are searched for SBR
- a multi-frame ADTS packet that fails leaves the decoder as it was, and
  one whose frames change the channel count is an error
- the whole ADTS frames before a damaged end of a packet decode
- a failed read of a raw stream's first frames is an I/O error
- an MP4 sample of several ADTS frames past the lookahead keeps its length
- an Ogg file with leading bytes keeps its track order and mask comments

Writing:
- ProgramConfig::write, AudioSpecificConfig::to_bytes and
  AdtsHeader::write refuse what their fields cannot hold

Tests:
- the routing test checks one alignment for the ffmpeg version under test
- a default decode past an Ogg stream symphonia has no track for

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 8 (#443)

From wanikhawar's review:
- AudioSpecificConfig::parse reads backward-compatible SBR and PS signalling
  (a 0x2b7 sync extension after the config), which craft-media's Matroska
  demuxer writes for legacy HE-AAC IDs. Without lookahead those streams decoded
  as their core.
- An MP4 media edit after one empty edit is applied: its priming skip and its
  end. The empty edit's delay is not added as silence.
- A packet lost whole is concealed as long as the last one decoded, so a stream
  of multi-frame ADTS packets keeps its timing.
- ADTS-framed MP4 samples keep their sample table's timing, instead of every
  sample being read at open to look for several frames in one.
- AdtsHeader::write refuses an object type, sampling frequency index or channel
  configuration its fields cannot hold; it masked them into other values.
- AudioSpecificConfig::lc no longer panics for a sampling frequency index with no
  rate, and the decoder refuses such a config.
- AudioSpecificConfig::to_bytes refuses channel configuration 0 without a program
  config element, which would not parse back.
- Upsample2x starts each channel's history afresh when the channel count
  changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 9 (#443)

From CodeRabbit's review of round 8:
- AudioSpecificConfig::parse consumes extensionFlag3 after the config when
  extensionFlag is set, so the 0x2b7 sync extension after it is found.
- AudioSpecificConfig::to_bytes refuses a sampling rate of 0, which would not
  parse back.
- ADTS-framed HE-AAC in MP4 whose sample durations count core frames is not cut
  to the durations' half length: durations short of one frame per sample no
  longer end an ADTS-framed track, and its length is unknown until decoded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 10 (#443)

ADTS-framed AAC in MP4: durations short of the frames in the samples read
ahead no longer end the track. With HE-AAC samples of two ADTS frames each,
timed as two core frames, the durations matched one output frame per sample,
so the undercount check passed and the track stopped at half its length,
which probe reported as known. The frames each read-ahead sample holds are
now counted, and durations short of them leave the length unknown until
decoded, as durations short of one frame per sample already did. Correctly
timed AAC-LC samples of several frames keep their end trim.

Reported by FredApps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 11 (#443)

Corrects round 10. A short ADTS-framed MP4 track, whose last sample is in
the read-ahead, lost that sample's trim when it cut more than one frame:
the trim counted as durations falling short, so the length was left to
decoding and its padding was presented. The track's last sample is now left
out of the read-ahead comparison.

Reported by CodeRabbit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: document the ADTS-in-MP4 seek and edit limits (#443)

AAC in MP4 stored as ADTS frames, with sample durations that count fewer
frames than the samples hold, decodes in full from the start, but seeks
place samples by those durations and land later than asked, and an edit
list measured in them can end the track early. The README's Limits section
now says so, and the seek example points to it.

Reported by FredApps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: say why the ADTS-in-MP4 seek limit is kept (#443)

The Limits entry from 79f5839 said what goes wrong but not why it is left.
It now says that such a file is outside the MP4 specification: an MP4 AAC
sample is one raw access unit with the configuration in the sample
description (ISO/IEC 14496-14, 14496-3), so ADTS headers do not belong in
it, and a sample's duration is the time it decodes to (ISO/IEC 14496-12).
It says why files within the specification are not affected, what still
works, and why seeks and edits in such files are not corrected: finding
where each sample's audio starts means counting the ADTS frames in every
sample before it, one read per sample, which would slow every ADTS-framed
MP4 at open to work around files outside the specification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: AAC encoder

FilmCraft's AAC-LC encoder behind `aac-encode`: psychoacoustic model,
window switching, M/S, TNS, bit reservoir, CBR and VBR, 1-8 channels,
optional ADTS framing. `rayon` encodes channels in parallel.
`aac::aac_order` maps WAV order to the encoder's input order. Settings
the encoder cannot use are `aac::Error::InvalidSettings`, which
converts to `AudioError::Encode`. `encode` now turns on `aac-encode`.

Tests that need AAC streams written by our own encoder come with it:
the encoder oracle and robustness tests, packet concealment, and
`TrackDecoder` on ADTS without extradata.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#444)

From CodeRabbit's first review of the AAC encoder:

- CBR bitrate limits use the exact bits per frame (bitrate * 1024
  against channels * sample_rate) instead of the truncated quotient, so
  288001 b/s at 48 kHz mono is rejected.
- Rates below 48 bits per channel per frame are rejected. That is the
  largest silent frame per channel measured across 1-8 channels; below
  it the encoder overshot the requested rate (96 kHz mono at 1000 b/s
  came out near 4500 b/s).
- Fill elements are capped so the byte-aligned access unit stays within
  6144 bits per channel. write_fill can write 8 bits over its request,
  which gave 769-byte mono units at the maximum rate and when the
  reservoir was flushed on the last frame.
- encode() buffers one frame of input at a time. Appending the whole
  slice and draining 1024 samples per frame copied the rest of the
  input once per frame, quadratic in the slice length. Output is
  byte-identical to feeding the input in chunks.
- README: 8 channels are taken in channel configuration 7's element
  order; a WAV 7.1 buffer has no equivalent there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#444)

From CodeRabbit's review of round 1:

- The CBR minimum adds the program config's bits when ADTS is on. The
  7-channel layout repeats it in every ADTS frame, and at full
  bandwidth a silent frame took 344 bits against the 336 bits that
  31,500 b/s gives at 96 kHz. Bandwidth alone stays within 48 bits
  per channel.
- write_fill never writes more than it is asked for: each element's
  size comes from the bits left, so the total ends 0-6 bits short.
  The old version could pass the request by 8 bits and could also
  stop early (asked for 14, it wrote 7).
- The encoder fills to the unit's target end, capped at
  max_frame_bits, instead of keeping round 1's 8-bit margin. Frames at
  the maximum rate were 767 bytes when there was little to code; they
  are now exactly 768 bytes per channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 3 (#444)

Encoder::new rejects a NaN VBR quality, the known issue ported from
FilmCraft and listed in the PR description.

NaN passes f32::clamp, so the rate-distortion multiplier became NaN.
The frame-size loop compares it with `<` and never ran, and no band
was skipped, so every frame was coded at full precision: mono units
reached 2465 bytes against the 768-byte limit, and 8-channel units of
up to 17024 bytes did not fit the ADTS length field, so they went out
without headers. Infinities and other out-of-range values are clamped
and stay within the limit; the test checks both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 4 (#444)

From CodeRabbit's review of round 3:

- The CBR fill target is rounded down to a whole byte. Round 2 filled
  to an unaligned target, so after byte alignment a unit could end up
  to 7 bits past the bits it had: in a sweep of silent CBR encodes,
  1188 of 13200 frames overspent, all of them frames with fill, hidden
  by the level clamp at 0. The unit now ends exactly at the rounded
  target. The 0-7 bits rounded off stay in the reservoir and are left
  unspent after the last frame; the comment says so instead of
  claiming an exact total.
- The NaN-quality test checks that infinities and out-of-range values
  encode exactly as the nearest end of the 0.5-6.0 range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 07:21:26 -05:00
139a39e94c cms: write the profile ID into encoded profiles (#749)
encode() stamps the header profile ID: the MD5 of the profile with the
flags, rendering-intent and ID fields zeroed (ICC.1:2022 7.2.18).
profile_id(bytes) computes it for any profile. Adds md-5 (RustCrypto,
MIT OR Apache-2.0, no default features); digest was already in the lock.

Closes #748.

Co-authored-by: KenM76 <KenM76>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 07:17:22 -05:00
a9608afe43 cms: read named-colour (ncl2) profiles and convert their colours (#747)
* cms: read named-colour (ncl2) profiles and convert their colours

A named-colour profile's ncl2 tag (ICC.1:2022 §10.17) was never read, so
a spot-colour library could not be used. Profile::named_colors() returns
its entries: the name joined from prefix, root and suffix, the PCS value
and any device coordinates. NamedColors::to_device(name, dst) converts
one into a destination profile, relative colorimetric, through the Lab
identity profile.

Lab in ncl2 uses the legacy 16-bit encoding (0xFF00 = L* 100) in every
profile version; reading it as v4 puts L* about 0.4 low. Counts are
checked against the tag size before allocating, and more than 15 device
coordinates is refused.

Adapted from icccer's named_color module (MIT, same author), rewritten
onto craft-cms types.

Refs #746

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* cms: cap the ncl2 tag-table walk; doc fixes

Cap the tag count at 1024, as Profile::parse does, so the table offsets
cannot overflow on a 32-bit target. Correct the named_colors doc and add
one for is_empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* cms: README notes named-colour tables are readable

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: KenM76 <KenM76>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 07:12:16 -05:00
Tex JerniganandClaude Opus 5.5 7480fc2ac4 craft-media: write and read MP4/MOV chapter lists (#526)
* craft-media: write and read MP4/MOV chapter lists

WriterOptions::chapters (Chapter { start, title }, starts in the movie
timescale) makes Mp4Writer write a chapter list in the two forms players
read:

- a QuickTime chapter track (a disabled text track referenced by
  tref/chap from the first video or audio track, one UTF-8 sample with an
  encd atom per chapter) for QuickTime / AVFoundation, and
- a Nero udta/chpl list (version 1, 100 ns units) for VLC and
  ffprobe -show_chapters.

Chapters are normalized (sorted, one per start, inside the movie, at most
255 chapters with 255-byte titles cut at a character boundary, control
characters replaced) and a late first chapter is placed with an empty
edit. FragmentedWriter writes chpl only. The demuxer reads both forms
back: Mp4File::chpl_chapters and Mp4File::read_chapters(src).

Ported from FilmCraft's own isobmff crate (storytold/filmcraft#2821, for
storytold/filmcraft#2747), now that FilmCraft uses craft-media.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-media: read chapters only from enabled tracks, inside the edit, with bounded titles

read_chapters took a tref/chap from any track, though QuickTime requires the referencing track to be
enabled. It ignored the chapter track's edit list, so a chapter before the edit start became a second
chapter at zero and samples past its end were returned. It kept titles of any length.

Now a disabled host is skipped; chapters outside the first non-empty edit are dropped, and one spanning
its start begins there; each sample read is capped at 4098 bytes and titles are cut to
MAX_CHAPTER_TITLE_BYTES, which bounds what is kept. Later edits are still ignored.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* craft-media: a chapter title longer than the sample read keeps its start

A sample declaring more title than the 4096 bytes read parsed to nothing, so the
chapter lost its whole title rather than being cut to MAX_CHAPTER_TITLE_BYTES.
A sample cut by the read now decodes what it holds; one read whole but declaring
more than it has is still no title.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 07:11:59 -05:00
Arshiya Rahgozar 1e4271e369 media: bound Matroska resync work and header reads (#457)
* media: bound Matroska resync work and header reads

What was wrong: the Matroska demuxer did no bounded work when recovering from
damage. `resync` tested every plausible-looking element ID, and each test could
refill the 64 KiB read window at a far offset, so a damaged region cost about
13,000x read amplification (a 1 MB crafted file read 4.3 GB). The EBML header
was read with no size cap, so a header as large as the file read the whole file
into memory. The warnings list grew without limit, and block and lace sizes
were narrowed to u32 with `as`, so a block over 4 GiB was indexed with a
truncated size.

What changed: damage recovery now has a 64 MiB per-file budget, charged by the
bytes actually read to refill the window; the scan stops when it is spent and
a warning says so. The EBML header is limited to 4 KiB (real ones are a few
dozen bytes). At most 256 warnings are kept, plus one saying the rest were
dropped. Block and lace sizes use checked conversion; a block that does not fit
is skipped like other damaged blocks.

How it was tested: four new tests in synthetic_tests.rs failed before the fix
and pass after it: a counting ByteSource over dense fake Clusters (reads stay
within the budget), an EBML header as large as the file (quick NotMatroska
error, small reads), a SeekHead with thousands of bad entries (warnings
capped), and a sparse source with a block over 4 GiB (not indexed). The rest of
the matroska tests still pass, and clippy is clean with -D warnings.

This carries over the intent of storytold/filmcraft#2951 (a recovery-work
budget for Matroska) now that the demuxer lives in craft-libs, and adds the
header, warning and size limits from the audit.

Completes the Matroska part of storytold/filmcraft#2794

* media: cap laced Matroska block reads and give each index pass its own recovery budget

Review follow-ups to the Matroska recovery budget.

A laced block's whole body was read into memory before any size check, so a
block declaring 4 GiB and carrying a lacing flag allocated that much. Laced
blocks hold small frames, so a body over 16 MiB is now skipped like other
damaged blocks, without reading it.

The recovery budget lived in the read cache, which open and build_index share,
so open(index: false) followed by build_index() charged the same damaged
region twice and could drop later clusters. Each pass over the file (open_impl)
now starts with the whole budget, and packet reading starts with its own.

The warnings test now pins the exact count (256 kept plus the note) and the
note's text.

Tested: a sparse file with each lacing flag fails to open before and opens with
no samples after; a file whose pre-cluster damage costs about 50 MiB of recovery
reads loses its cluster in the second pass before and keeps it after. The
matroska unit tests pass, clippy -D warnings and fmt are clean.

* media: restore the packet-reading recovery budget after build_index

build_index runs a full pass over the file on the shared read cache, so it
spent the recovery budget that next_packet then relied on. After
open(index: false) and build_index(), reading stopped at a damaged region the
index had recovered across, ending playback before the next cluster.

build_index now gives packet reading a whole budget again once its own pass is
done.

Tested: a file with a cluster, about 40 MiB of recovery reads of damage, then a
second cluster; after open(index: false) and build_index() the index held both
clusters but reading ended after the first before the change, and returns both
after it.
2026-11-11 07:11:34 -05:00
JohnandClaude Opus 5.5 9c5cc064e9 craft-media: Ogg oracle tests against ffprobe (#579)
* craft-media: Ogg oracle tests against ffprobe

oracle_ogg.rs compares the Ogg demuxer with ffprobe on ffmpeg-made fixtures,
like oracle_matroska.rs: every packet byte for byte (Adler-32 from
`-show_data_hash`), codec, rate, channels and Theora picture size, and
OpusTiming's starts, durations, pre-skip and total against ffmpeg's pts,
durations and Skip Samples side data. Fixtures: Opus (stereo, mono 16 kHz with
2.5 ms frames, 60 ms frames with an odd length, 5.1, one 10 ms page, a page per
packet), Vorbis, FLAC in Ogg, Theora + Vorbis, grouped streams and a chained
file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-media: Ogg oracle compares Opus starts on ffmpeg's clock

OpusTiming's starts now equal ffprobe's pts as they are, not relative to the
first packet, so a shift of every start fails the test; a stream that starts
at granule 0 also starts at -pre_skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 07:11:24 -05:00
JohnandClaude Opus 5.5 4970d4cfae craft-media: roundtrip test builds on 32-bit targets (#578)
`(writes * MIB) as u64` multiplies in `usize` before widening: 4,097 MiB
overflows a 32-bit `usize`, so `cargo test --target i686-pc-windows-msvc`
failed to compile ("this arithmetic operation will overflow"). Widen first.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 07:11:07 -05:00
Dylan M. Taylor 73702b326a Add craft-silhouette-studio: Silhouette Studio document reader (#805)
Bounded, fail-closed reader for .studio and .studio3 documents: outlines, line colours and page size in millimetres. Protected designs are refused.
2026-11-11 06:40:49 -05:00
Dylan M. Taylor 6154d957f8 plate-test: bound text sampling, image size, resource depth and plate bytes (#544)
* plate-test: bound text sampling, image size, resource depth and plate bytes

- judge: sample text regions over integer pixel ranges clipped to the plate,
  so a huge or NaN region cannot stall the float counter.
- reference: the writer enforces the scene validator's image pixel cap
  (MAX_IMAGE_PX, shared with scene.rs) before generating pixels.
- structure: Walk::content follows nested Form/pattern/soft-mask resources
  at most MAX_RESOURCE_DEPTH (16) levels deep.
- tiff: a plate may hold at most MAX_PLATE_PIXELS (1 GiB) and its buffer is
  reserved fallibly.

* address review: report resource-depth truncation as an error; re-expand shared streams reached shallower

* address review: list shared groups and graphics states once; accept empty terminal Resources
2026-11-11 06:26:53 -05:00
Dylan M. Taylor 3dcaa07a81 Add craft-svg-paths: SVG geometry reader (#802)
Reads an SVG's shapes, clips and page size into kurbo paths in millimetres, with a bounded CSS cascade and fail-closed handling of unsupported content. The caller maps style to its own shape kinds.
2026-11-11 06:23:12 -05:00
Dylan M. Taylor 34685c4cdc Add craft-gcode: Grbl-style G-code writer for laser cutters and plotters (#801)
Polylines in millimetres to G0/G1 moves with laser power, feed and passes. Comments are kept free of Grbl real-time bytes.
2026-11-11 06:22:20 -05:00
Dylan M. Taylor 3e5b5ecb60 i18n: keep fold search, isolate wrapping and locale fallbacks linear (#539)
* i18n: keep fold search, isolate wrapping and locale fallbacks linear

- Folded::matches restarted str::find one character past every occurrence
  turned down at a combining mark, so overlapping occurrences of a long
  needle cost haystack x needle. It now finds every occurrence, overlapping
  ones included, in one Knuth-Morris-Pratt pass and checks the boundaries
  as before.
- wrap searched the whole open-control stack for an isolate at every PDI;
  it now keeps a count of open isolates.
- candidates made one joined prefix per subtag of a locale tag of any
  length; a tag longer than 128 bytes is not a locale and has none.

* address review: length check only on the public candidates entry
2026-11-11 06:17:52 -05:00
Dylan M. Taylor acc262e69c font-catalog: hand fontique only bounded font-store files (#537)
* font-catalog: hand fontique only bounded font-store files

fontique 0.11 imports a collection by registering the whole file once
for each face in it, scanning every face again each time and rebuilding
the family's font list, so a downloaded collection whose header declares
many faces costs far more than its size. The store is now walked here
(bounded depth and file count, no symbolic links) and collections that
declare more than 64 faces are left to the catalog's own scan, which
reads each face once.

* font-catalog: load font-store files one at a time and keep fontique's walk

fontique 0.11 re-adds every font gathered in one load_fonts_from_paths
call to its family for each face it registers, so handing it the whole
store at once repeats the store's fonts quadratically in the file count
(126,230 family entries for 2,084 faces on a 400-file test store, 4 s;
one call per file gives the same families, fonts, order and default
fonts with 2,084 entries in 90 ms).

The store walk now matches what fontique did with the folder before:
links are followed (to regular files and folders, each folder once),
depth 16, read_dir order, with a 200,000-entry budget like the
catalog's scan. Pipes, devices and dangling links are skipped.
2026-11-11 06:08:19 -05:00
Dylan M. Taylor 0230aef364 prepress: bound flattener, tiling, image, raster grid and ink-set work (#545)
* prepress: bound flattener, tiling, image, raster grid and ink-set work

- compose: a group with more than MAX_GROUP_SEGMENTS (200k) path segments
  is rasterized whole (ComplexReason::TooManySegments) before its
  arrangement, whose crossings grow with the square of the segment count.
- geom: transparent_groups returns at once when nothing is transparent;
  overlap_groups folds the open boxes into one group past
  MAX_OVERLAP_COMPARISONS (4M) instead of comparing every pair.
- pdf: item_count charges path elements and glyphs, so the tiling budget
  bounds the geometry a cell expansion copies; Channels::apply refuses to
  spread samples past MAX_IMAGE_SAMPLES (256 MiB).
- shade: pixel_size refuses grids past MAX_GRID_PIXELS after rounding, so
  a hairline-thin area cannot make the other side unbounded.
- raster: Grid::within_budget rechecks the grid after scaling and scales
  by the pixel ratio when the square-root pass is not enough.
- ink: InkSet keeps a name index, so adding many spots is linear.

* address review: count image report only after channels apply

* address review: validate finiteness before segment-limit fallback, loop raster grid budget reduction
2026-11-11 05:55:27 -05:00
Dylan M. Taylor 208171b920 update: stage replacements exclusively and refuse oversized tar metadata (#563)
* update: stage replacements exclusively and refuse oversized tar metadata

- self_update: stage_copy creates the staged file beside the target with
  an unpredictable name (tempfile, O_EXCL) and writes bytes and mode
  through the handle, for both the AppImage and the tarball standalone
  replacement; a link at the old .<name>.update names is never followed.
- install::tree::unpack: check_metadata walks the tar headers first and
  refuses PAX or GNU long-name records over 1 MiB, which the tar crate
  buffers whole before the entry loop sees a size; both passes read at
  most MAX_TAR_BYTES of decompressed stream.

* address review: refuse size-truncated update archives, fixed staging prefix
2026-11-11 05:36:05 -05:00
Dylan M. Taylor 5279ecdc0e font-catalog: bound name records before pairing legacy names (#536)
* font-catalog: bound name records before pairing legacy names

The scanner paired every legacy family record with every legacy style
record of the same language before removing duplicates, so a small name
table repeating those records allocated millions of strings (a million
pairs from 4,000 records). Name records are now read per ID up to a cap,
deduplicated as they are read and kept up to a cap, and the legacy pairs
have a budget of their own.

* font-catalog: count distinct legacy pairs against the pair budget

Collect the legacy (family, style) pairs into a sorted set and stop at
MAX_LEGACY_PAIRS distinct pairs, so the same pair repeated across
languages cannot use up the budget. Output order is unchanged (sorted,
deduplicated), and the test now expects exactly the full budget.

* address review: typographic family decided by name(), distinct-text cap for all-language names, document the legacy pair cap

* address review: keep selected full and PostScript names in lookup keys past the name cap
2026-11-11 05:26:36 -05:00
FredApps 60ede99985 print: xps_driver says whether a Windows printer's driver renders XPS itself (#804)
* print: xps_driver says whether a Windows printer's driver renders XPS itself

craft_print::xps_driver(printer) reads PRINTER_DRIVER_XPS through winprint's new
PrinterDevice::is_xps_driver (storytold/winprint.rs#21): Some(true) for v4 drivers and v3
drivers with an XPS rendering filter, Some(false) for GDI drivers, None when it can't be read and
on other systems. On a GDI driver, reached through Windows' XPS-to-GDI conversion, a page with
transparency prints with halos, as black boxes or not at all (storytold/pdfcraft#6155, #5838,
#5940), so PdfCraft will flatten it for those drivers only.

The Windows end-to-end test checks the CI queue's "Generic / Text Only" driver reads as GDI.

* Test: the GDI-driver check in a helper (keeps the end-to-end test under the complexity limit)

* winprint bc06a7c48b768eb27a828c7986f6d2868b245ee0: storytold/winprint.rs main with #21 (PrinterDevice::is_xps_driver)
2026-11-11 07:48:44 +01:00
smittie2000 797d21a3f3 craft-update: accept exactly one download link per asset (craft-launcher#764); keep the Linux self-update off other copies
Fixes storytold/craft-launcher#764 in craft-update, and with it storytold/craft-launcher#765 (each link is tied to its own release's tag). Also brings back a Linux self-update rule that the extraction lost.

## Why

craft-update was copied out of ArtCraft Launcher in #311, so its `release.rs`, `version.rs`, `platform.rs`, `distro.rs` and `proxy.rs` are the launcher's files. craft-launcher#764 was fixed only in the launcher's copy. This PR brings the fix here, where every Craft app's self-update runs. Once this PR and #389 merge, the launcher drops its copy (storytold/craft-launcher#930).

## 1. Each asset has exactly one acceptable download link (#764)

**The problem:** an asset's link could point at another release, or climb out of the app's releases with `..` or `%2e%2e`, which github.com decodes before routing.

**The fix:**
- `release::expected_download_url(repo, tag, name)` builds the only link an asset may have: `https://github.com/<repo>/releases/download/<tag>/<name>`. The tag and name may only use ASCII letters, digits and `._+-`. A `+` is written `%2B`, as GitHub writes it.
- Reading the GitHub API, an asset is kept only when its link equals that URL. This also ties each link to its own release.
- `release_from_sums` builds its links with the same function.
- `bind_artifact_version`, which compares the tag and file name inside the link with the release's, reads `%2B` there as `+`. It decodes nothing else.
- An encoded dot (`%2e`) is rejected like an encoded slash (`%2f`, `%5c`), both in links and in file names from `SHA256SUMS.txt`.
- `tag_from_redirect` refuses `.` and `..`.
- `install::check_download_link` checks the asset's link, and the link of the release's `SHA256SUMS.txt`, just before downloading. It runs in `install_at` **and** in `self_update::update`, because a self-update doesn't go through `install_at`.

## 2. Linux self-update: only `/usr/bin/<id>` counts as the package

Before #311, the launcher checked the package databases (dpkg, rpm, pacman) only for `/usr/bin/<id>`. The extraction dropped that check and added a `Standalone` fallback, which broke two cases:
- **A copy in `~/bin`** was treated as the deb/rpm/Arch package whenever that package was also installed. The update installed the package with pkexec, then relaunched the `~/bin` copy, which it never touched.
- **An AUR build in `/usr/bin`**, which the code comment says is pacman's to update, became `Standalone`. The update then tried to overwrite a file pacman owns.

`linux_location` brings the old rule back:

| Where the executable is | What happens |
|---|---|
| `/usr/bin/<id>` | checked against the package databases, as before |
| anywhere else under `/usr`, `/opt`, `/bin`, `/sbin`, `/nix`, `/snap` or `/gnu` (Guix), or in a Homebrew `.linuxbrew` prefix | no self-update: its package manager updates it |
| `/usr/local`, the home folder, anywhere else | updates itself (`Standalone`) |

**Symlinked installs are left alone on purpose.** `current_exe()` follows symlinks, so a package that puts the real file in `/opt` and a symlink in `/usr/bin` lands in the second row and doesn't self-update. That's intended. All 14 Craft apps' packages (`packaging/linux/nfpm.yaml`) put the real file at `/usr/bin/<id>`, so this only happens with someone else's repackaging, such as an AUR `-bin` package, and that copy belongs to its package manager.

## 3. Fixes from reviewing this PR with craft-launcher (3 November)

Rebased onto main (`55332dd`) instead of merging it; #460, #392, #481 and #478 changed the same files. Then two reviews against craft-launcher main, which feeds this crate its releases:

- **A release knows its repository.** `Release::repo` is the repository the release was read from, and `check_download_link` checks links against it (empty means the app's own). The launcher's feed can rename a repository (`Feed::repo_for`); checking against `app.repo` refused the renamed repository's downloads.
- **A `+` matches either way.** `release::same_download_url` reads `%2B` as `+` on both sides. The launcher's own `release_from_sums` and feed write a plain `+`, and so do releases saved by an older version, so only accepting `%2B` refused those self-updates. Nothing else is decoded (`%2b` still fails).
- **The launcher's tag and file-name rule (craft-launcher#1024).** `valid_tag`: up to 64 of `[A-Za-z0-9._+-]`, no leading dot, used for the API's `tag_name`, the redirect and download links. `valid_file_name`: the same characters, no leading or trailing dot, no Windows device name (`CON`, `nul.txt`, `COM1.msi`). The launcher's feed calls `release::valid_tag`, so craft-launcher#930 needs it here.
- **`ask_pages` fetches `SHA256SUMS.txt` from `expected_download_url`**, the link it stores.
- **Guix and Homebrew on Linux** are left to their package manager (table above), like Homebrew and Nix on macOS (#460).

## What changes for apps

- **Loopback mirrors:** a build that downloads from a developer's loopback mirror must turn on `craft-update/local-mirror`, or the new check refuses the mirror's links. The launcher's `local-mirror` feature turns it on (craft-launcher#930). Apps rolling out under #319 without a mirror need no change.
- **Linux:** copies under the package-manager folders above no longer update themselves. That's how the launcher behaved before #311.
- **`+` in a tag:** handled (section 1). No Craft release uses `+` today. GitHub's link form was checked on bell-sw/Liberica's releases: the tag and name keep `+`, the link has `%2B`, and the `releases/latest` redirect has a plain `+`.

## Compared with the launcher

I compared launcher 2d12245 (the snapshot #311 copied) and launcher `main` with this crate.
- **All of the launcher's security fixes from 15–25 October are here:** #747/#864, #865/#748, #867, the proxy part of #868, #689, #815, #821 and the archive limits. The launcher's two commits after the snapshot (#871, #903) change no behaviour here.
- **Two things are here but not yet in the launcher:**
  - the minute-long `hdiutil attach` retry (#349): craft-launcher#930 takes it;
  - the zip 9 drive-prefix test (#335): it waits for the launcher's zip 9 bump (storytold/craft-launcher#899).
- **Follow-ups that don't belong in this PR** are filed separately (linked below once filed).

## Tests

- **New:** `a_download_link_must_be_its_releases_own`, `a_plus_in_the_tag_is_percent_encoded_in_the_link`, `install_accepts_a_plus_release_up_to_the_download`, `update_accepts_a_plus_release_up_to_the_download`, `checksum_names_with_encoded_separators_make_no_download`, `update_refuses_a_link_that_is_not_its_releases_own`, `only_the_packages_own_path_is_a_package_and_system_folders_are_left_alone`.
- **Extended:** `install_rejects_an_artifact_whose_version_is_not_the_advertised_one` (another repo's link, a foreign checksums link), `drops_downloads_that_point_elsewhere`, `file_names_cannot_escape`, `tag_from_redirect`.
- **Mutation check:** `update_refuses_a_link…` fails when the call in `self_update::update` is removed.
- **Mutation check:** both `…_accepts_a_plus_release_up_to_the_download` tests fail with the bind error when the `%2B` decoding is removed.
- **On Windows, after merging main (31 October):** `cargo test -p craft-update`: 232 passed, 3 ignored, the same without default features. Clippy `-D warnings` is clean with default features and with `egui,local-mirror`, and `cargo fmt --check` is clean.
- **Merged with #389 (before the 31 October merge of main):** 173 passed, 28 without default features, and clippy for wasm32 is clean.
- **Launcher (before the 31 October merge of main):** with this branch merged with #389 (craft-launcher#930 pins that merge), the full ArtCraft Launcher suite passes: 401 tests, plus clippy, fmt and the wasm32 build of its core.
- **Added for review (3ad9074):** `two_links_for_one_asset_keep_only_the_releases_own`, `install_refuses_links_off_the_releases_own_before_downloading` (other hosts, and a `SHA256SUMS.txt` linked elsewhere), `install_refuses_while_the_installed_copy_is_running`, `a_download_that_fails_its_checksum_replaces_nothing`; `a_download_link_must_be_its_releases_own` also refuses other hosts. On Windows: 236 passed, 3 ignored; clippy and fmt clean. The bad-checksum test fails when the server sends the real file.
- **Added in section 3:** `tags_are_plain`, `a_plus_link_matches_either_way_and_nothing_else_does`, `a_release_knows_its_repository`; `file_names_cannot_escape` takes #1024's cases; the install test checks a renamed repository and a plain `+`; the Linux location test adds Guix and Linuxbrew. On Windows after the rebase: 251 passed, 3 ignored; the workspace builds; clippy `-D warnings` is clean for this crate (with `local-mirror`) apart from `translate::button` being unused, which comes from main's #508.
- **Not built yet:** the Linux-only `detect` code, and the Unix branch of the running-app test. CI builds it first, once a maintainer approves the workflow runs.


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
  * Update downloads are now restricted to assets belonging to the selected release, including checksum files. Unrelated or invalid links are rejected before a download begins.
  * Release links with unsafe path components or invalid redirect tags are rejected. Encoded plus signs in release tags and filenames are handled correctly.
  * Linux installation detection more accurately distinguishes package-managed installations, other system locations, and standalone installations, improving whether self-updates are available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 07:47:52 +01:00
Joseph Anglada 49d865dbdd Add caller-styled dialog section frames
Group related dialog and inspector settings in reusable frames with optional accessible headings.

Makes the reusable part of [WordCraft #4038](https://github.com/storytold/wordcraft/pull/4038) available to other Craft apps. Includes API documentation, integration guidance and regression coverage. Consumer adoption remains separate; applications retain their themes, localization and command adapters.

Validation on Windows: workspace formatting, strict Clippy and tests; craft-ui Clippy/tests with all features; craft-ui WASM check with all features. Upstream CI results are tracked on this PR.

Includes the shared test-portability prerequisite [#267](https://github.com/storytold/craft-libs/pull/267); the feature itself is independent of the other UI proposals.

[Integration guide](https://github.com/fluorine/craft-libs/blob/fad5c05170d43f9bdf86c6063131e28669fcb176/crates/ui/README.md).

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
  * Added customizable sections for grouping content vertically, with optional headings and configurable spacing.
  * Sections can fill the available width or use a caller-defined width, and return the result of their content callback.
* **Documentation**
  * Added usage guidance and an example for styling sections in dialogs and inspector panels.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 07:31:25 +01:00
.CooCoo 5a93050c9f update: integration of the open fixes to craft-update (#763)
* update: an unsigned Windows copy gets Download instead of Update and Restart

The update helper installs nothing for an unsigned copy, so the prompt's
Update and Restart button on a local or unsigned build downloaded the
release and then stopped with "this copy isn't signed". is_self_updating
now answers false for an unsigned copy on Windows, and the prompt offers
Download. CRAFT_UPDATE_DEV still offers the update.

* update: check that a Windows copy can install before downloading

windows_precheck ran after the download, so a copy that could not install
downloaded the whole release first. The checks that do not depend on the
download now run before it; a new MSI's signature is checked after it.
The cache folder is created first, so a first update passes the check.

* update: a failure recorded before the running program is not reported

take_outcome reported any recorded failure at the next start, so a copy
put in place after an update had failed still opened with "The update
could not be installed". A failure record older than the running program
is now removed without being reported. A failure recorded after the
program was put in place, such as a failed helper's, is still reported.

* update: a Cargo build with a separate build directory is a development build

With build.build-dir set, Cargo keeps .rustc_info.json and the profile's
.fingerprint, deps and build folders in the build directory, and the
program's own folder has only its lock file. Such a build counted as an
installed copy and was offered Update and Restart. A profile folder with
Cargo's .cargo-lock under a target root with CACHEDIR.TAG is now a
development build, and gets the download link.

* update: a cache folder that can't be created is reported as that

same_cache_as_helper ignored the error from creating the folder, so a
failure read as a cache in the wrong place. It now names the folder and
the error.

* craft-update: a slow macOS signature check is not reported as a bad signature

`signed_by` gave `codesign` 120 seconds and treated every failure as a bad
signature, so a Mac that could not finish in time was told the app "isn't
signed by its developer".

The check now has 10 minutes and checks one bundle at a time. A job waiting
for its turn reports `JobPhase::Waiting` and can be canceled. A timeout, a
`codesign` that cannot start or is stopped by a signal, and any exit other
than 1 or 3 report that the signature couldn't be checked. Exit 3 refuses the
app as not signed by its developer. Exit 1 refuses it with `codesign`'s own
reason, because `codesign` also exits 1 for a bundle it cannot read. The app
is not installed in any of these cases.

On macOS, an install or uninstall now stops when `pgrep` gives no answer,
where a timeout used to read as "not running". `pgrep` has 30 seconds. An
install looks again after the signature check, which can wait behind other
apps' checks, and does not replace an app opened during the wait. A
self-update skips that second look.

* craft-update: a copy that cannot be replaced gets Download

A macOS bundle, an AppImage or a Linux standalone program was offered
Update and Restart even when the update could not replace it, so the
update failed only after the whole download.

`current_at` now leaves out such a copy when `access(2)` refuses what
the update needs. On macOS that is adding a subdirectory to the parent,
and adding a subdirectory to and deleting the bundle. On Linux it is
writing and searching the folder. A macOS copy under App Translocation
is also left out. Any other error from the check leaves the update on
offer, and `CRAFT_UPDATE_DEV` still offers it.

* craft-update: a macOS standalone copy that cannot be replaced gets Download

macOS `detect` returns `Install::Standalone` for a loose program, and
the update replaces it by writing beside it, as on Linux. `can_replace`
now checks that folder on macOS too, so a copy in a folder the user
cannot write gets Download instead of failing after the download.
`CRAFT_UPDATE_DEV` still offers the update.

* craft-update: a standalone copy whose share files cannot be replaced gets Download

The update replaces a standalone copy's files under `share` as well as
the program when the app's `share/<id>` folder sits next to `bin`. The
offer now also checks those folders, the `applications` and
`icons/hicolor` folders a release writes to, and the nearest existing
parent of one that doesn't exist yet.

* craft-update: a standalone macOS program without a release tarball gets Download

On macOS a program outside an app bundle counts as a standalone copy, and
the self-update replaces it from a macOS .tar.gz. Releases carry a disk
image and a command-line zip for macOS and no .tar.gz, so the prompt showed
Update and Restart and the update stopped with "has no Tarball (.tar.gz)
for macOS".

The prompt now also asks release_installs, which on macOS offers Update and
Restart to a standalone program only when the release lists that asset.
Without it the prompt shows Download. A release whose file list was not
loaded keeps the offer, because the install fetches the full release first.
CRAFT_UPDATE_DEV still offers the update. Other platforms and other kinds of
install are not checked against the release.

* craft-update: CRAFT_UPDATE_DEV no longer skips the release check

A standalone macOS program run with CRAFT_UPDATE_DEV was still offered
Update and Restart for a release without a macOS .tar.gz, and that update
failed. release_installs now checks the release for that copy too.

release_installs_at takes the executable, the opt-in setting and the
platform, and a new test checks a loose program in a temporary folder
against releases with and without the .tar.gz. The tests use SoundCraft's
release files, since craft-launcher ships no command-line zip.

* craft-update: a failed self-update keeps its download for the next try

On macOS, Linux and FreeBSD, prepare_update removed the downloaded
release file whether or not the install succeeded, so every retry after
a failed copy, signature check, package install or swap downloaded the
whole release again. A failed install now keeps the file, as install_at
already does, and the next try reuses it through Http::download, which
checks it against the published SHA-256 first and fetches it again when
it does not match.

- The cache folder's self-update-download.txt records the name and size
  of the release file before the download starts. A self-update for a
  different release file removes the finished and partial download that
  the record names, so each app keeps the files of one release at most.
- An install that succeeds removes the file and the record. So does a
  failed install whose record could not be written, since nothing would
  remove that file later.
- On macOS, when hdiutil cannot attach the disk image, the file is still
  removed: an image left attached by an install that ended part way
  makes hdiutil refuse the same file with "Resource busy", while a new
  download of it attaches.
- Windows is unchanged: its update helper removes the download after
  the app exits.

* craft-update: a macOS install that stopped partway no longer blocks the next install

When a macOS install ends before it finishes (the app quits or
crashes), the copy it was making stays beside the app and the update's
disk image stays attached. The next attach of the same file fails with
"Resource busy".

- Each staging folder's name now carries the installing process's id,
  and the folder holds an empty marker file. Before it stages a copy, an
  install removes the marked staging folders of processes that are no
  longer running, and their empty unmarked ones. It also removes a
  staging folder from earlier launcher versions when it holds only the
  app's bundle and has not changed for a day. Backup folders and other
  apps' folders are left alone.
- Before it attaches an image, an install detaches the images that
  earlier installs of the same app left at their mount points in the
  cache, when their process has ended or, in this process, when no
  install uses them any more.
- When the attach fails, the install detaches its image if it is
  attached with no volume mounted and no other attach of the same file
  is running.
- A staging folder that can't be removed, or an image that stays
  attached, is logged, and the error of a failed install says so after
  its reason.

* craft-update: the kept-download test accepts either signature refusal

An unsigned bundle is refused as "isn't signed" today and as a failed signature check once codesign's exit status is read, so the test checks that the install stopped on its signature.

* craft-update: the share check covers nested and pixmaps folders

A standalone copy in a prefix gets Update and Restart only when the
update can write every `share` folder it replaces files in. The check
covered `share/<id>`, `share/applications` and `share/icons/hicolor`
only, but the update writes a copy of each file next to its
destination first. A folder inside `share/<id>` (such as `models`), an
icon folder such as `icons/hicolor/256x256/apps`, or `share/pixmaps`
(where an older flat release puts its icon) that the user can't write
stopped the update after the whole download.

The check now also takes in every folder inside `share/<id>`, each
`icons/hicolor/<size>/apps` folder that holds the app's icon, and
`share/pixmaps`. Icon folders of other packages and other themes are
not checked. A linked folder is checked but not walked. The walk reads
at most 4,096 folder entries, so a large tree does not slow every
start; a folder past that is reported by the install as before.

The same check runs for a standalone copy on Linux, so a Linux copy
whose nested share folder or `share/pixmaps` it can't write now gets
the download link too.

* craft-update: a release folder that can't be read stops a standalone update

A standalone update in a prefix copies every file under the release's
`share` folder. The walk skipped any folder it could not list, and the
archive keeps each folder's mode, so a folder that nobody may list left
its files out while the update reported success. An error reading an
entry of the release's `bin` folder was dropped the same way.

These are now errors that name the folder, and the update stops before
any installed file changes. A release without a `share` folder still
updates.

A release whose `share` or `bin` is a link was followed, so the update
copied files from outside the release into the prefix. A `share` or
`bin` that is a link, or a `share` that is a file, now stops the update.
Links further down under `share` are still skipped.

The unpacked release is removed after a failed update even when it holds
a folder that its owner can't list. Before, such a folder stayed in the
cache.

* craft-update: the share check covers doc, metainfo and mime, and follows links

The share check for a standalone copy in a prefix now also takes in
`share/doc/<id>` and every folder inside it, `share/metainfo` and
`share/mime/packages`. Release archives such as VectorCraft's Linux and
FreeBSD ones put files in all three.

The walk now follows links, as the update's writes do. A copy whose
`share/<id>` is itself a link had none of its folders checked, and a
folder behind a link inside `share/<id>` was not checked either. A
folder the walk reaches twice is read once.

When `doc/<id>`, `metainfo`, `mime/packages` or `pixmaps` is missing,
the check no longer falls back to `share` itself. A copy whose `share`
the user can't write, but whose app, `applications` and `icons/hicolor`
folders exist, gets Update and Restart again, as it did before the
previous commit.

The entry limit of 4,096 now applies to each of `share/<id>`,
`doc/<id>` and `icons/hicolor` separately, and it counts files and
folders. A large app folder no longer turns off the icon folder check.

In a folder with the sticky bit, such as `/tmp`, the user can rename
over a file only when the user owns the file or the folder. A copy now
gets the download link when such a folder holds one of the app's files
that another user owns: the program itself, its other programs
(`<id>-<name>`, such as `<id>-cli`), anything in `share/<id>` or
`share/doc/<id>`, or a `share` file named `<id>.<ext>` or
`<bundle id>.<ext>`.

An icon size that the installed copy doesn't have yet is still not
checked, and a read-only folder that the user added under `share/<id>`
does count. Linux runs the same checks for a standalone copy, and the
sticky check on the program file for an AppImage.

* craft-update: the app-program check has its own name

The check for the app's programs in a bin folder was named is_program, the name of the Windows check for an MZ header, so the crate did not build on Windows. It is now is_app_program.

* craft-update: a missing doc, metainfo or mime folder is checked through share

A Linux release writes files into `share/doc/<id>`, `share/metainfo`
and `share/mime/packages`, and the update creates each of these
folders when it is missing. When one was missing, the share check took
its nearest existing parent inside `share` and never `share` itself. A
copy whose `share` the user can't write and that lacked one of these
folders was offered Update and Restart, and the update failed after
the whole download.

For a missing `doc/<id>`, `metainfo` or `mime/packages`, the check now
takes the nearest existing parent with `share` included, as it already
did for `applications`. Only the old flat release layout writes to
`share/pixmaps`, so a missing `pixmaps` is still checked inside
`share` only, and a copy whose `share` the user can't write keeps
Update and Restart when every other release folder exists.

A folder in the app's folder or in `doc/<id>` that the user can't
read is still checked against its own permissions, as before. The
copy gets Download when the user can't write and search that folder.
The check can't list the folders inside it and does not check them.
A new test keeps this rule, and the check does not refuse a copy only
because a folder in it can't be read.

* craft-update: the staging cleanup's doc says which folders it removes

A marked staging folder is removed once its process has ended,
whichever app that install was for: the folder's name does not record
the app. Only staging folders of earlier launcher versions are checked
for the app's bundle.

* craft-update: a failed attach leaves its image attached when pgrep gives no answer

detach_unmounted, from #766, checked for another attach of the same
image file with is_running, which reads a pgrep that gives no answer as
"not running", and then force-detached the image's unmounted
attachments. It now uses process::running, from #753, and detaches only
when the check answers that nothing is running.
2026-11-11 00:07:01 -05:00
Dylan M. Taylor 081078bb1b craft-heif: optional Apple-style chroma upsampling (#799)
* craft-heif: optional Apple-style chroma upsampling

* craft-heif: the Chroma docs say rows blend too, and the README example compiles
2026-11-10 22:32:39 -05:00
Dylan M. Taylor 009a46a6b5 sonar: analyse the sources, not only the 3 platform test files (#798)
* sonar: analyse the sources, not only the 3 platform test files

* sonar: every crate's tests are tests, as the coverage job treats them
2026-11-10 22:30:00 -05:00
Dylan M. Taylor 006e4df225 Add craft-dxf: DXF tag reader and writer, from CADCraft (#796)
Moved from cadcraft-dxf (cadcraft 2f8f1861) unchanged apart from names and docs:
ASCII and binary DXF as group-code tags, sections and entities, pre-2007
code pages, versions, and writing. VectorCraft's hand-written DXF writer
can move onto it next.

Part of #778, #791.
2026-11-10 20:25:03 -05:00
Dylan M. Taylor 3b3104981b Add craft-tiff: TIFF and BigTIFF IFD reader and writer, from LightCraft (#795)
Moved from lightcraft-tiff (lightcraft b188295e) unchanged apart from names
and docs. The first step of shared RAW support (#783, #587): LightCraft's raw
and metadata crates read through it, and PhotoCraft's and FilmCraft's own
TIFF parsing can follow.

Part of #778.
2026-11-10 20:24:43 -05:00
Dylan M. Taylor 6e21c5b0b1 Add craft-heif: the HEIF/HEIC decoder shared by LightCraft and PhotoCraft (#794)
Moved from lightcraft-heif (lightcraft b188295e), the more complete of the
two copies: libheif-matching YCbCr conversion from the VUI, monochrome,
thumbnails and the box caps. PhotoCraft's grid-tile ICC inheritance tests
are ported to it.

Part of #778. Closes #784 once both apps switch.
2026-11-10 20:24:21 -05:00
Dylan M. Taylor f45f71cb48 jpeg: count the upsampling maps against max_alloc (#540)
render keeps a column map and a row map for each component while it writes
the output, but the check before allocating counted only the coefficients,
the reconstructed planes and the output. The maps are now counted too, at
their real element size.
2026-11-10 18:35:50 -05:00
Juan JuradoandFredApps 1ee01a4c58 craft-print: ask macOS for Auto Select when a job names no tray (#232)
* craft-print: ask macOS for Auto Select when a job names no tray

CUPS gives a job that names no tray the PPD's default tray. On a queue
whose default tray is empty (a Xerox AltaLink defaulting to Tray 1), the
printer holds every job ("Tray 1 is empty") while CUPS reports it
completed, so nothing prints. Preview prints because the macOS Print
dialog's Paper Feed: Auto Select sends AP_D_InputSlot with an empty value,
which drops the PPD's default tray (storytold/pdfcraft#3752).

- lp_args sends `-o AP_D_InputSlot=` on macOS when the job's options name
  no InputSlot. A tray the job names wins; checked with cupsfilter on a
  synthetic PPD: with both, CUPS emits the InputSlot's code.
- apply_queue_defaults (now shared with apps that read options
  themselves, such as PdfCraft) gives the driver's tray option an
  AUTO_SELECT choice, labelled "Auto Select", first and as the default,
  so a dialog shows what lp_args sends. It is left out when the user's
  lpoptions defaults name another tray, since lp adds that tray to every
  job and it wins.

Linux jobs are unchanged: AP_D_InputSlot is Apple's.

* craft-print: match queue defaults to the PPD without regard to case

CUPS treats PPD option and choice names as case-insensitive. A queue
default spelled otherwise than the PPD (inputslot=tray2 for InputSlot
Tray2) was ignored, so on macOS apply_queue_defaults showed Auto Select
although lp would send Tray2. Keys and choices now match case-
insensitively and keep the PPD's spelling. lpoptions -l prints the PPD's
spelling, so this matters for callers passing defaults from elsewhere.

Review: storytold/craft-libs#232 (CodeRabbit).

* README: keep both the macOS tray and the Flatpak notes (the merge left the conflict markers in)

---------

Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 19:15:15 +01:00
el-jandFredApps 41ff22dcd4 craft-media-testkit: shared media test helpers and ffmpeg oracle discovery (#701)
* media-testkit: shared test helpers and ffmpeg oracle discovery

* media-testkit: report fixture rename failure via skip; media: unlink relative craft-video path

* media-testkit: failed fixture generator honours CRAFT_MEDIA_REQUIRE_ORACLES

generate() returned None directly when the generator reported failure, so a
required-oracle run could pass with a missing fixture. Route it through skip,
like run_generator, and cover both policies with tests.

---------

Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 18:56:30 +01:00
FredAppsandFredApps 6768a9098b ci: install the WASM test runner prebuilt, and move the CRW sample cache off Node 20 (#776)
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 16:45:11 +01:00
FredApps 1146cdc4f9 print: grey pages print on a Windows driver with no grey option; ask for the one it lists (#774)
storytold/pdfcraft#5806: Print in grayscale was refused ("the printer's driver did not retain
the requested grayscale; nothing was sent") on drivers that list no grey option the ticket can
name: Microsoft Print to PDF lists only Color, and an HP Photosmart driver has only its own
ns0000:JobHPPrintInGrayScale.

- Job::grey_pages: the caller has already made the pages grey. Grayscale is still asked of the
  driver, but a driver that has no grey option, or doesn't keep it, prints the job instead of
  refusing it. Off (the default), such a driver is refused as before, since the pages would
  come out in colour.
- The ticket asks for psk:Grayscale when the driver lists that and not psk:Monochrome
  (grey_option); Windows dropped the Monochrome it didn't list and colour came back.
- Offered::grey records whether PageOutputColor has a grey option at all.
2026-11-10 15:16:57 +01:00
FredApps 3edc157a2c print: the CUPS printer list waits at most 15 s for lpstat, 10 s for discovery (#773)
printers() ran `lpstat -l -p -d` and `lpstat -e` with no time limit. When the CUPS scheduler
stops answering (macOS cupsd stopped serving its local socket in storytold/pdfcraft#6761), both
block forever and the Print dialog looks for printers until the app is closed.

Both now run through output_within (the bounded wait pdfcraft's lpoptions read already uses):
the queues get 15 s, network discovery 10 s. A scheduler that doesn't answer gives an empty list
and a warning naming cupsd; discovery that hangs only loses the driverless printers.
2026-11-10 15:16:51 +01:00
FredAppsandFredApps 02ef0c597a ci: save the dependency cache from main only, and cancel superseded PR runs (#772)
* ci: save the dependency cache from main only, and cancel superseded PR runs

* ci: give runs outside pull requests their own concurrency group

---------

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 15:06:39 +01:00
FredAppsandFredApps dc53e98eb1 fix(ui): keep rail scrolling above the footer in short windows (#493)
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 13:57:48 +01:00
ruchirguitar bb8320f5fa ui: resize iconic flyouts from their free edges (#627)
* ui: resize iconic flyouts from their free edges (#5713)

* ui: persist independent flyout widths per group (#5713)

* ui: keep constrained flyouts resizable (#5713)

* ui: document flyout helpers for storytold/vectorcraft#5713

* ui: honor host header minimums for storytold/vectorcraft#5713

* ui: keep constrained frame axes resizable for storytold/vectorcraft#5713
2026-11-10 12:23:09 +01:00
FredApps 0e4943cb5e print: winprint 3c978dfc: pages at the job's resolution, QName ticket values, padded integers (#761)
winprint.rs#19 draws each page at the resolution the job's ticket asks for instead of ~96 dpi
(storytold/pdfcraft#4941, #2676). #12 writes QName property values as prefix:local, which
drivers such as the Microsoft IPP Class Driver otherwise reject when merging the ticket
(storytold/pdfcraft#3823). #3 accepts whitespace around xsd:integer values in
PrintCapabilities (Toshiba Universal Printer 2, storytold/pdfcraft#2430).

The public API craft-print uses is unchanged; only the rev and the lockfile move.
2026-11-10 01:55:59 +01:00
art804andFredApps b6b8a55b2c craft-print: two_sided(printer) reads one printer's two-sided support (#133)
printer_queues() (#618) lists the queues without asking any driver, so
every printer's two_sided is unknown there. A Print dialog listing from it
still needs to know whether the printer the user picks takes two-sided
settings, without asking every other driver as printers() does: Windows
takes about 48 s to give up on a printer it can't reach, such as a WSD
printer on another network (storytold/pdfcraft#2192).

two_sided(printer) finds that queue and reads its driver's capabilities
with the same check printers() makes, so the answers match. It is None
when there is no such queue, when the capabilities can't be read, and off
Windows, as printers() reports there.

The paused-printer test now checks that two_sided(name) gives printers()'s
answer for the test queue from exactly one driver query, and a new test
checks that a name no queue has is unknown. The query counting moves into
a small helper, which keeps windows_jobs_reach_the_spooler under the
cognitive-complexity limit.

Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 00:03:35 +01:00