Every source keeps up to 384 MiB of decoded frames plus its live decoders
for the whole session, and the 1 GiB process budget only trims caches that
were idle while another cache asks for frames. The 8/16-bit plane shelves
(192 MiB each) were never trimmed. A long-lived process (MCP) therefore
held 2-3 GiB after an export and ran out of memory on the second one.
Add filmcraft_codecs::release_idle(), which empties every cache nobody is
decoding on (try-locked, so users are untouched), and
filmcraft_frame::pool::trim_planes(). A standalone export calls both next
to trim_f32 when its pipeline drops; parts of a batch keep them.
On the CPU compositor every monitor frame was converted from Y'CbCr to a
premultiplied linear f32 image (16 bytes per pixel) and from that to sRGB
bytes, even when the frame is a single opaque clip shown as it is. The canvas
now keeps such a frame unconverted until something needs its pixels (another
layer, a transition, an adjustment layer, a display transform, captions).
render_sequence_rgba8, which the monitor's CPU path uses, converts a frame
that stays that picture alone directly to the bytes the float path gives,
through per-code-pair tables for R and B and one table for G, at full size and
decimated. Tests compare it bit for bit with the float path for every matrix,
range, chroma layout and decimation, and through the compositor's shortcut
scenes.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Move the owned Cb and Cr vectors returned by rotate_chroma into the output frame in both Yuv8 and Yuv16 paths, avoiding two redundant allocations and copies per rotated frame.
Validation: contributor reports all tests passed. No tests or compilation were run by the agent.
Co-authored-by: Codex <noreply@openai.com>
- The H.264 encoder's RGBA -> 4:2:0 conversion runs on the render
threads (VideoEncoder::prepare_ahead), which are idle while a batch is
encoded; encode() picks the planes up by picture index and converts a
picture itself when there are none (then on the encoder's own pool,
not the busy global one).
- rgba_to_yuv420_8 uses per-code tables of the same products, rounds
without roundf and works row by row so the divisions vectorise; the
output is pinned against the float formula for all 2^24 colours.
- Full-size opaque 8-bit Y'CbCr -> linear conversion tabulates R' and
B' per code pair with the expressions of the per-pixel loop.
- over_black_rgba8 fetches the sRGB table once per picture.
bench_export (1080p H.264 + AAC to H.264 + AAC, Ryzen 16 threads, 7
interleaved rounds) with the encoder change: median 7.86 s -> 6.14 s
(-21.9 %), min 7.78 s -> 6.02 s (-22.7 %). Same file (MD5) as
before, and the same for a ProRes export of the clip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
convert_region normalised every 10- / 12-bit sample with two divisions.
It builds a per-code table for the luma and chroma normalisation instead
(exactly the values normalize_y / normalize_c return); a code past the
table (a corrupt plane) is still computed.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The one-to-one path cleared a whole output row when a plane held only
part of it. A row now takes the samples the plane holds; only the pixels
without them come out transparent (review feedback). The short-plane
test gains a row the luma plane ends in the middle of.
At full size (n == 1: export, full-size playback) the general box-filter
loop still clamped and shifted its way to every sample, indexed the three
planes with bounds checks and ran one-step dy / dx loops per pixel. The
one-to-one case now takes each output row's Y, Cb, Cr (and alpha) rows
once with get() and runs the same arithmetic, in the same order, per
pixel: every value is the same, bit for bit.
frame_bench, single thread, camera frame -> linear f32 image: median
20.7 -> 14.9 ms; render checksums unchanged.
A frame whose planes are shorter than its size panicked here (index out
of bounds); at full size its rows without data now come out transparent,
as the float path's already did.
The MP4 source reported the sample entry's size, only swapped for the display rotation, so two
crops were lost: the codec's own (H.264 SPS frame cropping, HEVC conformance window), which the
decoder applies but the reported size and sequences made from the clip didn't, and the clean
aperture (`clap`), parsed but never used. An iPhone clip (1920x1440 coded, 1744x1308 clean
aperture, -90 degree display matrix) showed 1440x1920 instead of 1308x1744.
H.264 / HEVC sizes now come from the stream's parameter sets; the clean aperture is a rectangle
inside that picture (ignored when it makes no sense), reported as the size and cut from every
frame by the GOP cache before the display rotation, through the new VideoFrame::cropped.
Tests: bit-exact against ffmpeg (which applies both crops) for an H.264 SPS crop, an HEVC
conformance window, centred and off-centre clean apertures and a clean aperture with a rotation;
clean apertures that make no sense are ignored without a panic; VideoFrame::cropped on every pixel
format with odd sizes and misaligned corners.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pool::trim() from Pipeline::drop flushed the whole process-wide pool after every
render-preview segment and proxy, and threw away playback's U8/U16 planes too.
Now Pipeline::drop calls pool::trim_f32() only when !settings.part_of_batch.
PoolStats gains f32_idle_bytes so the tests can check the float shelf alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve inherited physical key collisions when loading built-in presets,
while preserving established explicit winners and deliberate custom conflicts.
Read every ffprobe display-frame timestamp and register the requested credit.
Verify exact frame-plane reuse without depending on concurrent global metrics.
Co-authored-by: Codex <codex@openai.com>
The export pipeline hands the float image of every frame it converts back to
`filmcraft_frame::pool` (#184), which keeps up to 320 MiB of them, and the decoders' plane shelves
hold up to 192 MiB of each type. An export's images are the size of its frames, which nothing else
asks for, so a finished export left them idle until the next one.
`pool::trim()` frees every idle buffer. `Pipeline` calls it when it is dropped, one place for every
way an export ends (done, failed, cancelled, dropped half way by a host that steps it, as the web
build does) and for every caller of `export` / `Exporter` (previews, proxies and AAF/OMF mixdowns
share the pipeline). A job still rendering at that moment allocates a few images again: the pool is
only a cache.
Measured with a synthetic 3 s 1080p24 matte exported as PNG frames three times in one process
(Apple M1, `footprint`, 10 s after the last export): 575 MB without the trim, the pool holding
332 MB idle, and 258 MB with it. macOS returns the freed pages over a few seconds, not at once.
Tests: `crates/frame/tests/pool_trim.rs` (every shelf emptied, the pool works afterwards) and
`crates/export/tests/pool_trim.rs` (an image sequence, a finished and a cancelled video, an export
that could not start and a stepped export dropped half way each leave nothing idle; without the
trim they fail).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The CPU compositor, which is what export runs, works on premultiplied linear f32 images,
33 MB each at 1080p, and every layer was one of them. A profile of a 9:29 export with a
hardware H.264 encoder (so the encoder is no longer the bottleneck) put 61-64 % of the busy
CPU in compositing, 12 % in zeroing and copying memory, 8-11 % each in the float -> 8-bit
and RGBA -> Y'CbCr conversions and 4-7 % in decoding the ProRes overlays: a banner that is
transparent except for its bottom 100 rows still cost a whole conversion and a whole mix,
and the bottom layer paid for a zero-filled canvas and a pass that mixes it over that canvas.
Three changes, each of which gives the bits the general path gives:
- The canvas is allocated when a layer needs it, and an opaque first layer (a Y'CbCr frame
without an alpha plane, opacity 100 %, Normal blend, the size of the output) becomes the
canvas instead of being mixed over zeros.
- A clip of plain media whose frame has an alpha plane (a ProRes 4444 banner) is converted
and mixed only inside the rectangle where its alpha is not zero: `VideoFrame::alpha_region`,
`to_linear_f32_region` and `blend::composite_at` (`composite` is `composite_at` at the
origin; Dissolve's pattern follows the pixel's place in the output). A clip with effects,
opacity masks, frame blending, colour management or a picture that is not placed one to
one on the output takes the general path, as before.
- `filmcraft_frame::pool` keeps idle f32 images (320 MiB at most). `take_f32_overwritten`
hands one back with its old contents, which the conversions overwrite whole (so no
zero-fill), and the export pipeline gives each frame's image back after the 8-bit
conversion.
Apple M1 (8 cores), a machine that was not idle, so the two builds alternate and the
median is reported:
- Synthetic 1080p frame (`cargo run --release -p filmcraft-render --example frame_bench`,
5 rounds): camera only 8.9 -> 6.5 ms, camera + banner 16.2 -> 7.8 ms, same picture checksum.
- A real project (1080p25 camera clip, ProRes 4444 banner and subscribe bar), 30 s with the
banner exported with the built-in encoder, which is deterministic: 46.4 -> 36.7 s, and
both exports are the same file (same MD5), so every frame of the real pipeline matches.
- The same project with a hardware encoder, 3 rounds: 30 s with the banner 17.4 -> 11.2 s
(CPU 99 -> 56 s), 30 s of the camera alone 11.4 -> 8.8 s, 20 s with the subscribe bar
11.1 -> 7.0 s.
Tests: `render/src/region_tests.rs` compares each shortcut with the general path bit for bit
(an opaque bottom layer, banners over and under a camera, an 8-bit sticker, every blend mode,
several opacities, moved, scaled and effect clips, a bigger source, reduced-resolution
rendering), each render twice so the second starts from recycled buffers, and checks which
layers take a shortcut; the frame tests check that a region is the crop of the whole
conversion, that `alpha_region` is the box around what is not transparent, and that a buffer
full of NaN never leaks into a conversion, for every pixel format. Breaking a shortcut on
purpose (a region one pixel off, an opaque layer taken as the canvas at half opacity, a
conversion that skips the alpha channel) makes them fail.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
With MEM1-MEM3 twelve clips on a timeline still took 5.1 GB, only 2.5 GB of
it live: each decoded picture allocated three planes, each eviction freed
them, and the system allocator kept the freed pages.
filmcraft_frame::pool keeps the planes of frames the GOP caches evict (up to
192 MiB per sample type; a frame something else still holds is left alone).
The H.264 decoder, through a plane allocator hook that keeps the crate free
of dependencies, and the VideoToolbox path take their output planes from it.
perf.stats reports decode.planePoolMB and decode.planesReused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aac, bitstream, cfb, color, format, frame, geom, isobmff, media, scopes, speech, the desktop
app and the web app have no production unwrap/expect/panic/unreachable (the web app's two
`expect`s — frame-cache eviction and chunked file reads — become `let … else`); each now
carries `#![deny(clippy::unwrap_used, …)]` outside tests so it stays that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rust/clippy 1.99 added `chunks_exact_to_as_chunks` and tightened
`needless_late_init` and `drain_collect`, failing `clippy -D warnings`
on main.
- `chunks_exact(N)` / `chunks_exact_mut(N)` with a constant N become
`as_chunks::<N>()` / `as_chunks_mut::<N>()` (fixed-size arrays, so
several byte reads drop their per-element indexing). relink's
fingerprint uses the `(chunks, remainder)` split; its hash output is
unchanged.
- Late-initialised `let`s in ac3 bit allocation and opus CELT
bands/rate become `let x = if/match …`.
- `dpb.drain(..).collect()` becomes `std::mem::take(&mut dpb)`.
No behaviour change; `cargo xtask ci` passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phones record portrait video as landscape pictures plus a rotation in the
container: the MP4/MOV tkhd matrix (ISO/IEC 14496-12 §8.3.2) or the
Matroska Projection pose roll (RFC 9559 §5.1.4.1.28.46). The matrix was
parsed but never used, so every portrait iPhone clip imported sideways
at its landscape size.
- isobmff: Track::display_rotation() reads 0/90/180/270° from the matrix
signs (mirrored / sheared matrices are left alone).
- matroska: parse Projection (type, yaw, pitch, roll);
VideoInfo::display_rotation() for rectangular projections.
- frame: VideoFrame::rotated() turns every plane at its own resolution
(4:2:2 chroma is widened to 4:4:4 for quarter turns) and swaps the PAR.
- codecs: Mp4Source / MkvSource report the displayed size and PAR, and
GopCache rotates decoded frames before caching them.
- web: the WebCodecs source configures the decoder with the coded size
and rotates its output the same way.
Tests: a one-frame ProRes MOV built in the test with 0/90/180/270°
matrices (reported size + which half ends up on top), plane rotation for
RGBA / 4:2:0 / 4:2:2, and a synthetic Matroska file with a -90° roll.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Settings > Playback > Draft decoding (off by default): the Program monitor
asks for draft frames only while playing at 1/2 resolution or lower
(FrameKey::draft, filmcraft_media::cancel::with_draft). The GOP cache runs
the decoder in draft mode for those requests and serves draft frames to draft
requests only, so paused frames, renders and exports always get exact
pictures (codecs test: exports after draft playback match ffmpeg). Draft
plans hand the GPU box-decimated YUV planes at the drawn size. perf.stats
reports draft frames, H.264 threads and the preference; the bench gains
2160p quarter and draft playback cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- model: SequenceSettings::color (ColorPipeline: working space, wide gamut, Auto Tone Map Media;
serde default Rec. 709) and Interpretation::color_space (override; None = metadata)
- render::colorman: source colour space from the override or VUI/colr/MKV metadata, decode tables
(camera log with super-whites, PQ, HLG + OOTF), gamut matrix, tone mapping into SDR working
spaces, gamut mapping; working → SDR monitor transform at the top level
(RenderOptions::working_output keeps working values for HDR export and scopes). Plain Rec. 709
media in a Rec. 709 sequence keeps the old fast path and stays on the GPU; managed layers and
HDR/wide-gamut sequences composite on the CPU
- export: H.264 and ProRes of a PQ/HLG sequence encode BT.2020 PQ/HLG and signal it: VUI / ProRes
header, colr (nclx/nclc), and for PQ mdcv + clli boxes and H.264 SEI (payloadTypes 137/144);
`sdr: true` (and render previews) export tone-mapped SDR. isobmff reads/writes mdcv and clli
- commands: sequence.colorSettings (Sequence ▸ Color Management…), clip.interpretFootage
(Clip ▸ Modify ▸ Interpret Footage…, Project panel context menu), color.spaces,
media.colorInfo; dialogs with automation ids colorDialog.*
- scopes: HDR sequences show a cd/m² waveform on a PQ scale (203 marked) and a BT.2020
vectorscope from the working-space render (scopes.hdrWaveform)
- display-referred wide-gamut input is desaturated into gamut keeping in-gamut colours exactly;
the soft compression is reserved for camera gamuts
- tests: colorman unit tests, engine (interpret/undo, PQ working space values and display),
goldens log_to_rec709 and hdr_tone_map (own renders + sidecars), export PQ/HLG × ProRes/H.264
read back by our demuxer and ffprobe (color_transfer/primaries/space, mastering display and
content light side data), headless UI (dialogs, HDR scopes)
- docs: architecture §5.2 (pipeline, HDR export, display colour management: no macOS EDR yet)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aiongg