* 3D: Orientation and X/Y/Z Rotation turn Z first, keyed Orientation slerps, as After Effects
After Effects composes a 3D layer's (and a camera's) Orientation and its X/Y/Z
Rotation as Rx * Ry * Rz each, Orientation before the axis rotations; EffectCraft
used Rz * Ry * Rx. Measured on synthetic AE 2026 cases: Orientation [10,20,30]
(488) was 5.2 % of the frame off and now matches to the anti-aliased edge (0.03 %);
X/Y/Z Rotation 10/20/30 (800), Orientation + X Rotation (801) and one-node
cameras (804, 805) likewise.
Keyed Orientation moves along the shortest arc between the keys (quaternion
slerp, eased in time, angles written in [0, 360)): keys [0,0,0] -> [10,20,30]
read back [6.2643, 9.3531, 15.2190] half-way where EffectCraft gave [5,10,15];
[0,0,0] -> [60,80,40] linear and eased match AE's read-backs within 0.002 deg.
lookAt() returns AE's angles (z axis on the target, x axis level; two AE
read-backs within 1e-3 deg). The camera tracker, VR editor, Look At and orbit
commands use the new decomposition.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keyframes: a linear side next to an eased side has 16.67 % influence, as After Effects
In a segment that is Bezier on one side and linear on the other, the linear
side keeps the segment's average speed, but with the 16.67 % influence After
Effects shows for linear keys in the Keyframe Velocity dialog (and reports to
scripts), not 33.3 %. Measured on an After Effects 2026 time-remap render (a
square at x = 50 + 455 * remap): out of a key at 2 s (3.5, speed 0, influence
50 %) into a linear key at 4 s (0.5), AE plays 3.290 at 2.5 s and 2.657 at
3 s; EffectCraft played 3.308 and 2.629 (8 and 13 px off), now the same
square positions to the pixel.
LINEAR_SIDE_INFLUENCE is used by evaluation (scalar, per-dimension, paths and
spatial), side_ease (Graph Editor, velocity dialog, keyIn/OutTemporalEase) and
the Lottie export, so exported easing plays as EffectCraft does. Segments
linear on both sides stay straight lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Timewarp: retime the layer's source without the effects above it, as After Effects
After Effects' Timewarp reads the layer's source (with its masks) at the
retimed times and ignores the effects applied before it: a red solid with a
Gradient Ramp and then Timewarp (Speed 25) renders as the red solid in AE
2026 (EXPORTER e2 read-back). EffectCraft retimed the layer with its
effects (mean difference 127.5/255 over the comp), now identical (0, CPU and
GPU). Echo, Posterize Time and the other time effects keep reading the
effects before them, as measured before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Image sequences: a gap in the numbering holds the previous frame, as After Effects
After Effects 2026 shows the last existing file across a gap in a sequence's
numbering (case 860: files 1, 2, 4, 5 show frame 2 at frame 3; VERIFY
read-back). EffectCraft drew colour bars there. The media pool now holds the
previous file (Footage::sequence_file_held); colour bars remain only where no
file precedes the frame. Missing-frame reports on import are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review: keyed Orientation uses LINEAR_SIDE_INFLUENCE for linear sides; real NaN assert
evaluate_orientation still gave a linear side (and the unreachable missing-key fallback)
1/3 influence while segment_value, side_ease and the Lottie export use
LINEAR_SIDE_INFLUENCE (1/6), so a keyed Orientation next to an eased key moved on a
different curve than every other property (CodeRabbit on #1876). New test:
orientation_linear_side_next_to_an_eased_side_matches_a_scalar (both directions).
The always-true NaN assert in keyed_orientation_slerps_as_after_effects now checks that
a non-finite key comes back as given; Property::value_at says why the match_id
"orientation" routing is safe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Format keyframe orientation test (cargo fmt)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Move the image-sequence gap change out of this branch
The gap change (hold the previous file instead of colour bars) reverses
#297's documented behaviour and goes to its own PR, as requested in review.
pool.rs, sequence.rs, the footage test and docs/footage.md / docs/agents.md
are back to main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
* Subspace Warp: QL eigen solver and banded mesh solve (#2319)
plan() spent 43-96 % of its time in the cyclic Jacobi sym_eigen, whose
absolute stop test never triggers on Gram matrices of magnitude ~1e8, and
then ran a dense Cholesky on a mesh normal matrix of bandwidth ~30.
Use Householder tridiagonalisation plus implicit QL inside subspace.rs
(only the leading subspace is used, so basis and sign do not matter), and
accumulate and factor the mesh normal equations in lower-band storage,
without a Vec per data row. camtrack/linalg.rs is untouched.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Subspace Warp: replace the QL eigen solver with a clean-room symmetric QR
The previous sym_eigen_ql followed a well-known published tred2/tqli listing
too closely for the clean-room rules. It is replaced by sym_eigen_qr, written
from the textbook method (Golub & Van Loan, Matrix Computations, ch. 8) in
its own structure:
- householder_tridiagonal: explicit reflectors with the symmetric rank-2
update B -= v w^T + w v^T (p = beta B v, w = p - (beta p^T v / 2) v), Q
accumulated as rows of Q^T.
- wilkinson_qr_step: one implicit QR step with the Wilkinson shift, Givens
rotations chasing the bulge from the top of the unreduced block down.
- sym_eigen_qr: deflation of negligible off-diagonal entries and the step
driver, split so each function stays under the complexity limit.
It returns None for non-finite input, a size mismatch or no convergence
within 30 n steps; smooth_trajectories then falls back to the Jacobi
sym_eigen, so it can never loop for ever or be worse than before.
Tests: random symmetric matrices (n = 1..61) against Jacobi eigenvalues,
residual |A v - lambda v| and V^T V = I; identity, repeated 3x3 blocks, zero,
1x1, 0x0, rows scaled from 1e-8 to 1e8 and a 1-2-1 tridiagonal; NaN, infinity
and size mismatch return None; the leading-subspace projector of Gram-like
low-rank matrices matches Jacobi. An ignored benchmark times plan() on a
300-frame synthetic parallax shot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
* Roto Brush: propagate on the band bounding box
propagate() ran the signed distance, gauss, graph cut, model fit and
clean over the whole frame. Work on the crop of the band bounding box
(|sd| <= r plus the core, padded) and paste the labels back; fall back
to the full frame when the crop is not smaller. warp_prior stays
full-frame.
Fixes#2323
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Roto Brush: band crop padded by the prior blur, models fitted on the full frame
Review follow-up for the band bounding-box crop in propagate:
- The crop pad is now the reach of the prior blur (three box passes of
matting::gauss, exposed as matting::gauss_reach) instead of 1 px, so every
band pixel sees the same blur window as on the full frame.
- When the previous frame has no colour models they are fitted on the full
frame before cropping, as the full-frame path does.
- The free pixels and the clean seeds are cut out of the full-frame signed
distance instead of being recomputed on the crop; the seeds equal
core_of(prior.soft), which saves one signed distance per frame.
- The full-frame path and the crop share recut_band/cut_band; full frame is
just the whole rectangle. A frame with no band and no core (no
foreground) skips the cut, which was a no-op.
- The w - 1 / h - 1 underflow on zero-sized frames is gone (band_rect
returns None); regression test for 4x0, 0x4 and 0x0 (panicked before in
debug builds).
Tests: crop vs full frame labels on a disk, a disk at the frame corner, a
donut, r = 2 and r = 3 with prior edges wider than the band, and no
foreground, each with fitted, given and equal colour models; the padded crop
blurs the prior like the full frame at every band pixel while a 1 px pad
does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
* Roto Brush: reuse scratch vectors in the distance transform
matting::distance allocated four vectors per column plus a staging copy
and three vectors per row. Run the columns and rows through four reused
scratch vectors instead; the dt1 arithmetic is unchanged.
Fixes#2324
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Roto Brush: keep the distance transform parallel, with per-split scratch
The previous commit made the column and row passes serial, which was measured
on one thread only. Both passes are parallel again: the column pass runs on a
transposed buffer so every column is a contiguous chunk, and rayon's
for_each_init gives each work split one set of scratch vectors instead of
four allocations per line. The square root is folded into the row pass.
A test keeps the old per-line implementation and checks that the output is
bit-identical on a disk, its inverse, 1xN, Nx1, all-true, all-false, seeds on
the frame edge and a sparse 333x211 pattern. An ignored benchmark prints the
1080p median and the rayon thread count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Roto matting: size the distance-transform scratch per pass (columns h, rows w)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Strokes carry an object (0 for older projects, nothing written for 0). The
tracker follows every object in one pass and the mattes are united; the
confidence of a frame is the least sure object's. Object picker in the Layer
panel's bar; roto.options/stroke/clearStrokes/status take or report objects.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
--clip FRAMES_DIR --truth MASKS_DIR runs the trackers on image sequences with true
mattes (strokes made from the truth; nothing is bundled or copied), tested end to end.
Trackers report the most memory their banks held (peakMB column).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Simulated user paints a foreground stroke through the true object and a background
stroke on the biggest false positive on the first frame under IoU 0.9, up to 6
rounds. SAM 2 needs none on every clip; classical and MobileSAM need more than 6 on
occlusion and re-entry. All trackers now take strokes on any frame.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TrackRequest takes a list of objects (each with its prompts on its frames) and
the tracker reports (object, frame, result); frame encodings are shared, objects
may start on different frames. VideoTracker::propagate_objects runs a subset.
Limits: 32 objects. Test: two discs, one prompted on frame 0 and one on frame 2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
On real footage the stroke frame came out partial (clicks make SAM 2 pick a
part of the object). Measured on a multi-part figure: clicks only IoU 0.41,
with a box around the figure IoU 1.00. Chain::track now adds the box of the
classic engine's matte of the strokes to each prompt; a box stroke wins.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
run_tracker follows the object with the model's video tracker on the same
synthetic clips (--sam2 FILE or the installed weights). 12 frames, dev build:
re-entry clip mean IoU 0.632 classical vs 0.975 SAM 2 (0.000 vs 0.967 after
the object returns); the other clips 0.98 vs 0.99.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
When the chosen model has a tracker, roto.propagate runs one tracking pass over
the span: prompts from the strokes (a handful of points, which SAM 2 trusts),
the clip tracked both ways, each frame's probability map refined by the same
graph cut with the strokes as hard constraints. Results are cached under their
own keys with a per-frame confidence and an occluded flag; roto.status reports
them (source: tracker|classical). Without a tracker nothing changes.
Real weights: IoU 0.98 on all 21 frames, confidence 0.99.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Synthetic Roto Brush clips with known mattes (movement, lighting change,
occlusion, leaving and re-entering) and a runner that compares the classical
engine with MobileSAM when installed. Needs no weights; SAM 2 plugs in later.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflict in crates/track/src/camtrack/bundle.rs: kept both the branch's
PtSys/preconditioner/iterative_schur block and upstream's new
add_focal_prior helper. Adapted to upstream's explicit linalg imports
(conjugate_gradient, and I3/log_rot/mt/norm in the tests) and moved the
explicit point elimination into reduce_points so adjust_impl stays under the
cognitive-complexity lint.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflict in crates/track/src/camtrack/bundle.rs (imports): upstream replaced
the linalg glob import with an explicit list; kept that list and added the
branch's SolveError import (use super::{Distortion, SolveError}). Input
validation, size budget and Result-returning adjust/cost are unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflict in crates/track/src/face.rs: upstream moved the feature-blob search
into FaceTracker::feature_blobs. Re-applied the branch there: the per-pixel
point-in-polygon loop is replaced by non_skin_inside (one crossing pass per
window row) and the roll rotation is hoisted with face_mapper.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Dropping the points a later stroke of the other kind painted over could leave
a stroke with none, even though most of it was still painted: a small Alt tap
in the middle of a large foreground dot removed its only point, and the
selected model was skipped for the classical cut. Points are now kept by the
resolved ownership under them, and a stroke that loses every sampled point
but still owns pixels is prompted from the one deepest inside them. A stroke
painted over completely still prompts nothing.
A trained model is no longer prompted with both labels at one spot: points
along a stroke that a later stroke of the other kind painted over are
dropped, and the propagated matte's points, box and mask take this frame's
strokes over it.
A Refine Edge stroke over an earlier Alt (erase) stroke now restores the band
there; the band carried from the previous frame stays under this frame's
strokes.
A green stroke over an earlier Alt (red) stroke now restores the area: where
strokes overlap, the one painted last sets the hard constraint, for the cut,
the colour samples and the cleanup seeds alike.
Upstream split lib.rs files into modules and moved engine commands/ to cmd/; this branch's change merges without textual conflicts.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The fine stage searched a 5x5 neighbourhood per pyramid level and seed.
When the best coarse match is reliable (NCC score >= 0.8) and lies within
one top-level pixel of the motion prediction, a 3x3 neighbourhood is
enough; otherwise the 5x5 search stays.
Fixes#2349
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reuse consecutive caller-owned pyramids for regular Warp motion and trajectory tracking. Preserve the original three-level motion search by setting aside the trajectory-only extra level, without cloning planes. Keep detailed analysis at its existing distinct resolutions and test bitwise motion and complete JSON equality against separate pyramids.
Fixes#2322
Solve expensive camera systems with matrix-free Schur products and a block-preconditioned iterative solver. Check the true linear residual and fall back to the direct solve when convergence fails. Keep short systems on the existing path and compare reprojections, camera parameters and outlier decisions against direct solves.
Fixes#2328
Share the unchanged Jacobi factors between full SVD and null-vector callers. Select the same right singular vector directly instead of normalizing U and sorting copies that the caller discards.
Fixes#2329
Detect narrow lower-triangle bandwidth and skip structural zeros while preserving dot-product lanes and signed-zero arithmetic. Keep a dense fallback for wide systems and compare factors and solutions against the original algorithm.
Fixes#2325
Return errors for invalid bundle observations and oversized workspaces, reject non-finite projections and mismatched correspondence slices, and prevent track frame overflow. Cover the reported failures with synthetic regression tests.
Fixes#2330
Use a bounded inline array for the current camera parameters and preserve the existing accumulation order. Extra parameters spill safely if the parameter set grows.
Fixes#2327
The normalized cross-correlation only had a fast path for an exactly
identity shape, which tracked footage never has after the first frame, so
every candidate went through Plane::sample. All candidates of a search are
the reference centre plus integer offsets, so each template pixel has the
same fractional offsets for all of them. Build one table of taps per
(level, seed, shape) and score each candidate with integer-indexed lerps,
with one bounding-box check up front (outside the plane the old path is
used).
lk_refine only reads the level-0 gradients around the match, so compute
them for that window (with an interior fast path in Plane::gradients_in)
and fall back to the full gradients if the iteration leaves it.
The results are unchanged: same lerp form and accumulation order.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
step() cloned the whole RGBA frame to remember it (33 MB at 1080p,
133 MB at 4K) and then cropped that clone, although only the region
around the outline is ever read. The tracker now stores the previous
frame already cropped to the region the next step will use, computed
from the updated outline with the same margin and rounding as before,
so the crops, and therefore the tracking result, are unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Store the two four-element DLT rows in arrays while preserving arithmetic and accumulation order. This removes two heap allocations for every triangulated view.
Fixes#2326
The feature mask tested every non-skin window pixel against the 48-edge
face outline (about 25 M edge tests per frame). A scanline hits the same
edges for every pixel of a row, so the crossings are now computed once
per row with the same expression and comparisons as inside_polygon, and
a pixel's parity follows from how many lie to its right. The face-frame
rotation is also worked out once per frame instead of once per blob
pixel.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Every pixel of a (2r+1)^2 window shares one fractional offset per level
and iteration, but lk_point called Plane::sample (with a libm floor, a
bounds test and the clamp path) up to three times per pixel. The offset
and the clamped column/row index tables are now computed once per
window and the pixels run the same lerps on integer indices.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The nightly FreeBSD run measured 119 ms/frame against a 100 ms limit: the mean included
Tracker::new and any scheduling stalls on the two-core VM. The best single step with a
250 ms bound still catches an order-of-magnitude regression.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
