* SAM 2 mask decoder: transposed convolutions as matrix product plus pixel shuffle (#3031)
Both transposed convolutions of the mask decoder have a 2x2 kernel, stride 2
and no padding, so each input pixel owns its own 2x2 output block. That is one
matrix product plus a pixel shuffle, which is about twice as fast on the CPU
as candle's conv_transpose2d. The first convolution also takes the decoder's
key tokens directly, without the transpose and copy to an image.
The candle path stays in the code for the tests. They compare the two to
1e-6 (relative to the largest output value) on the decoder's real sizes,
odd and degenerate sizes, large values, zeros and a single hot pixel, check
that other kernels are refused, and compare a whole decoder forward pass
(single and multi mask) on random weights.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* SAM 2 decoder test: bound the weight discovery loop
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Candle's gelu_erf runs on one thread and was a third of the Hiera block time.
A rayon CustomOp1 calls the same erf per element (bit-identical, tested). All
reference tests unchanged. op_bench example times the block's operations.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Encode 5.0 s -> 3.4 s per 1024x1024 frame in release on 16 threads (fused
softmax, queries scaled before the matmul, flat 2-D linear layers); a click
on an encoded frame is about 120 ms. Reference tests unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A click on the synthetic image: mask logits within 1.1e-4 (range 11.6), IoU
head 1e-6, object score 2e-5, prompt embeddings 1e-7.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
