Files
effectcraft/crates/plugin/src/lib.rs
T
just_anshhandjustanshh 544eebf904 WebAssembly ABI 2: neighbouring frames for effect plug-ins (#2773) (#3329)
* WebAssembly ABI 2: neighbouring frames for effect plug-ins (#2773)

A wasm plug-in can export ec_api_version 2 and ec_render2 to receive the layer's previous and next frames, their layer times and validity flags. Neighbours come from PluginHost::input_at, are resampled onto the current grid, and are zeroed when absent. ABI 1 modules (ec_render) load and render unchanged; the manifest api stays independent. A layer with an ABI 2 plug-in keys the render cache on layer time; the Plug-in Library accepts ABI 1-2 releases. Docs, help text and tests updated.

* Fix ABI 2 neighbour times on NTSC frame boundaries

	ime * fps is often a few ulps off an integer at rates such as 24000/1001. The old boundary test then took the bracket branch and passed the current frame back as prev (or
ext). Treat a near-integer as a frame boundary within 1e-6 frames, and test 2400 frames at 24000/1001: every boundary must get the frames either side. Also document the loader helpers for the docstring coverage check.

---------

Co-authored-by: justanshh <294019246+justanshh@users.noreply.github.com>
2026-11-09 10:49:53 +01:00

120 lines
6.3 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! # effectcraft-plugin (L3)
//!
//! Loads **WebAssembly effect plug-ins** (WebAssembly ABI v1 and v2, see `docs/plugins.md`; the
//! native plug-in API 2 additions such as layer parameters and host services are not part of it)
//! into the effect registry ([`effectcraft_effects::plugin`]). A plug-in is a `.wasm` (or
//! `.wat`) module with no imports that exports:
//!
//! | export | signature | |
//! |---|---|---|
//! | `memory` | memory | linear memory shared with the host |
//! | `ec_api_version` | `() -> i32` | 1 for ABI 1, 2 for ABI 2 (neighbouring frames) |
//! | `ec_manifest_ptr`, `ec_manifest_len` | `() -> i32` | a UTF-8 JSON [`PluginManifest`] in memory |
//! | `ec_alloc` | `(bytes: i32) -> i32` | a buffer of at least `bytes` bytes, 8-byte aligned; it may reuse the previous one |
//! | `ec_render` (ABI 1) | `(pixels: i32, width: i32, height: i32, params: i32, nparams: i32, time: f64, scale: f64) -> i32` | process the pixels in place; 0 = ok |
//! | `ec_render2` (ABI 2) | `ec_render`'s arguments, then `prev: i32, prev_time: f64, next: i32, next_time: f64, flags: i32` | the same, with the neighbouring frames; 0 = ok |
//!
//! For each render the host asks `ec_alloc` for one buffer holding the parameters (`nparams`
//! little-endian `f64`s) followed by the pixels (`width × height × 4` little-endian `f32`s,
//! premultiplied RGBA, row-major), calls the render export, and reads the pixels back. An ABI 2
//! module gets two more pixel buffers after the current frame: the layer's previous and next
//! frames, resampled onto the current grid, and their layer times. The buffers are always
//! allocated (zeroed when a frame does not exist); `flags` bit 0/1 says which are real.
//!
//! Modules run in the [wasmi](https://github.com/wasmi-labs/wasmi) interpreter (pure Rust,
//! MIT/Apache-2.0) with deterministic floats, no imports (no files, network or clock) and a
//! fuel limit per frame, so a plug-in can't escape its sandbox, hang the app, or render
//! differently on another machine. The runtime is the `wasm` feature (on for the desktop app and
//! the CLI); without it [`load_wasm`] says plug-ins are unavailable.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
use effectcraft_effects::EffectSpec;
pub use effectcraft_effects::plugin::{
EffectPlugin, MIN_WASM_API_VERSION, PLUGIN_API_VERSION, PluginFrame, PluginImage, PluginManifest, PluginParams, WASM_API_VERSION, register_plugin,
};
#[cfg(feature = "wasm")]
mod wasm;
/// Whether this build can load WebAssembly plug-ins.
pub fn wasm_available() -> bool {
cfg!(feature = "wasm")
}
/// Load and register a WebAssembly plug-in (`bytes` = a `.wasm` binary or `.wat` text;
/// `source` = where it came from, for listings). Returns the registered effect.
pub fn load_wasm(bytes: &[u8], source: &str) -> Result<&'static EffectSpec, String> {
#[cfg(feature = "wasm")]
{
let p = wasm::WasmPlugin::new(bytes, source)?;
register_plugin(std::sync::Arc::new(p))
}
#[cfg(not(feature = "wasm"))]
{
let _ = (bytes, source);
Err("WebAssembly plug-ins are not available in this build".into())
}
}
/// [`load_wasm`] for the engine's plug-in loader hook: `{id, name, category, version, api}`.
pub fn loader(bytes: &[u8], source: &str) -> Result<serde_json::Value, String> {
let spec = load_wasm(bytes, source)?;
let m = effectcraft_effects::plugin::plugin(spec.id).map(|p| p.manifest().clone());
Ok(serde_json::json!({
"id": spec.id,
"name": spec.name,
"category": spec.category,
"version": m.as_ref().map(|m| m.version.clone()).unwrap_or_default(),
"api": m.as_ref().map(|m| m.api).unwrap_or(PLUGIN_API_VERSION),
"params": spec.params.len(),
}))
}
/// Render a module's effect without registering it (the Plug-in Library's preview): default
/// parameters, `pixels` (premultiplied RGBA, `width × height`) at each of `times` (seconds).
/// Returns the manifest as JSON and one frame per time.
pub fn preview(bytes: &[u8], width: u32, height: u32, pixels: &[[f32; 4]], times: &[f64]) -> Result<(serde_json::Value, Vec<Vec<[f32; 4]>>), String> {
#[cfg(feature = "wasm")]
{
use effectcraft_effects::plugin::{EffectPlugin, PluginFrame, PluginParamKind, PluginParams};
let p = wasm::WasmPlugin::new(bytes, "preview")?;
let m = p.manifest();
let mut flat = Vec::with_capacity(m.flat_len());
for param in &m.params {
match &param.kind {
PluginParamKind::Slider { default, .. } | PluginParamKind::Angle { default } => flat.push(*default),
PluginParamKind::Checkbox { default } => flat.push(if *default { 1.0 } else { 0.0 }),
PluginParamKind::Popup { default, .. } => flat.push(*default as f64),
// Points default to fractions of the layer: the frame is the layer.
PluginParamKind::Point { default } => flat.extend([default[0] * f64::from(width), default[1] * f64::from(height)]),
PluginParamKind::Color { default } => flat.extend(default.iter().copied()),
// API 2 kinds: a WebAssembly module can't declare them (`WasmPlugin::new` refuses them).
PluginParamKind::Point3 { default } => flat.extend(default.iter().copied()),
PluginParamKind::Layer => flat.push(-1.0),
PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. } => {}
}
}
let params = PluginParams { manifest: m, flat: &flat, texts: &[] };
let mut frames = Vec::with_capacity(times.len());
for &t in times {
let mut px = pixels.to_vec();
let mut frame = PluginFrame { width, height, pixels: &mut px, scale: 1.0, origin: [0.0; 2], layer_size: [f64::from(width), f64::from(height)] };
p.render(&mut frame, &params, t)?;
frames.push(px);
}
let manifest = serde_json::to_value(m).map_err(|e| e.to_string())?;
Ok((manifest, frames))
}
#[cfg(not(feature = "wasm"))]
{
let _ = (bytes, width, height, pixels, times);
Err("WebAssembly plug-ins are not available in this build".into())
}
}
// Two attributes, so clippy sees `cfg(test)` and treats the module as test code.
#[cfg(test)]
#[cfg(feature = "wasm")]
mod tests;