mirror of https://github.com/storytold/effectcraft.git synced 2026-11-07 12:13:22 +00:00

Files

T

611a282bea Plug-ins: external effect helpers (safe, out-of-process native effects) + plug-in API 2 (#889 )

* Plug-ins: API 2 (PluginHost, padding, layer_span)

Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI
stays at 1.

- point3 / layer / text / hidden parameters, integer sliders, parameter groups.
- EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side).
- EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters
  at other times, frame rate, per-instance key, the layer's source range.
- EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024.
- plugin_error: a plug-in effect's last render error, for the UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Plug-ins: external effect helpers (crates/helper-host)

Native plug-in formats run in separate helper programs, never in EffectCraft's process
(docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3,
native only, unsafe forbidden): it finds helper programs, starts each as a child process and
registers its effects through plug-in API 2, so the UI and command surface need nothing new.

- Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are
  prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on
  Linux), never through the pipe. Render-time callbacks for the input at other times, other layers
  and parameters at other times, with nested renders through the same helper.
- A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails
  with a message, the frame is left as it was, and the helper restarts for the next render. An
  effect that stops its helper 3 times is turned off (helper_blocklist.txt).
- Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts
  in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing
  to the settings folder alone can't run code. effect.helpers.scan / load are refused for
  confined agent sessions (confine.rs), like file.executeFile.
- Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings
  folder, EFFECTCRAFT_HELPER_PATH.
- Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render
  error of an effect shows as a warning sign after its name in Effect Controls.
- serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render,
  padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper,
  with nothing to install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Effect helpers: address review (start failures, frame file mode, nextest listing)

- A helper that can't start (not installed any more, restarting too often, no frame exchange
  file) is reported but no longer charged to the effect being rendered: it is Failure::Refused,
  not Failure::Broken, so it never counts toward turning that effect off. New check in
  tests/it: once the helper is out of restarts, renders of an effect fail without blocking it
  (fails without the fix: the effect was turned off after 3 such renders).
- The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and
  /dev/shm is shared. Asserted in the frames unit test.
- tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses:
  `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and
  `helper_protocol --exact` runs the checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step

- Writes to a helper go through a writer thread with the request timeout: a
  helper that stops reading its stdin (hung with a large callback answer or
  render request pending) no longer blocks the render thread forever while
  holding the helper.
- stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that
  isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE /
  SIGPIPE on its next write); lines were unbounded.
- Callback images that can't be written are answered with `null` instead of
  failing the render while the helper waits (which left the protocol out of
  step); a frame that can't be written before the request is sent is not
  charged to the effect.
- Mid-render calls use the render's own process: no respawn or heartbeat ping
  while the helper is waiting for an answer.
- Waiting for a busy helper is bounded (load timeout) instead of forever;
  status() no longer blocks on a running render.
- Reap the child when start-up fails after spawning; cap the block list read.
- Scripts start helpers only with Allow Scripts to Write Files and Access
  Network on (they run native code), as for File.execute().
- helper-host declares the workspace's no-panic crate lints; the desktop
  main() keeps its doc comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again

- WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while
  PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the
  library's compatibility check use it, so it never installs an API 2 release that
  can't load.
- with_running: a process that is already gone was stopped by what broke it (a nested
  render, charged there), so this render fails without charging its own effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frames: plug-in effects count as reading other frames (plug-in API 2)

With main's narrowed frame invalidation (#2795), an edit inside a precomp under an
effect that reads its layer at other times must reach the layer's whole span.
Plug-in API 2 gives every plug-in effect the input at other times, so any effect
outside ec. counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>

2026-11-05 11:11:57 +01:00

4.5 KiB

Raw Permalink Blame History

ADR 0001: External effect helpers

Status: proposed
Date: 2026-10-11
Context: review of storytold/effectcraft#508 (an in-process OpenFX host)

Context

People want to use effects that already exist as native plug-ins: OpenFX suites (Sapphire, ReelSmart Motion Blur, Twixtor, ...) and, later, other native formats. storytold/effectcraft#508 hosted OpenFX in-process. That needs unsafe to call a C ABI and runs closed-source native code in EffectCraft's own process, which conflicts with two non-negotiables:

Never crash (AGENTS.md): one null dereference in a third-party plug-in takes the whole app, and the user's unsaved work, down with it. catch_unwind can't catch a segfault, and a blocklist only helps after the first crash.
No unsafe in the workspace (unsafe_code = "forbid").

Decision

EffectCraft never loads native plug-in code into its own process. Native effects are served by external effect helpers: separate programs that EffectCraft starts as child processes and talks to over a small, versioned protocol. The core gains one generic, safe-Rust crate, effectcraft-helper-host (L3); format-specific hosts (OpenFX, ...) live in helper programs that are built, released and installed separately.

Crash containment. The process boundary contains everything a helper does. A helper that exits, hangs past a timeout or breaks the protocol is stopped. The render that hit it fails with a message shown on the effect, the frame stays as it was, and the helper starts again for the next render. An effect that takes its helper down three times is turned off and remembered in helper_blocklist.txt until the user retries.
Same effects surface. Each remote effect is registered through the existing plug-in API 2 (EffectPlugin), so the UI, the property tree, keyframes, expressions, scripting, the CLI, the control channel and MCP need nothing new.
Protocol (docs/external-helpers.md): JSON lines over the helper's stdin / stdout (hello → describe → render with callbacks → done, plus padding, load, destroy, ping, shutdown). Pixels go through a frame exchange file (in /dev/shm on Linux, the temp folder elsewhere) addressed by offsets, never through the pipe. Callbacks (input_at, layer_input, params_at) let temporal effects (motion blur, retiming) read other frames while rendering; nested renders through the same helper are supported.
Discovery. EffectCraft looks in helpers/ next to the executable (and Contents/Helpers in a macOS bundle), helpers/ in the settings folder, and EFFECTCRAFT_HELPER_PATH. Helper programs are named effectcraft-helper-*.
Security. Helpers are local programs the user installed, just like the plug-ins they host. EffectCraft runs only programs from those locations, passes them nothing beyond the protocol (no network, no project files), and runs no helper on the web build.

Alternatives considered

In-process host behind a feature flag (#508). Rejected: a crash is still a crash, and it needs unsafe in the workspace.
ipc-channel for transport. Its wire format and shared-memory handles are Rust-specific (and differ per OS), so a helper written in C++ against the OpenFX reference host couldn't speak it. The chosen transport, JSON lines plus a plain file, works from any language without a library.
Memory-mapped shared memory on the EffectCraft side. Mapping needs unsafe (or a crate whose safe API is unsound if the other process truncates the mapping). Positional reads and writes on a file in the page cache cost one copy per frame each way and stay safe. A helper is free to map the file.
WebAssembly-compiled plug-ins. Not available for closed-source native plug-ins.

Consequences

Native effects cost one copy of the frame each way plus process round trips. That's fine for CPU plug-ins working on whole frames.
Renders through one helper are serialized (one process, one conversation at a time). A pool of helper processes is a possible later improvement.
Out of scope for the core: any OpenFX (or other native-format) code. The OpenFX helper is a separate program in its own repository (an example: ec-extension-ofx, unofficial) that reuses the existing Rust OpenFX host behind serve(). A C++ helper built on the OpenFX HostSupport library is an equally valid way to implement the same protocol.
Later: the ArtCraft launcher can install helpers into helpers/ the same way it installs apps.

4.5 KiB Raw Permalink Blame History