Commands: each built-in command's parameter keys are parsed once per process

Session::execute asked about a command's params description several
times per command (the key check, whether `id` stands for `ids`, whether
it takes `ids`), and each question parsed the description again, with a
command lookup for each word after the first of each `…` entry. The
engine now reads every built-in description, and the ones its `…`
entries name, once into a table keyed by the description's address and
length, and answers accepted_keys, takes_key, id_stands_for_ids and
check_keys from it.

A plug-in command's description, and a built-in description whose `…`
entries name a plug-in command, are still parsed where they are used,
since plug-ins change while the app runs. The keys, answers and error
messages are the same; tests::params_cache compares them for every
built-in command against a fresh parse.

In a release build on a one-object document, object.move with ids and
dx takes 48 us of thread CPU time per command instead of 61 us, and
select.set with ids 2.3 us instead of 3.2 us (medians of 7 alternating
runs of 20,000 commands in one process, while other builds were running;
tests::params_cache::bench_params_keys_once). The time is saved per
command, in scripts, batches and MCP or plug-in call loops, and does not
grow with the size of the document.
This commit is contained in:
dotCooCoo
2026-11-17 17:12:35 -08:00
parent ee60c51fd9
commit de06497716
4 changed files with 300 additions and 9 deletions
+130 -9
View File
@@ -36,6 +36,9 @@
//! `channel` that the effect doesn't have are refused by `effect.apply` and `effect.setParams`
//! themselves ([`check_effect_color`], [`check_effect_channel`]).
use std::collections::HashMap;
use std::sync::OnceLock;
use serde_json::Value;
use vectorcraft_apischema::shown_key;
@@ -43,7 +46,7 @@ use super::{bad, command_specs, find_command};
use crate::Result;
/// What a params description says about the keys of the params object.
#[derive(Debug, Default, PartialEq)]
#[derive(Clone, Debug, Default, PartialEq)]
pub struct ParamKeys<'a> {
/// The keys it names.
pub keys: Vec<&'a str>,
@@ -107,6 +110,8 @@ pub(crate) fn is_key(s: &str) -> bool {
/// The keys a params description names, or what in it doesn't read as one.
pub fn doc_keys(doc: &str) -> std::result::Result<ParamKeys<'_>, String> {
#[cfg(test)]
PARSES.with(|n| n.set(n.get() + 1));
let mut out = ParamKeys::default();
let (mut body, mut rest) = object_body(doc)?;
let mut entries = split_top(body, ',');
@@ -155,7 +160,12 @@ fn named_commands(more: &str) -> impl Iterator<Item = &'static super::CommandSpe
/// `None`: any key (a `…` entry naming no command, or a description that doesn't parse, which
/// the tests rule out).
fn accepted(doc: &'static str, depth: usize) -> Option<Vec<&'static str>> {
let parsed = doc_keys(doc).ok()?;
resolve(doc, depth, &mut |doc| doc_keys(doc).ok())
}
/// [`accepted`], with `parse` reading each description it reaches.
fn resolve(doc: &'static str, depth: usize, parse: &mut dyn FnMut(&'static str) -> Option<ParamKeys<'static>>) -> Option<Vec<&'static str>> {
let parsed = parse(doc)?;
let mut keys = parsed.keys;
for more in parsed.more {
let mut named = named_commands(more).peekable();
@@ -168,22 +178,120 @@ fn accepted(doc: &'static str, depth: usize) -> Option<Vec<&'static str>> {
if UNCHECKED.contains(&spec.id) {
return None;
}
keys.extend(accepted(spec.params, depth + 1)?);
keys.extend(resolve(spec.params, depth + 1, parse)?);
}
}
Some(keys)
}
/// What the params description of a built-in command says about its keys, read once per process
/// ([`known`]).
#[derive(Debug)]
struct Known {
/// The keys it names itself; none when it doesn't parse.
own: Vec<&'static str>,
/// [`accepted`] of it.
accepted: Option<Vec<&'static str>>,
/// [`id_stands_for_ids`] of it.
id_stands_for_ids: bool,
}
impl Known {
/// [`takes_key`] of the description.
fn takes(&self, key: &str) -> bool {
self.own.contains(&key) || self.accepted.as_ref().is_some_and(|keys| keys.contains(&key))
}
}
/// The place and length of `doc`'s text: a description with the same words elsewhere (a plug-in's)
/// has another place.
fn text_key(doc: &str) -> (usize, usize) {
(doc.as_ptr().addr(), doc.len())
}
/// [`Known`] of each description in `docs`, each one and each one its `…` entries name parsed once.
/// A description is left out when one it reaches has a `…` entry naming a plug-in command, whose
/// keys change as plug-ins are installed and removed.
fn read_descriptions(docs: impl IntoIterator<Item = &'static str>) -> HashMap<(usize, usize), Known> {
let names_plugin =
|more: &str| more.split_whitespace().any(|w| w.trim_matches(|c: char| !c.is_ascii_alphanumeric()).starts_with(vectorcraft_plugins::PREFIX));
let mut parsed: HashMap<(usize, usize), (Option<ParamKeys<'static>>, bool)> = HashMap::new();
let mut table = HashMap::new();
for doc in docs {
if table.contains_key(&text_key(doc)) {
continue;
}
let mut live = false;
let mut parse = |doc: &'static str| {
let (keys, plugin) = parsed.entry(text_key(doc)).or_insert_with(|| {
let keys = doc_keys(doc).ok();
let plugin = keys.as_ref().is_some_and(|k| k.more.iter().any(|m| names_plugin(m)));
(keys, plugin)
});
live |= *plugin;
keys.clone()
};
let own = parse(doc).map(|p| p.keys).unwrap_or_default();
let accepted = resolve(doc, 0, &mut parse);
if live {
continue;
}
let mut known = Known { own, accepted, id_stands_for_ids: false };
known.id_stands_for_ids = known.takes("ids") && known.takes("id") && no_required_id_beside_ids(doc);
table.insert(text_key(doc), known);
}
table
}
/// [`Known`] of `doc` when it is the params description of a built-in command.
fn known(doc: &str) -> Option<&'static Known> {
#[cfg(test)]
if UNCACHED.get() {
return None;
}
static TABLE: OnceLock<HashMap<(usize, usize), Known>> = OnceLock::new();
TABLE.get_or_init(|| read_descriptions(command_specs().iter().map(|c| c.params))).get(&text_key(doc))
}
#[cfg(test)]
thread_local! {
/// The times [`doc_keys`] parsed a description on this thread.
pub(crate) static PARSES: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
/// Whether [`known`] is skipped on this thread ([`uncached`]).
static UNCACHED: std::cell::Cell<bool> = const { std::cell::Cell::new(false) };
}
/// Whether [`read_descriptions`] keeps what `doc` says.
#[cfg(test)]
pub(crate) fn read_once(doc: &'static str) -> bool {
read_descriptions([doc]).contains_key(&text_key(doc))
}
/// `f`'s result with every description parsed again where it is used, as without [`known`].
#[cfg(test)]
pub(crate) fn uncached<T>(f: impl FnOnce() -> T) -> T {
let before = UNCACHED.replace(true);
let out = f();
UNCACHED.set(before);
out
}
/// Every key the description `doc` takes, those of the commands its `…` entries name included, or
/// `None` when that isn't known: it doesn't parse, a `…` entry names no command, or one names a
/// command in [`UNCHECKED`].
pub(crate) fn accepted_keys(doc: &'static str) -> Option<Vec<&'static str>> {
accepted(doc, 0)
match known(doc) {
Some(known) => known.accepted.clone(),
None => accepted(doc, 0),
}
}
/// Does the description `doc` name `key` as a parameter (its own keys, and those of the commands
/// its `…` entries name)?
pub fn takes_key(doc: &'static str, key: &str) -> bool {
if let Some(known) = known(doc) {
return known.takes(key);
}
doc_keys(doc).is_ok_and(|parsed| parsed.keys.contains(&key)) || accepted(doc, 0).is_some_and(|keys| keys.contains(&key))
}
@@ -191,12 +299,17 @@ pub fn takes_key(doc: &'static str, key: &str) -> bool {
/// (instead of id)}`, `{id} or {ids: […]}`): it names both keys, and no params object of it names
/// `ids` beside a required `id`, which is something else (`plugin.run {id, ids?}`: the plug-in's)?
pub(crate) fn id_stands_for_ids(doc: &'static str) -> bool {
if let Some(known) = known(doc) {
return known.id_stands_for_ids;
}
takes_key(doc, "ids") && takes_key(doc, "id") && no_required_id_beside_ids(doc)
}
/// No params object of `doc` names `ids` beside a required `id`.
fn no_required_id_beside_ids(doc: &str) -> bool {
fn head(alternative: &str) -> &str {
alternative.split([':', '=', '(']).next().unwrap_or_default().trim()
}
if !takes_key(doc, "ids") || !takes_key(doc, "id") {
return false;
}
let mut rest = doc;
while let Ok((body, after)) = object_body(rest) {
let entries = split_top(body, ',');
@@ -249,9 +362,17 @@ pub fn check_keys(id: &str, doc: &'static str, params: &Value) -> Result<()> {
if given.is_empty() {
return Ok(());
}
let Some(keys) = accepted(doc, 0) else { return Ok(()) };
let live;
let keys = match known(doc) {
Some(known) => known.accepted.as_deref(),
None => {
live = accepted(doc, 0);
live.as_deref()
}
};
let Some(keys) = keys else { return Ok(()) };
match given.keys().find(|k| !keys.contains(&k.as_str())) {
Some(key) => Err(unknown(id, key, &keys)),
Some(key) => Err(unknown(id, key, keys)),
None => Ok(()),
}
}
+1
View File
@@ -122,6 +122,7 @@ mod paintproxy;
mod panelcmds;
mod paragraphs;
mod params;
mod params_cache;
mod path_menu_enabled;
mod pathfinder_clip;
mod pathops;
+168
View File
@@ -0,0 +1,168 @@
//! The params description of each built-in command is parsed once per process; the answers read
//! from it are those of parsing it where they are used.
use std::hint::black_box;
use serde_json::{Value, json};
use crate::cmd::params::{PARSES, accepted_keys, check_keys, doc_keys, id_stands_for_ids, read_once, takes_key, uncached};
use crate::{Session, command_specs};
/// Every answer the key functions give about `doc` for command `id`, as text.
fn answers(id: &str, doc: &'static str) -> Vec<String> {
let own = doc_keys(doc).map(|p| p.keys).unwrap_or_default();
let accepted = accepted_keys(doc);
let mut out = vec![format!("{accepted:?}"), format!("{}", id_stands_for_ids(doc))];
let near = own.first().map(|k| format!("{k}q"));
for key in own.iter().copied().chain(["ids", "id", "zzz"]).chain(near.as_deref()) {
out.push(format!("{key}: {}", takes_key(doc, key)));
}
let all: serde_json::Map<String, Value> = accepted.iter().flatten().map(|k| (k.to_string(), Value::Null)).collect();
let mut cases = vec![json!({}), json!({"zzz": 1}), Value::Object(all)];
cases.extend(near.map(|k| json!({ k: 1 })));
for params in cases {
out.push(format!("{params}: {:?}", check_keys(id, doc, &params)));
}
out
}
#[test]
fn every_command_gets_the_answers_of_its_parsed_description() {
for c in command_specs() {
assert_eq!(answers(c.id, c.params), uncached(|| answers(c.id, c.params)), "{}", c.id);
}
}
#[test]
fn a_built_in_description_is_not_parsed_again() {
let mut s = Session::new();
s.execute("file.new", &json!({"width": 200, "height": 200, "created": null})).unwrap();
let id = s.execute("shape.rectangle", &json!({"x": 10, "y": 10, "width": 50, "height": 40})).unwrap()["id"].as_u64().unwrap();
let run = |s: &mut Session| {
s.execute("select.set", &json!({"ids": [id]})).unwrap();
s.execute("object.move", &json!({"ids": [id], "dx": 1})).unwrap();
s.execute("object.move", &json!({"ids": [id], "dx": -1})).unwrap();
for c in command_specs() {
black_box((accepted_keys(c.params), takes_key(c.params, "ids"), takes_key(c.params, "id"), id_stands_for_ids(c.params)));
black_box(check_keys(c.id, c.params, &json!({"zzz": 1})).is_err());
}
};
run(&mut s);
PARSES.set(0);
run(&mut s);
assert_eq!(PARSES.get(), 0, "descriptions parsed again");
// Without the table each command parses its description where it is used.
uncached(|| run(&mut s));
assert!(PARSES.get() > command_specs().len());
}
/// A description that isn't a built-in command's, even with the same words (a plug-in's), is parsed
/// where it is used, and so is one that names a plug-in command, whose keys change as plug-ins are
/// installed and removed.
#[test]
fn other_descriptions_are_parsed_where_they_are_used() {
let spec = command_specs().iter().find(|c| id_stands_for_ids(c.params)).unwrap();
let copy: &'static str = String::from(spec.params).leak();
assert_eq!(answers(spec.id, copy), answers(spec.id, spec.params));
PARSES.set(0);
assert!(takes_key(copy, "ids") && id_stands_for_ids(copy));
assert!(PARSES.get() > 0);
assert!(read_once("{a, …settings as file.new}"));
assert!(!read_once("{a, …settings as plugin.some.command}"));
assert!(!read_once("{a, …the options of `plugin.some.command`}"));
}
/// This thread's CPU time in ns where Linux reports it, else the time since `start`.
fn now(start: std::time::Instant) -> (u128, bool) {
let cpu = std::fs::read_to_string("/proc/thread-self/schedstat").ok().and_then(|s| s.split_whitespace().next()?.parse().ok());
match cpu {
Some(ns) => (ns, true),
None => (start.elapsed().as_nanos(), false),
}
}
fn median(mut v: Vec<u128>) -> u128 {
v.sort_unstable();
v[v.len() / 2]
}
/// `cargo test --release -p vectorcraft-engine --lib params_cache -- --ignored --nocapture` prints
/// the time of the key questions over every built-in description and of `Session::execute` on a
/// one-object document, with descriptions parsed where they are used (`parse`) and read once
/// (`once`), alternating in one process: the median of 7 runs, in this thread's CPU time on Linux.
#[test]
#[ignore = "benchmark"]
fn bench_params_keys_once() {
let start = std::time::Instant::now();
let specs = command_specs();
let questions = || {
for c in specs {
black_box((accepted_keys(c.params), takes_key(c.params, "ids"), takes_key(c.params, "id")));
black_box((takes_key(c.params, "ids"), takes_key(c.params, "id"), id_stands_for_ids(c.params)));
}
};
let time = |f: &dyn Fn()| {
let t = now(start);
f();
(now(start).0 - t.0, t.1)
};
questions();
uncached(questions);
let (mut parse, mut once, mut cpu) = (Vec::new(), Vec::new(), true);
for _ in 0..7 {
let (p, c1) = time(&|| uncached(|| (0..200).for_each(|_| questions())));
let (o, c2) = time(&|| (0..200).for_each(|_| questions()));
parse.push(p);
once.push(o);
cpu &= c1 && c2;
}
let per = |ns: u128| ns / (200 * specs.len() as u128);
println!("clock: {}", if cpu { "thread CPU time" } else { "wall time" });
println!("key questions, ns per description: parse {} once {}", per(median(parse)), per(median(once)));
for (cmd, params) in [("select.set", "ids"), ("object.move", "move")] {
let doc = || {
let mut s = Session::new();
s.execute("file.new", &json!({"width": 200, "height": 200, "created": null})).unwrap();
let id = s.execute("shape.rectangle", &json!({"x": 10, "y": 10, "width": 50, "height": 40})).unwrap()["id"].as_u64().unwrap();
let p = if params == "ids" { json!({"ids": [id]}) } else { json!({"ids": [id], "dx": 1}) };
(s, p)
};
let run = |s: &mut Session, p: &Value| {
for _ in 0..20_000 {
black_box(s.execute(cmd, p).unwrap());
}
};
let ((mut a, pa), (mut b, pb)) = (doc(), doc());
assert_eq!(uncached(|| a.execute(cmd, &pa).unwrap()), b.execute(cmd, &pb).unwrap(), "{cmd}: both ways give the same result");
let timed = |s: &mut Session, p: &Value, cached: bool| {
let t = now(start).0;
if cached {
run(s, p)
} else {
uncached(|| run(s, p))
}
now(start).0 - t
};
let (mut parse, mut once) = (Vec::new(), Vec::new());
for i in 0..7 {
let (mut s1, p1) = doc();
let (mut s2, p2) = doc();
if i % 2 == 0 {
parse.push(timed(&mut s1, &p1, false));
once.push(timed(&mut s2, &p2, true));
} else {
once.push(timed(&mut s2, &p2, true));
parse.push(timed(&mut s1, &p1, false));
}
// Each document has a `uid` of its own.
let state = |s: &Session| {
let mut d = crate::inspect::document(s);
d.as_object_mut().unwrap().remove("uid");
d
};
assert_eq!(state(&s1), state(&s2), "{cmd}: same document");
}
let ms = |ns: u128| ns as f64 / 1e6;
println!("{cmd} x 20,000: parse {:.1} ms once {:.1} ms", ms(median(parse)), ms(median(once)));
}
}
+1
View File
@@ -944,6 +944,7 @@ Moved from the ROADMAP's "Shipped so far" on 2026-10-10. Update it when a featur
- Plug-in, recovery and script report replies under `--automation-read-root`: `plugin.list`, `plugin.info` and `plugin.safeMode` name the plug-in folder, failed files, plug-in sources and disabled plug-ins' files outside the read roots by file name or `null`, `file.recovery.*` leave out a document's original file and the recovery folder outside them, and `ui.inspect`'s `scriptReport` names files a run wrote outside them by file name. `crates/engine/src/tests/plugins.rs` (`plugin_list_leaves_out_paths_outside_the_read_root`), `crates/engine/src/tests/recovery.rs` (`recovery_replies_leave_out_paths_outside_the_read_root`), `crates/ui-egui/src/tests/automation.rs` (`ui_inspect_script_report_leaves_out_files_outside_the_read_root`).
- Scripts: `alert`, `confirm` and `prompt` open dialogs during a run, each with a Stop button that ends the run and puts every document back. In the app the run waits at the call for the person's answer; a run over MCP or the command line takes its answers in order from `script.run`'s `answers`, and the report lists each dialog with its answer. `crates/engine/src/tests/script_dialogs.rs`, `crates/ui-egui/src/tests/script_dialogs.rs`.
- Parameter keys: the engine refuses a key that a command's params description doesn't name before the command runs, in every build and for every caller, including the app's own calls, the commands a command runs and the steps of `command.batch` and `action.batch`. An action recorded by an older version still plays from the Actions panel and the Batch dialog (`Session::replay`). `crates/engine/tests/it/param_keys.rs`.
- Parameter key lookups: the engine reads the params description of each built-in command, with the descriptions its `…` entries name, once per process, and answers the key check and the `ids` and `id` questions of each later command from that reading. A plug-in command's description, and a description whose `…` entries name a plug-in command, are still read each time. The accepted keys, the answers and the error messages are the same. In a release build on a one-object document, `object.move` with `ids` and `dx` takes 48 µs of thread CPU time per command instead of 61 µs, and `select.set` with `ids` 2.3 µs instead of 3.2 µs (medians of 7 alternating runs of 20,000 commands in one process, measured while other builds were running; `tests::params_cache::bench_params_keys_once`). The time is saved per command, in scripts, batches and MCP or plug-in call loops, and does not grow with the size of the document. `crates/engine/src/tests/params_cache.rs` compares every built-in command's answers with those of a fresh parse.
- Effect parameter types: `effect.apply`, `appearance.addEffect` and `effect.setParams` refuse an effect parameter of another type than the effect reads, for keys only the params text names too (the earlier 3D `perspective`, `lightAzimuth` and `lightElevation`, Scribble's `variation`, Outline Stroke's `width`, Gaussian Blur's `linear`), a Crop Marks `style` other than `roman` or `japanese`, and an installed plug-in effect's parameter of another type than its manifest gives it, in a `command.batch` step too, with the document and undo history unchanged; a replayed action still plays. `effect.setParams` checks the parameters against a Warp's new style, and refuses a `visible` that is not true or false, in a replayed action too. `crates/engine/src/tests/params.rs` (`an_effect_parameter_without_a_default_is_held_to_its_type`, `effect_commands_refuse_an_undefaulted_parameter_of_the_wrong_type`), `crates/engine/src/tests/plugins.rs` (`live_effect_plugins_reshape_editable_art_and_expand`), `crates/engine/src/cmd/effectcmd/tests.rs` (`set_params_refuses_a_visible_that_is_not_true_or_false`, `set_params_checks_the_effect_it_leaves`).
- MCP server with 33 tools (drawing, text, effects, Pathfinder, transforms, graphs, text wrap, export, screenshots, undo/redo, UI inspect/screenshot aliases, any command), attached to the running app or headless. Headless and remote `command_list` include desktop `UI_COMMANDS` (`ui: true`); a UI-only id still needs `--connect` to run. Eight prompts (poster, icon-set, recolor, trace-and-style, export-set, check-spelling, variables-bind, path-tools). Effect ids are not command ids. Prefer a CLI built from the checkout when a release binary lags (#6460). `vectorcraft://commands/reference` serves the API reference. Tool descriptions and replies agree with the engine: format lists come from its format table, `save_file` says it keeps the document's own format, `inspect_document` reports the paint type's characters show, and selection exports report the encoder's warnings. `--automation-read-root` / `--automation-write-root` (on `vectorcraft-cli mcp` and on `vectorcraft --control`) confine the files commands read and write to chosen folders, links followed, with the flags PhotoCraft takes (#832); each flag may be given more than once, a path inside any of its folders being allowed (#3546, `file_access::tests::several_roots_of_a_kind_each_grant_their_access`, `cli::mcp_automation_roots_take_several_folders`). A long export can be cancelled (`notifications/cancelled` stops it at the next page or group and writes nothing), reports page progress to a client that sent a progress token, and does not make other requests wait: engine-free ones answer at once, the rest are refused as busy after two seconds (#2165, #2171, #2172).
- MCP window captures from an app that does not send the image back go through a file in a new folder inside the system's temporary folder, which only the user can open on Unix, instead of a fixed name in the shared folder; the folder is removed after the read, also when the capture or the read fails (`tests::conventions::an_old_app_captures_the_window_in_a_new_private_folder`).