diff --git a/crates/engine/src/cmd/scripts.rs b/crates/engine/src/cmd/scripts.rs index 3400075b2..66f3f8bf8 100644 --- a/crates/engine/src/cmd/scripts.rs +++ b/crates/engine/src/cmd/scripts.rs @@ -32,7 +32,7 @@ pub fn specs() -> Vec { "Run Script", [], None, - "{path?: a .js or .jsx file | source?: the script's text (exactly one of the two), name?: the name errors and the report use (default: the path's file name, or \"script\"), args?: {…} (at most 1 MiB as JSON; vc.args in the script), answers?: {} (no prompts exist yet: only an empty object is taken), maxFuel?: whole number from 1 to 100000000000 (instructions in all; default 1000000000)} run a JavaScript file against the open documents, through the vc object: vc.run(id, params) runs a command and returns its result, a failed command throws an error the script can catch and puts back the documents and the session state it changed, while any file it wrote stays written and is named; vc.document is a snapshot of the active document or null; vc.commands lists the commands a script may run; vc.log(value) adds to the report and the Script Console (script.console); $.write(...values) writes the values to the Script Console as String(value) gives them, and $.writeln(...values) does the same and ends the line; vc.args is args; vc.api.version and vc.api.has(feature) describe the API. The run is one transaction: on success each changed document gets one undo step, Run Script, and each preference it changed keeps its value, and on an uncaught error, a limit or script.stop every open document and the session go back as they were; files a command wrote before that stay written and are named. While it runs, other commands that change anything end with a busy error. Commands that change anything outside the documents (script.*, except script.check, plugin.*, prefs.*, actionSet.*, shortcutSet.*, help.*, print.*, library.*, importMatches.*, edit.undoGroup.*, file.recovery.*, stroke.widthProfile.*, saving, importing or deleting a preset, text.addFontFiles, text.findFontFiles, text.rescanFonts, links.findFiles, file.place.queue, color.loadProfile, color.removeProfile, file.print, view.proofSetup, view.proofColors, view.overprintPreview, view.separationsPreview) throw. VectorCraft's own limits: 8 MiB of source, 256 MiB of heap (128 MiB on the web), 2000000 host calls, 200000000 instructions without a pause, 64 levels, 16 MiB and 1048576 entries for a value crossing to or from a command, 10000 log entries and 4 MiB of log, and 100000 objects and 512 MiB of new document data, past which a person is asked to continue or stop and a run with nobody to ask ends → {ok: true, name, changed: [document names], written: [paths], log: [values], fuelUsed, hostCalls, runTimeMs (the time the script ran, its commands included, waits between frames and for a person's answer left out; 0 without a clock), timingAvailable (false on the web), hostCallCounts: {vcRun, documentReads, commandListReads, vcLog, consoleWrites}, slowestCommands: [{id, calls, totalMs, maxMs}] (at most 5, the longest in all first, then by id), console: {lines: [text], droppedLines, truncatedLines} (the run's own output, vc.log lines and error)}, or, in the app, {running: true, name} and the report later in ui.inspect's scriptReport; a run that fails ends with its error, and its report there also has kind, message, line, column, fuelLeft, fuelUsed, hostCalls and the fields above", + "{path?: a .js or .jsx file | source?: the script's text (exactly one of the two), name?: the name errors and the report use (default: the script's #script line, else the path's file name, or \"script\"), args?: {…} (at most 1 MiB as JSON; vc.args in the script), answers?: {} (no prompts exist yet: only an empty object is taken), maxFuel?: whole number from 1 to 100000000000 (instructions in all; default 1000000000)} run a JavaScript file against the open documents, through the vc object: vc.run(id, params) runs a command and returns its result, a failed command throws an error the script can catch and puts back the documents and the session state it changed, while any file it wrote stays written and is named; vc.document is a snapshot of the active document or null; vc.commands lists the commands a script may run; vc.log(value) adds to the report and the Script Console (script.console); $.write(...values) writes the values to the Script Console as String(value) gives them, and $.writeln(...values) does the same and ends the line; $.evalFile(path, timeout?) runs a file with the same globals and returns its last value (timeout, in milliseconds from 0, is taken and not waited on), $.fileName is the file being run (empty for text) and $.includePath the #includepath list; vc.args is args. Before anything runs, the script's preprocessor lines (# or //@ at the start of a line, outside strings and comments) are read: #include \"file\" and //@include put that file's text in place of the line, looked for beside the file that includes it, then in each #includepath folder in turn (separated by ;, relative to that file's folder; a later #includepath replaces the list), then in each JSINCLUDE folder; #script names the run; #target takes any value and changes nothing; #strict off is taken; #strict on, #targetengine and #engine are refused as unsupported, any other #name as unknown, and another //@name is a comment. A script run from a file reads its includes and $.evalFile's files only in that file's folder and the folders below it, refusing a path with a .. name or a link that leads out (a JSINCLUDE folder outside is passed over); a script given as text reads none. An error in another file is at that file's line and column; vc.api.version and vc.api.has(feature) describe the API. The run is one transaction: on success each changed document gets one undo step, Run Script, and each preference it changed keeps its value, and on an uncaught error, a limit or script.stop every open document and the session go back as they were; files a command wrote before that stay written and are named. While it runs, other commands that change anything end with a busy error. Commands that change anything outside the documents (script.*, except script.check, plugin.*, prefs.*, actionSet.*, shortcutSet.*, help.*, print.*, library.*, importMatches.*, edit.undoGroup.*, file.recovery.*, stroke.widthProfile.*, saving, importing or deleting a preset, text.addFontFiles, text.findFontFiles, text.rescanFonts, links.findFiles, file.place.queue, color.loadProfile, color.removeProfile, file.print, view.proofSetup, view.proofColors, view.overprintPreview, view.separationsPreview) throw. VectorCraft's own limits: 8 MiB of source a file and 32 MiB in all, 64 included or evaluated files 16 deep, 1024 paths looked up for them, 256 MiB of heap (128 MiB on the web), 2000000 host calls, 200000000 instructions without a pause, 64 levels, 16 MiB and 1048576 entries for a value crossing to or from a command, 10000 log entries and 4 MiB of log, and 100000 objects and 512 MiB of new document data, past which a person is asked to continue or stop and a run with nobody to ask ends → {ok: true, name, changed: [document names], written: [paths], log: [values], fuelUsed, hostCalls, runTimeMs (the time the script ran, its commands included, waits between frames and for a person's answer left out; 0 without a clock), timingAvailable (false on the web), hostCallCounts: {vcRun, documentReads, commandListReads, vcLog, consoleWrites}, slowestCommands: [{id, calls, totalMs, maxMs}] (at most 5, the longest in all first, then by id), console: {lines: [text], droppedLines, truncatedLines} (the run's own output, vc.log lines and error)}, or, in the app, {running: true, name} and the report later in ui.inspect's scriptReport; a run that fails ends with its error, and its report there also has kind, message, line, column, file (when the error is in another file than the script), fuelLeft, fuelUsed, hostCalls and the fields above", always, run ), @@ -50,7 +50,7 @@ pub fn specs() -> Vec { "Script Console", [], None, - "{} the Script Console's lines, oldest first: each run's start, its $.write and $.writeln output, its vc.log entries, its uncaught error as name:line:column: message, its end and a report of its time, host calls and slowest commands; a session keeps at most 2048 lines and 1 MiB of text, dropping the oldest, a line at most 4096 bytes and a script's name in its lines at most 256 bytes, each cut with … (VectorCraft's own limits); line feeds, carriage returns and the two together break lines, other control characters show as their Unicode Control Pictures → {lines: [{run: the run's number in the session, name, kind: start|output|log|error|end|report, text}], droppedLines, truncatedLines} (an end line's text is ended: finished|error|fuel|hostLimit|limit|stopped|internal)", + "{} the Script Console's lines, oldest first: each run's start, its $.write and $.writeln output, its vc.log entries, its uncaught error as name:line:column: message (name: file:line:column: message when it is in another file), its end and a report of its time, host calls and slowest commands; a session keeps at most 2048 lines and 1 MiB of text, dropping the oldest, a line at most 4096 bytes and a script's name in its lines at most 256 bytes, each cut with … (VectorCraft's own limits); line feeds, carriage returns and the two together break lines, other control characters show as their Unicode Control Pictures → {lines: [{run: the run's number in the session, name, kind: start|output|log|error|end|report, text}], droppedLines, truncatedLines} (an end line's text is ended: finished|error|fuel|hostLimit|limit|stopped|internal)", always, console_lines ), @@ -162,7 +162,8 @@ fn run(s: &mut Session, p: &Value) -> Result { _ => return Err(param("give exactly one of `path` and `source`")), }; let name = given_name.map(str::to_string).or_else(|| path.map(super::fileio::file_name)).unwrap_or_else(|| "script".to_string()); - s.script_start(script::Request { name: name.clone(), source: text, args, fuel })?; + let request = script::Request { name, name_given: given_name.is_some(), file: path.map(str::to_string), source: text, args, fuel }; + let name = s.script_start(request)?; if s.script_frames { return Ok(json!({"running": true, "name": name})); } diff --git a/crates/engine/src/file_access.rs b/crates/engine/src/file_access.rs index c6daff9e4..3f0a641e2 100644 --- a/crates/engine/src/file_access.rs +++ b/crates/engine/src/file_access.rs @@ -23,6 +23,9 @@ //! A path is checked, then opened: a process that can change the folders in between (swapping in //! a link) can still win that race. The roots keep an agent to the files it was given; they don't //! guard against other programs on the computer. +//! +//! [`ScriptScope`] holds the folders a running script's own files (its includes and `$.evalFile`) +//! are read from, inside the roots. use std::cell::RefCell; use std::ffi::OsString; @@ -347,6 +350,88 @@ fn windows_name_refusal(path: &str) -> Option<&'static str> { }) } +/// The folders a running script's own files are read from: the files its `#include` lines name and +/// the files it passes to `$.evalFile` (`Scope::Script`'s reads). VectorCraft's own rules: a script +/// run from a file may read inside that file's folder and the folders below it, and a script given as +/// text may read none. A path is refused when it has a `..` component, a Windows device name or +/// stream, or names a computer on the network, or when it leads (through its links, junctions +/// included) out of every granted folder. The automation roots ([`check_read`]) apply as well. +/// +/// The file opened is the path with its links followed, and after reading, the path is followed +/// again and must still lead to the same place: a process that changes the folders twice in between +/// can still win that race. +#[derive(Clone, Debug, Default)] +pub(crate) struct ScriptScope { + /// The granted folders, made absolute with their links followed. + roots: Vec, +} + +/// Why a script's file was not read. +#[derive(Clone, Debug, PartialEq, Eq)] +pub(crate) enum ScriptRead { + /// There is no such file. + Missing, + /// The scope does not let the script read it. + Refused, + /// The file has more than the bytes asked for. + TooLarge, + /// The file system gave this error. + Failed(String), +} + +impl ScriptScope { + /// The scope of a script run from a file in folder `dir`: that folder. + pub(crate) fn of_folder(dir: &Path) -> Self { + Self { roots: std::fs::canonicalize(dir).ok().into_iter().collect() } + } + + /// Is `path` refused by its name alone: a `..` component, a NUL, a Windows device name or + /// stream, or a computer on the network? + fn refused_by_name(path: &Path) -> bool { + let text = path.to_string_lossy(); + text.contains('\0') + || path.components().any(|c| c == Component::ParentDir) + || (cfg!(windows) && (windows_name_refusal(&text).is_some() || network_path(&text))) + } + + /// The real path of the file at `path` and its first `max` bytes, at most. + pub(crate) fn read(&self, path: &Path, max: usize) -> Result<(PathBuf, Vec), ScriptRead> { + use std::io::Read; + if self.roots.is_empty() || Self::refused_by_name(path) || check_read(&path.to_string_lossy()).is_err() { + return Err(ScriptRead::Refused); + } + let real = match std::fs::canonicalize(path) { + Ok(real) => real, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + // A link whose target is missing is there all the same: it is not followed. + return Err(if std::fs::symlink_metadata(path).is_ok() { ScriptRead::Refused } else { ScriptRead::Missing }); + } + Err(e) => return Err(ScriptRead::Failed(e.to_string())), + }; + if !self.roots.iter().any(|r| inside(&real, r)) { + return Err(ScriptRead::Refused); + } + let file = std::fs::File::open(&real).map_err(|e| ScriptRead::Failed(e.to_string()))?; + let meta = file.metadata().map_err(|e| ScriptRead::Failed(e.to_string()))?; + if !meta.is_file() { + return Err(ScriptRead::Missing); + } + if meta.len() > max as u64 { + return Err(ScriptRead::TooLarge); + } + let mut bytes = Vec::new(); + // One byte more than asked for shows a file that grew while it was read. + file.take(max as u64 + 1).read_to_end(&mut bytes).map_err(|e| ScriptRead::Failed(e.to_string()))?; + if bytes.len() > max { + return Err(ScriptRead::TooLarge); + } + if std::fs::canonicalize(path).ok().as_ref() != Some(&real) { + return Err(ScriptRead::Refused); + } + Ok((real, bytes)) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/engine/src/script/include.rs b/crates/engine/src/script/include.rs new file mode 100644 index 000000000..df7702d77 --- /dev/null +++ b/crates/engine/src/script/include.rs @@ -0,0 +1,431 @@ +//! A script's preprocessor lines, read before it runs, as the public guide lists them +//! (): +//! `#include` and `//@include` put the text of another file in place of the line, `#includepath` +//! sets the folders `#include` looks in, `#script` names the run, and `#target` and `#strict off` +//! are accepted. The same lines are read in a file `$.evalFile` runs. +//! +//! VectorCraft's own rules: +//! +//! - A line counts only at the start of a line (blanks before it), outside strings and comments, as +//! the lexer reads it, and each file must be a whole script on its own. +//! - An included file is looked for beside the file that includes it, then in each `#includepath` +//! folder in turn (a relative one is relative to that same file's folder), then in each folder of +//! the `JSINCLUDE` environment variable. A later `#includepath` replaces the list. +//! - Files are read through the script's [`ScriptScope`]: a refused path ends the run, except a +//! `JSINCLUDE` folder outside the scope, which is passed over. +//! - `#target` takes any value and changes nothing. `#targetengine`, `#engine` and `#strict on` are +//! refused as unsupported, and any other `#name` as unknown; `//@name` with another name stays a +//! comment. +//! - A run reads at most [`MAX_DEPTH`] levels of nested files, [`MAX_FILES`] files, +//! [`MAX_TOTAL_BYTES`] of source in all (the script's own included) and 8 MiB a file, and looks up +//! at most [`MAX_PROBES`] paths. A file may be included again once its earlier inclusion has +//! ended; one that includes itself, directly or not, is an error. +//! +//! The expanded source keeps a [`SourceMap`], so an error is reported at the line and column of +//! the file it is in. + +use std::path::{Path, PathBuf}; +use std::sync::Arc; + +use vectorcraft_js::{DirectiveName, Options}; + +use crate::file_access::{ScriptRead, ScriptScope}; + +/// The most files nested inside each other, below the script. +pub const MAX_DEPTH: usize = 16; +/// The most files a run includes or evaluates. +pub const MAX_FILES: usize = 64; +/// The most bytes of source a run reads, its own included. +pub const MAX_TOTAL_BYTES: usize = 32 << 20; +/// The most paths a run looks up for its files. +pub const MAX_PROBES: usize = 1024; +/// The longest value a preprocessor line may have, in bytes. +const MAX_VALUE_BYTES: usize = 4096; + +/// Where the lines of an expanded source came from: file 0 is the file expanded, the others the +/// files it included. +#[derive(Clone, Debug, Default)] +pub(crate) struct SourceMap { + files: Vec>, + segments: Vec, +} + +/// From line `out_line` of the expanded source on, the lines are those of file `file` from line +/// `line`; on `out_line` itself the columns are `shift` units further right. +#[derive(Clone, Copy, Debug)] +struct Segment { + out_line: u32, + file: usize, + line: u32, + shift: u32, +} + +impl SourceMap { + /// Line `line`, column `column` of the expanded source → the file it is in (its index and name) + /// and the line and column there. + pub(crate) fn locate(&self, line: u32, column: u32) -> (usize, Arc, u32, u32) { + let at = self.segments.partition_point(|s| s.out_line <= line); + let Some(seg) = at.checked_sub(1).and_then(|i| self.segments.get(i)) else { + return (0, self.files.first().cloned().unwrap_or_else(|| Arc::from("")), line, column); + }; + let column = if line == seg.out_line { column.saturating_sub(seg.shift).max(1) } else { column }; + let name = self.files.get(seg.file).cloned().unwrap_or_else(|| Arc::from("")); + (seg.file, name, seg.line.saturating_add(line - seg.out_line), column) + } +} + +/// How a preprocessor failure ends the run. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum FailKind { + /// A line that is not a valid preprocessor line, or a file that is not a valid script. + Syntax, + /// A file that is missing, refused or unreadable, or a cycle. + Error, + /// One of the run's limits. + Limit, +} + +/// A preprocessor failure, at line `line`, column `column` of the file `file` (index `index` of +/// the expansion's [`SourceMap`]). +#[derive(Clone, Debug, PartialEq, Eq)] +pub(crate) struct Failure { + pub kind: FailKind, + pub message: String, + pub index: usize, + pub file: Arc, + pub line: u32, + pub column: u32, +} + +/// A source with its preprocessor lines read: the text to run, where its lines came from, and the +/// name its `#script` line gave. +pub(crate) struct Expanded { + pub source: String, + pub map: SourceMap, + pub name: Option, +} + +/// A file being run: its name for people and the folder its relative paths start from (none for a +/// script given as text). +struct Frame { + shown: Arc, + dir: Option, +} + +/// The files of one run: its scope, the `#includepath` list, what it has read and the files open +/// now, innermost last. +pub(crate) struct Files { + scope: ScriptScope, + jsinclude: Vec, + include_path: Vec, + files: usize, + bytes: usize, + probes: usize, + /// The real paths of the files open now, for cycles. + chain: Vec, + /// How many of them are below the script. + levels: usize, + /// The files `$.evalFile` is running, the script first. + frames: Vec, +} + +/// The output being built: its UTF-16 text, the line it is on, the units since that line began, +/// and whether its last unit is a carriage return. +struct Out { + units: Vec, + line: u32, + column: u32, + after_cr: bool, + map: SourceMap, +} + +impl Out { + fn emit(&mut self, text: &[u16]) { + for &u in text { + match u { + 0x0A if self.after_cr => self.column = 0, + 0x0A | 0x0D | 0x2028 | 0x2029 => { + self.line = self.line.saturating_add(1); + self.column = 0; + } + _ => self.column = self.column.saturating_add(1), + } + self.after_cr = u == 0x0D; + } + self.units.extend_from_slice(text); + } + + /// The lines from here on come from file `file`, from its line `line`. + fn begin(&mut self, file: usize, line: u32) { + self.map.segments.push(Segment { out_line: self.line, file, line, shift: self.column }); + } + + /// Ends the line the output is on, unless it has just ended. + fn end_line(&mut self) { + if self.after_cr || !matches!(self.units.last(), Some(0x0A | 0x2028 | 0x2029)) { + self.emit(&[0x0A]); + } + } +} + +/// A file's text read as a script: a byte order mark is left out, and compiled scripts and bytes +/// that are not UTF-8 are refused. +fn decode(bytes: &[u8], value: &str) -> Result { + if bytes.starts_with(b"@JSXBIN@") { + return Err(format!("script file is compiled: {value}")); + } + let text = std::str::from_utf8(bytes).map_err(|_| format!("script file is not UTF-8 text: {value}"))?; + Ok(text.strip_prefix('\u{feff}').unwrap_or(text).to_string()) +} + +/// `value` without the single or double quotes around it, or `None` when a quote is not closed. +fn unquote(value: &str) -> Option<&str> { + match value.chars().next() { + Some(q @ ('"' | '\'')) => value.get(1..).and_then(|v| v.strip_suffix(q)), + _ => Some(value), + } +} + +/// The folder the file at `path` is in, without Windows' `\\?\` prefix: under that prefix `..` and +/// `/` in the names joined to it would be read as plain characters. +fn folder_of(path: &Path) -> Option { + path.parent().map(|p| crate::cmd::findfiles::plain(p.to_path_buf())) +} + +/// Does the path `p` have a `..` name, with `/` or `\` between names? +fn has_parent(p: &str) -> bool { + p.split(['/', '\\']).any(|name| name == "..") +} + +/// A real path as people read it. +fn display(path: &Path) -> Arc { + Arc::from(crate::cmd::findfiles::plain(path.to_path_buf()).to_string_lossy().as_ref()) +} + +impl Files { + /// The files of a run of the script at `path` (`None` for a script given as text), with the + /// `JSINCLUDE` folders `jsinclude`. + pub(crate) fn new(path: Option<&str>, jsinclude: Vec) -> Self { + let real = path.and_then(|p| std::fs::canonicalize(p).ok()); + let dir = real.as_deref().and_then(folder_of); + Self { + scope: dir.as_deref().map(ScriptScope::of_folder).unwrap_or_default(), + jsinclude, + include_path: Vec::new(), + files: 0, + bytes: 0, + probes: 0, + chain: real.into_iter().collect(), + levels: 0, + frames: vec![Frame { shown: Arc::from(path.unwrap_or_default()), dir }], + } + } + + /// The `JSINCLUDE` folders of this process: absolute folders separated by `;`. + pub(crate) fn jsinclude_env() -> Vec { + #[cfg(test)] + if let Some(list) = super::TEST_JSINCLUDE.with(|t| t.borrow().clone()) { + return list; + } + std::env::var("JSINCLUDE").map(|v| v.split(';').map(str::trim).filter(|p| !p.is_empty()).map(PathBuf::from).collect()).unwrap_or_default() + } + + /// The file being run now, as `$.fileName` gives it (empty for a script given as text). + pub(crate) fn file_name(&self) -> String { + self.frames.last().map(|f| f.shown.to_string()).unwrap_or_default() + } + + /// The `#includepath` list, as `$.includePath` gives it. + pub(crate) fn include_path(&self) -> String { + self.include_path.join(";") + } + + /// The script's own text, `text`, with its preprocessor lines read. + pub(crate) fn expand_script(&mut self, text: &str) -> Result { + self.bytes = self.bytes.saturating_add(text.len()); + let shown = self.frames.first().map(|f| f.shown.clone()).unwrap_or_else(|| Arc::from("")); + let dir = self.frames.first().and_then(|f| f.dir.clone()); + self.expand(shown, dir, text, true) + } + + /// The file `path` names for `$.evalFile`, relative to the file being run now, read and with its + /// preprocessor lines read. It becomes the file being run until [`Files::end_eval`]. + pub(crate) fn eval_file(&mut self, path: &str) -> Result { + let current = self.frames.last().map(|f| f.shown.clone()).unwrap_or_else(|| Arc::from("")); + let fail = |kind, message: String| Failure { kind, message, index: 0, file: current.clone(), line: 0, column: 0 }; + let dir = self.frames.last().and_then(|f| f.dir.clone()); + let (real, bytes) = self.open(path, dir.as_deref(), false).map_err(|(kind, message)| fail(kind, message))?; + let text = self.admit(&real, bytes, path).map_err(|(kind, message)| fail(kind, message))?; + let shown = display(&real); + self.chain.push(real.clone()); + self.levels += 1; + self.frames.push(Frame { shown: shown.clone(), dir: folder_of(&real) }); + let r = self.expand(shown, folder_of(&real), &text, false); + if r.is_err() { + self.end_eval(); + } + r + } + + /// The file `$.evalFile` ran has ended. + pub(crate) fn end_eval(&mut self) { + if self.frames.len() > 1 { + self.frames.pop(); + self.chain.pop(); + self.levels = self.levels.saturating_sub(1); + } + } + + /// Checks a file read at `real` against the run's limits and the files open now, and decodes + /// it. `value` is the path as the script wrote it. + fn admit(&mut self, real: &Path, bytes: Vec, value: &str) -> Result { + if self.chain.iter().any(|p| p == real) { + return Err((FailKind::Error, format!("include cycle: {value}"))); + } + if self.levels >= MAX_DEPTH { + return Err((FailKind::Limit, format!("include depth limit ({MAX_DEPTH}) reached"))); + } + if self.files >= MAX_FILES { + return Err((FailKind::Limit, format!("include file limit ({MAX_FILES}) reached"))); + } + if self.bytes.saturating_add(bytes.len()) > MAX_TOTAL_BYTES { + return Err((FailKind::Limit, "script source limit reached (32 MiB)".to_string())); + } + self.files += 1; + self.bytes += bytes.len(); + decode(&bytes, value).map_err(|m| (FailKind::Error, m)) + } + + /// Finds and reads the file `value` names, from the file in folder `dir`; with `search`, also in + /// the `#includepath` and `JSINCLUDE` folders. + fn open(&mut self, value: &str, dir: Option<&Path>, search: bool) -> Result<(PathBuf, Vec), (FailKind, String)> { + let path = Path::new(value); + let refused = || (FailKind::Error, format!("script file read refused: {value}")); + // A script given as text has no folder, and so no file it may read. + let Some(dir) = dir.filter(|_| !has_parent(value)) else { return Err(refused()) }; + // Each candidate, and whether the scope's refusal passes it over. + let mut candidates: Vec<(Option, bool)> = vec![(Some(dir.join(path)), false)]; + if search && !path.is_absolute() { + candidates.extend(self.include_path.iter().map(|p| ((!has_parent(p)).then(|| dir.join(p).join(path)), false))); + candidates.extend(self.jsinclude.iter().filter(|p| p.is_absolute()).map(|p| (Some(p.join(path)), true))); + } + for (candidate, passed_over_when_refused) in candidates { + let Some(candidate) = candidate else { return Err(refused()) }; + self.probes += 1; + if self.probes > MAX_PROBES { + return Err((FailKind::Limit, format!("include search limit ({MAX_PROBES} paths) reached"))); + } + match self.scope.read(&candidate, crate::script::MAX_SOURCE_BYTES) { + Ok(found) => return Ok(found), + Err(ScriptRead::Missing) => {} + Err(ScriptRead::Refused) if passed_over_when_refused => {} + Err(ScriptRead::Refused) => return Err(refused()), + Err(ScriptRead::TooLarge) => return Err((FailKind::Limit, format!("script file is too large: {value}"))), + Err(ScriptRead::Failed(e)) => return Err((FailKind::Error, format!("cannot read script file {value}: {e}"))), + } + } + Err((FailKind::Error, format!("include file not found: {value}"))) + } + + /// `text`, the file `shown` in folder `dir`, with its preprocessor lines read. With `script`, + /// its `#script` line names the run. + fn expand(&mut self, shown: Arc, dir: Option, text: &str, script: bool) -> Result { + let mut out = Out { units: Vec::with_capacity(text.len()), line: 1, column: 0, after_cr: false, map: SourceMap::default() }; + let mut name = None; + self.file(&mut out, shown, dir.as_deref(), text, script, &mut name)?; + let source = String::from_utf16(&out.units).map_err(|_| Failure { + kind: FailKind::Syntax, + message: "the script is not valid text".into(), + index: 0, + file: out.map.files.first().cloned().unwrap_or_else(|| Arc::from("")), + line: 0, + column: 0, + })?; + Ok(Expanded { source, map: out.map, name }) + } + + /// Adds file `shown` (in folder `dir`, with text `text`) to `out`, its included files in place + /// of their lines. + fn file( + &mut self, + out: &mut Out, + shown: Arc, + dir: Option<&Path>, + text: &str, + script: bool, + name: &mut Option, + ) -> Result<(), Failure> { + let index = out.map.files.len(); + out.map.files.push(shown.clone()); + let units: Vec = text.encode_utf16().collect(); + out.begin(index, 1); + // A source with neither `#` nor `@` has no preprocessor line. + if !units.iter().any(|&u| u == u16::from(b'#') || u == u16::from(b'@')) { + out.emit(&units); + return Ok(()); + } + let fail = |kind, message: String, line, column| Failure { kind, message, index, file: shown.clone(), line, column }; + let options = Options { preprocessor_lines: true, max_source_units: MAX_TOTAL_BYTES, ..Options::default() }; + let program = vectorcraft_js::parse_utf16(&units, &options).map_err(|e| fail(FailKind::Syntax, e.message, e.line, e.column))?; + let mut cursor = 0usize; + for d in &program.directives { + let (start, end) = (d.span.start as usize, d.span.end as usize); + let line_start = + units.get(..start).and_then(|u| u.iter().rposition(|&c| matches!(c, 0x0A | 0x0D | 0x2028 | 0x2029))).map_or(0, |i| i + 1); + let column = u32::try_from(start - line_start + 1).unwrap_or(u32::MAX); + let at = |kind, message: String| fail(kind, message, d.line, column); + out.emit(units.get(cursor..start).unwrap_or_default()); + cursor = end; + let label = format!("{}{}", d.form, d.name); + if d.value.len() > MAX_VALUE_BYTES { + return Err(at(FailKind::Syntax, format!("the value of {label} is too long"))); + } + let value = unquote(&d.value).ok_or_else(|| at(FailKind::Syntax, format!("unterminated quote in {label}")))?; + match d.known { + Some(DirectiveName::Include) => { + if value.is_empty() { + return Err(at(FailKind::Syntax, format!("{label} needs a file name"))); + } + let (real, bytes) = self.open(value, dir, true).map_err(|(kind, message)| at(kind, message))?; + let included = self.admit(&real, bytes, value).map_err(|(kind, message)| at(kind, message))?; + self.chain.push(real.clone()); + self.levels += 1; + let r = self.file(out, display(&real), folder_of(&real).as_deref(), &included, false, name); + self.chain.pop(); + self.levels -= 1; + r?; + out.end_line(); + out.begin(index, d.line); + } + Some(DirectiveName::IncludePath) => { + self.include_path = value.split(';').map(str::trim).filter(|p| !p.is_empty()).map(str::to_string).collect(); + } + Some(DirectiveName::Script) => { + if value.is_empty() { + return Err(at(FailKind::Syntax, format!("{label} needs a name"))); + } + if script && index == 0 { + *name = Some(value.to_string()); + } + } + Some(DirectiveName::Target) => { + if value.is_empty() { + return Err(at(FailKind::Syntax, format!("{label} needs a value"))); + } + } + Some(DirectiveName::Strict) => match value.to_ascii_lowercase().as_str() { + "off" => {} + "on" => return Err(at(FailKind::Syntax, format!("unsupported strict checking: {label} on"))), + _ => return Err(at(FailKind::Syntax, format!("{label} takes on or off"))), + }, + Some(DirectiveName::TargetEngine) => return Err(at(FailKind::Syntax, format!("unsupported script directive: {label}"))), + None if d.name.eq_ignore_ascii_case("engine") => { + return Err(at(FailKind::Syntax, format!("unsupported script directive: {label}"))); + } + None => return Err(at(FailKind::Syntax, format!("unknown script directive: {label}"))), + } + } + out.emit(units.get(cursor..).unwrap_or_default()); + Ok(()) + } +} diff --git a/crates/engine/src/script/mod.rs b/crates/engine/src/script/mod.rs index 7c8edfbcf..f2e4cc5ba 100644 --- a/crates/engine/src/script/mod.rs +++ b/crates/engine/src/script/mod.rs @@ -23,12 +23,18 @@ //! and the 64 levels, 16 MiB and 1,048,576 entries of [`vectorcraft_js::DataLimits`]'s default, //! and the console's ([`console`]). //! +//! Before a run starts, its preprocessor lines are read ([`include`]): included files take the +//! place of their lines, and an error in one is reported at its own file, line and column. +//! `$.evalFile(path)` runs another file in the same global environment and gives its last value, +//! `$.fileName` is the file being run and `$.includePath` the `#includepath` list. +//! //! A run writes to the Script Console with `$.write` and `$.writeln`, and `vc.log` adds a line //! there too. The session's console gets a start line, the run's output, its uncaught error, an end //! line and a report line; the run's report holds its own output and error, its time, its host //! calls by kind and its slowest commands. pub mod console; +pub mod include; use std::cell::{Cell, RefCell}; use std::collections::HashMap; @@ -133,7 +139,11 @@ pub(crate) fn refused_text(id: &str) -> String { /// A request to run a script. pub(crate) struct Request { + /// The run's name; unless `name_given`, a `#script` line in the script replaces it. pub name: String, + pub name_given: bool, + /// The file the script was read from, which its included files are read beside. + pub file: Option, pub source: String, pub args: Map, pub fuel: u64, @@ -470,8 +480,20 @@ struct Host { commands: RefCell>, /// The time the run's slices took, in nanoseconds. run_nanos: Cell, + /// The files the run reads: its includes and `$.evalFile`'s. + files: RefCell, + /// Where the lines of the script's expanded source came from. + script_map: RefCell, + /// The same for each source `$.evalFile` gave, in order. + eval_maps: RefCell>, + /// The last error `$.evalFile` raised for a file it could not run, with the file, line and + /// column that error is at. + eval_failure: RefCell>, } +/// An error `$.evalFile` raised, as thrown, and the file, line and column it is at. +type EvalFailure = (String, Arc, u32, u32); + /// A run's host calls by kind, as its report gives them in `hostCallCounts`. #[derive(Default)] struct Counts { @@ -901,6 +923,8 @@ fn each_line(text: &str, mut done: impl FnMut(String, bool)) { /// The `$` object: `$.write(...values)` writes the values to the Script Console as `String(value)` /// gives them, one after another, and `$.writeln(...values)` does the same and ends the line. +/// `$.evalFile(path[, timeout])` runs a file and gives its last value, `$.fileName` is the file +/// being run and `$.includePath` the `#includepath` list. fn dollar_object(host: &Rc) -> Vec<(String, HostItem)> { let writer = |end: bool| { let h = host.clone(); @@ -912,7 +936,67 @@ fn dollar_object(host: &Rc) -> Vec<(String, HostItem)> { Ok(Data::Undefined) })) }; - vec![("write".into(), writer(false)), ("writeln".into(), writer(true))] + let getter = |read: fn(&include::Files) -> String| { + let h = host.clone(); + HostItem::Getter(function(move |_| { + let text = read(&h.files.borrow()); + h.charge_call(text.len() as u64 + 2)?; + Ok(Data::String(text)) + })) + }; + let eval_file = { + let h = host.clone(); + let end = host.clone(); + HostItem::Evaluator { source: function(move |a| eval_file(&h, a)), end: Rc::new(move || end.files.borrow_mut().end_eval()) } + }; + vec![ + ("write".into(), writer(false)), + ("writeln".into(), writer(true)), + ("evalFile".into(), eval_file), + ("fileName".into(), getter(include::Files::file_name)), + ("includePath".into(), getter(include::Files::include_path)), + ] +} + +/// `$.evalFile(path[, timeout])`: the source of the file `path` names, relative to the file being +/// run, with its preprocessor lines read, which the interpreter then runs. `timeout`, in +/// milliseconds, is taken and not waited on: the file is read at once. +fn eval_file(host: &Host, args: Vec) -> std::result::Result { + let Some(Data::String(path)) = args.first().filter(|p| !matches!(p, Data::String(s) if s.is_empty())) else { + return Err(HostError::Type(EVAL_NO_PATH.to_string())); + }; + match args.get(1) { + None | Some(Data::Undefined) => {} + Some(Data::Number(ms)) if ms.is_finite() && *ms >= 0.0 => {} + Some(Data::Number(_)) => return Err(HostError::Range(EVAL_TIMEOUT.to_string())), + Some(_) => return Err(HostError::Type(EVAL_TIMEOUT.to_string())), + } + if args.len() > 2 { + return Err(HostError::Type(EVAL_ARGUMENTS.to_string())); + } + host.charge_call(path.len() as u64 + 2)?; + let expanded = host.files.borrow_mut().eval_file(path).map_err(|f| { + let message = if f.line > 0 { format!("{}:{}:{}: {}", f.file, f.line, f.column, f.message) } else { f.message }; + if f.line > 0 { + *host.eval_failure.borrow_mut() = Some((message.clone(), f.file.clone(), f.line, f.column)); + } + match f.kind { + include::FailKind::Limit => HostError::Abort(message), + include::FailKind::Syntax => HostError::Syntax(message), + include::FailKind::Error => HostError::Error(message), + } + })?; + // A file without preprocessor lines is parsed here first, so its syntax error names its place. + let options = vectorcraft_js::Options { max_source_units: include::MAX_TOTAL_BYTES, ..vectorcraft_js::Options::default() }; + if let Err(e) = vectorcraft_js::parse(&expanded.source, &options) { + host.files.borrow_mut().end_eval(); + let (_, file, line, column) = expanded.map.locate(e.line, e.column); + let message = format!("{file}:{line}:{column}: {}", e.message); + *host.eval_failure.borrow_mut() = Some((message.clone(), file, line, column)); + return Err(HostError::Syntax(message)); + } + host.eval_maps.borrow_mut().push(expanded.map); + Ok(Data::String(expanded.source)) } fn function(f: impl Fn(Vec) -> std::result::Result + 'static) -> HostFunction { @@ -976,21 +1060,67 @@ fn kind_id(kind: &RunErrorKind) -> &'static str { } /// `message`, with what to do about a failure that comes from a script written for another program. -fn explained(kind: &RunErrorKind, message: &str, source: &str, line: u32) -> String { +fn explained(kind: &RunErrorKind, message: &str) -> String { const HOST_OBJECTS: &[&str] = &["app", "alert", "confirm", "prompt", "File", "Folder", "Window", "BridgeTalk", "activeDocument", "documents"]; let names_host_object = message.strip_prefix("ReferenceError: ").is_some_and(|m| HOST_OBJECTS.iter().any(|o| m.split_whitespace().next() == Some(o))); - let directive = - *kind == RunErrorKind::Syntax && line > 0 && source.lines().nth(line as usize - 1).is_some_and(|l| l.trim_start().starts_with('#')); if *kind == RunErrorKind::Thrown && names_host_object { format!("{message} (scripts here use the `vc` object; objects other applications give their scripts are not available)") - } else if directive { - format!("{message} (preprocessor lines such as `#target` and `#include` are not read yet)") } else { message.to_string() } } +/// The error of a run that stopped in its preprocessor lines (`f`), as the realm gives its own. +fn preprocessor_error(f: include::Failure) -> (RunError, Option>) { + let (kind, message) = match f.kind { + include::FailKind::Syntax if f.line > 0 => { + (RunErrorKind::Syntax, format!("SyntaxError: {} (line {}, column {})", f.message, f.line, f.column)) + } + include::FailKind::Syntax => (RunErrorKind::Syntax, format!("SyntaxError: {}", f.message)), + include::FailKind::Error => (RunErrorKind::Thrown, format!("Error: {}", f.message)), + include::FailKind::Limit => (RunErrorKind::Limit(LimitKind::Host), f.message), + }; + let file = (f.index > 0).then_some(f.file); + (RunError { kind, message, line: f.line, column: f.column }, file) +} + +/// `e`, which the realm raised in source `tag` ([`Realm::error_source`]), at the line and column of +/// the file it is in, with that file's name unless it is the script's own. +fn located(host: &Host, tag: u32, e: &RunError) -> (RunError, Option>) { + let map = |m: &include::SourceMap, own: bool| { + let (index, file, line, column) = m.locate(e.line, e.column); + let message = if e.kind == RunErrorKind::Syntax { relocated(&e.message, e.line, e.column, line, column) } else { e.message.clone() }; + (RunError { kind: e.kind.clone(), message, line, column }, (!own || index > 0).then_some(file)) + }; + // An error `$.evalFile` raised for a file it could not run is at that file's place. + if let Some((_, file, line, column)) = host.eval_failure.borrow().as_ref().filter(|(thrown, ..)| e.message.contains(thrown.as_str())) { + let message = if e.kind == RunErrorKind::Syntax { relocated(&e.message, e.line, e.column, *line, *column) } else { e.message.clone() }; + return (RunError { kind: e.kind.clone(), message, line: *line, column: *column }, Some(file.clone())); + } + if e.line == 0 { + return (e.clone(), None); + } + match tag { + 0 => map(&host.script_map.borrow(), true), + n if n != vectorcraft_js::UNKNOWN_SOURCE => match host.eval_maps.borrow().get(n as usize - 1) { + Some(m) => map(m, false), + None => (e.clone(), None), + }, + _ => (e.clone(), None), + } +} + +/// A syntax error's `message`, which ends with its place `(line L, column C)`, with the place +/// given as `line`, `column`. +fn relocated(message: &str, old_line: u32, old_column: u32, line: u32, column: u32) -> String { + let old = format!(" (line {old_line}, column {old_column})"); + match message.strip_suffix(&old) { + Some(m) => format!("{m} (line {line}, column {column})"), + None => message.to_string(), + } +} + impl Session { /// Is a script running? While one is, the commands that change anything are refused. pub fn script_running(&self) -> bool { @@ -1093,8 +1223,8 @@ impl Session { Ok(()) } - /// Begins a run of `request`. - pub(crate) fn script_start(&mut self, request: Request) -> Result<()> { + /// Begins a run of `request`. → the run's name. + pub(crate) fn script_start(&mut self, request: Request) -> Result { self.settle_for_script()?; // In a build where a fault ends the program, every unsaved document has a copy first. if self.script_copy_first && self.docs.iter().any(DocState::is_dirty) && cmd::recovery::save_copies(self).is_err() { @@ -1105,8 +1235,14 @@ impl Session { self.script_high_water.retain(|uid, _| open.contains(uid)); self.script_stop.clear(); self.script_report = None; + let mut files = include::Files::new(request.file.as_deref(), include::Files::jsinclude_env()); + let expanded = files.expand_script(&request.source); + let name = match &expanded { + Ok(include::Expanded { name: Some(name), .. }) if !request.name_given => name.clone(), + _ => request.name, + }; let limits = Limits { fuel: request.fuel, heap_bytes: HEAP_BYTES, ..Limits::default() }; - let options = Options { max_source_units: MAX_SOURCE_BYTES.saturating_mul(2), ..Options::default() }; + let options = Options { max_source_units: include::MAX_TOTAL_BYTES.saturating_mul(2), ..Options::default() }; let mut realm = Realm::with_options(options, limits); realm.share_interrupt(&self.script_stop); realm.set_data_limits(DataLimits::default()); @@ -1128,34 +1264,45 @@ impl Session { log_bytes: Cell::new(0), written: RefCell::new(Vec::new()), run: self.script_runs, - name: console::line_name(&request.name), + name: console::line_name(&name), console: RefCell::default(), pending: RefCell::default(), counts: Counts::default(), commands: RefCell::default(), run_nanos: Cell::new(0), + files: RefCell::new(files), + script_map: RefCell::default(), + eval_maps: RefCell::default(), + eval_failure: RefCell::default(), }); let args = to_data(&Value::Object(request.args), &DataLimits::default(), 0, &mut (0, 0)) .map_err(|place| EngineError::BadParams { cmd: "script.run".into(), msg: format!("`args{place}` is nested more than 64 levels deep") })?; let line = console::ConsoleLine { run: host.run, name: host.name.clone(), entry: console::Entry::Start }; self.script_console.push(line, false); - let started = realm - .define_host_object("vc", vc_object(&host, args)) - .and_then(|()| realm.define_host_object("$", dollar_object(&host))) - .and_then(|()| realm.start(&request.source)); - if let Err(e) = started { - let mut report = failure_report(&request.name, &e, &request.source, &host, 0, 0); + let started = match expanded { + Err(f) => Err(preprocessor_error(f)), + Ok(expanded) => { + *host.script_map.borrow_mut() = expanded.map; + realm + .define_host_object("vc", vc_object(&host, args)) + .and_then(|()| realm.define_host_object("$", dollar_object(&host))) + .and_then(|()| realm.start(&expanded.source)) + .map_err(|e| located(&host, realm.error_source(), &e)) + } + }; + if let Err((e, file)) = started { + let mut report = failure_report(&name, &e, file.as_deref(), &host, 0, 0); close_run(self, &host, &mut report, ending(&e.kind)); let message = failure_text(&report); self.script_report = Some(report); return Err(EngineError::Other(message)); } let scope = crate::file_access::ScopeHandle::capture(); - let job = Job { realm, host, world: World::working(self), start_uids: open, name: request.name, scope }; + let job = Job { realm, host, world: World::working(self), start_uids: open, name: name.clone(), scope }; let id = NEXT_JOB.fetch_add(1, std::sync::atomic::Ordering::Relaxed); JOBS.with(|jobs| jobs.borrow_mut().insert(id, job)); self.script_key = Some(JobKey(id)); - Ok(()) + Ok(name) } /// Advances the running script by one slice. `None` while it is still running; otherwise the @@ -1196,7 +1343,8 @@ impl Session { fn script_failed(&mut self, job: Job, e: &RunError) -> Result { let fuel_left = job.realm.fuel_left(); - let mut report = failure_report(&job.name, e, "", &job.host, fuel_left, job.realm.limits().fuel.saturating_sub(fuel_left)); + let (e, file) = located(&job.host, job.realm.error_source(), e); + let mut report = failure_report(&job.name, &e, file.as_deref(), &job.host, fuel_left, job.realm.limits().fuel.saturating_sub(fuel_left)); close_run(self, &job.host, &mut report, ending(&e.kind)); drop(job); raise_ids(self); @@ -1310,13 +1458,16 @@ const TOO_MANY_ARGUMENTS: &str = "vc.run: takes a command id and an object of pa const PARAMS_NOT_OBJECT: &str = "vc.run: the parameters must be an object"; const LOG_FULL: &str = "vc.log: the log of this run is full"; const NO_RECOVERY_COPY: &str = "a recovery copy could not be written, so the script was not run"; +const EVAL_NO_PATH: &str = "$.evalFile requires a file path"; +const EVAL_TIMEOUT: &str = "$.evalFile: the timeout must be a finite number of milliseconds, 0 or more"; +const EVAL_ARGUMENTS: &str = "$.evalFile: takes a file path and a timeout"; -/// The report of a run that failed. -fn failure_report(name: &str, e: &RunError, source: &str, host: &Host, fuel_left: u64, fuel_used: u64) -> Value { - json!({ +/// The report of a run that failed; `file` is the included file the error is in. +fn failure_report(name: &str, e: &RunError, file: Option<&str>, host: &Host, fuel_left: u64, fuel_used: u64) -> Value { + let mut report = json!({ "ok": false, "kind": kind_id(&e.kind), - "message": explained(&e.kind, &e.message, source, e.line), + "message": explained(&e.kind, &e.message), "name": name, "line": e.line, "column": e.column, @@ -1325,7 +1476,11 @@ fn failure_report(name: &str, e: &RunError, source: &str, host: &Host, fuel_left "fuelLeft": fuel_left, "fuelUsed": fuel_used, "hostCalls": host.calls.get(), - }) + }); + if let (Some(file), Some(o)) = (file, report.as_object_mut()) { + o.insert("file".into(), json!(file)); + } + report } /// How the run that failed with `kind` ended, as its console end line gives it. @@ -1363,8 +1518,11 @@ fn close_run(s: &mut Session, host: &Host, report: &mut Value, ending: console:: if report["ok"] == false { let (line, column) = (report["line"].as_u64().unwrap_or(0), report["column"].as_u64().unwrap_or(0)); let message = report["message"].as_str().unwrap_or_default(); - let text = - if line > 0 { format!("{}:{line}:{column}: {message}", host.name) } else { format!("{}: location unavailable: {message}", host.name) }; + let text = match report["file"].as_str() { + Some(file) if line > 0 => format!("{}: {file}:{line}:{column}: {message}", host.name), + _ if line > 0 => format!("{}:{line}:{column}: {message}", host.name), + _ => format!("{}: location unavailable: {message}", host.name), + }; each_line(&text, |line, cut| line_to(s, host, console::Entry::Error(line), cut)); } let timing = clock().is_some(); @@ -1388,7 +1546,11 @@ fn close_run(s: &mut Session, host: &Host, report: &mut Value, ending: console:: fn failure_text(report: &Value) -> String { let s = |k: &str| report[k].as_str().unwrap_or_default(); let (line, column) = (report["line"].as_u64().unwrap_or(0), report["column"].as_u64().unwrap_or(0)); - let at = if line > 0 { format!(" line {line}, column {column}") } else { String::new() }; + let at = match report["file"].as_str() { + Some(file) if line > 0 => format!(" line {line}, column {column} of {file}"), + _ if line > 0 => format!(" line {line}, column {column}"), + _ => String::new(), + }; let written = report["written"].as_array().map_or(0, Vec::len); let kept = if written > 0 { let paths: Vec<&str> = report["written"].as_array().into_iter().flatten().filter_map(Value::as_str).collect(); @@ -1456,4 +1618,6 @@ thread_local! { pub(crate) static PANIC_IN_SLICE: Cell = const { Cell::new(false) }; /// Tests: the objects a run may add in place of [`GROWTH_OBJECTS`]. pub(crate) static TEST_GROWTH: Cell> = const { Cell::new(None) }; + /// Tests: the `JSINCLUDE` folders in place of the environment's. + pub(crate) static TEST_JSINCLUDE: RefCell>> = const { RefCell::new(None) }; } diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index a8bf56b72..296b66bca 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -152,6 +152,7 @@ mod saveoptions; mod scalestrokes; mod script_console; mod script_folder; +mod script_include; mod script_run; mod scripts; mod select_ids_first; diff --git a/crates/engine/src/tests/script_include.rs b/crates/engine/src/tests/script_include.rs new file mode 100644 index 000000000..2d857b230 --- /dev/null +++ b/crates/engine/src/tests/script_include.rs @@ -0,0 +1,334 @@ +//! A script's preprocessor lines: `#include` and `//@include` in place, the folders they are looked +//! for in, the run's limits and scope, the other lines, `$.evalFile`, `$.fileName`, and errors at the +//! line of the file they are in. + +use std::path::{Path, PathBuf}; + +use serde_json::{Value, json}; + +use crate::script::TEST_JSINCLUDE; +use crate::*; + +/// A fresh folder for test `name`, with a `main` folder inside it for the script. +fn folder(name: &str) -> PathBuf { + let base = std::env::temp_dir().join(format!("vc-script-include-{name}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&base); + std::fs::create_dir_all(base.join("main")).unwrap(); + base +} + +fn write(path: &Path, text: &str) { + if let Some(dir) = path.parent() { + std::fs::create_dir_all(dir).unwrap(); + } + std::fs::write(path, text).unwrap(); +} + +/// Runs `main/main.jsx` in `base`, with the text `text`. +fn run(base: &Path, text: &str) -> Result { + let main = base.join("main/main.jsx"); + write(&main, text); + Session::new().execute("script.run", &json!({"path": main.to_string_lossy()})) +} + +/// The report of the run that failed with `e`. +fn failed(base: &Path, text: &str) -> Value { + let mut s = Session::new(); + let main = base.join("main/main.jsx"); + write(&main, text); + let e = s.execute("script.run", &json!({"path": main.to_string_lossy()})); + assert!(e.is_err(), "the run failed: {e:?}"); + s.script_report().unwrap().clone() +} + +#[test] +fn script_include_sibling_runs_in_place() { + let base = folder("sibling"); + write(&base.join("main/lib.jsxinc"), "var b = a + 1;"); + write(&base.join("main/two.jsxinc"), "b = b * 10;\r\n"); + let r = run(&base, "var a = 1;\n #include \"lib.jsxinc\"\n//@include 'two.jsxinc'\nvc.log(a + b);").unwrap(); + assert_eq!(r["log"], json!([21])); + // A file may be included again once its earlier inclusion has ended. + let r = run(&base, "var n = 0;\n#include \"three.jsxinc\"\n#include \"three.jsxinc\"\nvc.log(n);"); + assert!(r.is_err(), "three.jsxinc is missing"); + write(&base.join("main/three.jsxinc"), "n++;"); + let r = run(&base, "var n = 0;\n#include \"three.jsxinc\"\n#include three.jsxinc\nvc.log(n);").unwrap(); + assert_eq!(r["log"], json!([2])); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_nested_include_uses_including_folder() { + let base = folder("nested"); + write(&base.join("main/sub/one.jsxinc"), "#include \"two.jsxinc\"\nvar one = two + 1;"); + write(&base.join("main/sub/two.jsxinc"), "var two = 2;"); + write(&base.join("main/two.jsxinc"), "var two = 200;"); + let r = run(&base, "#include \"sub/one.jsxinc\"\nvc.log(one);").unwrap(); + assert_eq!(r["log"], json!([3]), "two.jsxinc beside one.jsxinc, not beside the script"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_missing_include_reports_origin() { + let base = folder("missing"); + let report = failed(&base, "vc.log(1);\n\t#include \"nope.jsxinc\"\n"); + assert_eq!(report["kind"], "thrown"); + assert_eq!((report["line"].as_u64(), report["column"].as_u64()), (Some(2), Some(2))); + assert!(report["message"].as_str().unwrap().contains("include file not found: nope.jsxinc"), "{report}"); + assert_eq!(report["log"], json!([]), "nothing ran before the include was read"); + assert!(report.get("file").is_none(), "the line is the script's own"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_includepath_searches_in_order() { + let base = folder("includepath"); + write(&base.join("main/a/x.jsxinc"), "var x = 'a';"); + write(&base.join("main/b/x.jsxinc"), "var x = 'b';"); + write(&base.join("main/b/y.jsxinc"), "var y = 'b';"); + let r = run(&base, "#includepath \"a; b\"\n#include \"x.jsxinc\"\n#include \"y.jsxinc\"\nvc.log(x + y + $.includePath);").unwrap(); + assert_eq!(r["log"], json!(["aba;b"])); + // A later #includepath replaces the list. + let report = failed(&base, "#includepath \"b\"\n#includepath \"a\"\n#include \"y.jsxinc\""); + assert!(report["message"].as_str().unwrap().contains("include file not found: y.jsxinc"), "{report}"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_include_cycle_and_depth_stop() { + let base = folder("cycle"); + write(&base.join("main/a.jsxinc"), "#include \"b.jsxinc\""); + write(&base.join("main/b.jsxinc"), "\n#include \"a.jsxinc\""); + let report = failed(&base, "#include \"a.jsxinc\""); + assert!(report["message"].as_str().unwrap().contains("include cycle: a.jsxinc"), "{report}"); + assert_eq!(report["line"], 2); + assert!(report["file"].as_str().unwrap().ends_with("b.jsxinc"), "{report}"); + // A script that includes itself. + let report = failed(&base, "#include \"main.jsx\""); + assert!(report["message"].as_str().unwrap().contains("include cycle"), "{report}"); + for i in 0..20 { + write(&base.join(format!("main/d{i}.jsxinc")), &format!("#include \"d{}.jsxinc\"", i + 1)); + } + let report = failed(&base, "#include \"d0.jsxinc\""); + assert_eq!(report["kind"], "limit"); + assert!(report["message"].as_str().unwrap().contains("include depth limit (16) reached"), "{report}"); + assert!(report["file"].as_str().unwrap().ends_with("d15.jsxinc"), "the 17th level is refused: {report}"); + let _ = std::fs::remove_dir_all(base); +} + +/// Makes `link` lead to the folder `target`: a junction on Windows, a symbolic link elsewhere. +/// `false` when the system does not allow it here. +fn link_folder(target: &Path, link: &Path) -> bool { + #[cfg(windows)] + { + let made = std::process::Command::new("cmd").args(["/C", "mklink", "/J"]).arg(link).arg(target).output().is_ok_and(|o| o.status.success()); + made && std::fs::canonicalize(link).ok() == std::fs::canonicalize(target).ok() + } + #[cfg(unix)] + { + std::os::unix::fs::symlink(target, link).is_ok() + } + #[cfg(not(any(unix, windows)))] + { + let _ = (target, link); + false + } +} + +#[test] +fn script_include_link_out_of_the_folder_is_refused() { + let base = folder("junction"); + write(&base.join("outside/x.jsxinc"), "var leaked = 1;"); + // `mklink` reads a `/` in its paths as a switch, so each name is joined on its own. + if !link_folder(&base.join("outside"), &base.join("main").join("away")) { + eprintln!("skipped: a folder link can't be made here"); + let _ = std::fs::remove_dir_all(base); + return; + } + assert!(std::fs::canonicalize(base.join("main/away/x.jsxinc")).unwrap().starts_with(std::fs::canonicalize(base.join("outside")).unwrap())); + let report = failed(&base, "#include \"away/x.jsxinc\"\nvc.log(leaked);"); + assert!(report["message"].as_str().unwrap().contains("script file read refused: away/x.jsxinc"), "{report}"); + let report = failed(&base, "$.evalFile('away/x.jsxinc');"); + assert!(report["message"].as_str().unwrap().contains("script file read refused: away/x.jsxinc"), "{report}"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_include_parent_component_refused() { + let base = folder("parent"); + write(&base.join("outside.jsxinc"), "var leaked = 1;"); + write(&base.join("main/sub/x.jsxinc"), "var x = 1;"); + let report = failed(&base, "#include \"../outside.jsxinc\""); + assert!(report["message"].as_str().unwrap().contains("script file read refused: ../outside.jsxinc"), "{report}"); + // Refused even where the path would come back inside the folder. + let report = failed(&base, "#include \"sub/../sub/x.jsxinc\""); + assert!(report["message"].as_str().unwrap().contains("script file read refused"), "{report}"); + let report = failed(&base, "#includepath \"..\"\n#include \"outside.jsxinc\""); + assert!(report["message"].as_str().unwrap().contains("script file read refused: outside.jsxinc"), "{report}"); + let absolute = base.join("outside.jsxinc").to_string_lossy().replace('\\', "/"); + let report = failed(&base, &format!("#include \"{absolute}\"")); + assert!(report["message"].as_str().unwrap().contains("script file read refused"), "an absolute path outside: {report}"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_jsinclude_outside_grants_ignored() { + let base = folder("jsinclude"); + write(&base.join("outside/x.jsxinc"), "var x = 'outside';"); + write(&base.join("main/lib/y.jsxinc"), "var y = 'lib';"); + TEST_JSINCLUDE.with(|t| *t.borrow_mut() = Some(vec![base.join("outside"), base.join("main/lib")])); + let report = failed(&base, "#include \"x.jsxinc\""); + let r = run(&base, "#include \"y.jsxinc\"\nvc.log(y);"); + TEST_JSINCLUDE.with(|t| *t.borrow_mut() = None); + assert!(report["message"].as_str().unwrap().contains("include file not found: x.jsxinc"), "passed over, not refused: {report}"); + assert_eq!(r.unwrap()["log"], json!(["lib"]), "a JSINCLUDE folder inside the script's folder is searched"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_target_other_app_does_not_route() { + let mut s = Session::new(); + let r = s.execute("script.run", &json!({"source": "#target someapp-12\n#target \"other\"\n//@target x\n#strict off\nvc.log(1);"})).unwrap(); + assert_eq!(r["log"], json!([1])); + for (source, message) in [ + ("#target\nvc.log(1);", "#target needs a value"), + ("#targetengine \"session\"", "unsupported script directive: #targetengine"), + ("#engine \"main\"", "unsupported script directive: #engine"), + ("#strict on", "unsupported strict checking: #strict on"), + ("#strict maybe", "#strict takes on or off"), + ("#frobnicate 1", "unknown script directive: #frobnicate"), + ("#include \"x.jsxinc", "unterminated quote in #include"), + ("#include \"x.jsxinc\"", "script file read refused: x.jsxinc"), + ] { + let e = s.execute("script.run", &json!({"source": format!("vc.log(0);\n{source}")})).unwrap_err().to_string(); + assert!(e.contains(message), "{source}: {e}"); + assert_eq!(s.script_report().unwrap()["line"], 2, "{source}"); + assert_eq!(s.script_report().unwrap()["log"], json!([]), "{source}"); + } + // An unknown `//@` name stays a comment. + assert_eq!(s.execute("script.run", &json!({"source": "//@frobnicate 1\nvc.log(2);"})).unwrap()["log"], json!([2])); +} + +#[test] +fn script_name_comes_from_the_script_line() { + let mut s = Session::new(); + let r = s.execute("script.run", &json!({"source": "#script \"First\"\n#script Second\nvc.log($.fileName);"})).unwrap(); + assert_eq!((r["name"].as_str(), r["log"].clone()), (Some("Second"), json!([""]))); + let r = s.execute("script.run", &json!({"source": "#script Second\n", "name": "given.js"})).unwrap(); + assert_eq!(r["name"], "given.js", "a name given to script.run wins"); + let base = folder("name"); + write(&base.join("main/lib.jsxinc"), "#script Inner\n"); + let r = run(&base, "#include \"lib.jsxinc\"").unwrap(); + assert_eq!(r["name"], "main.jsx", "an included file's #script line does not rename the run"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_strict_on_is_refused() { + let mut s = Session::new(); + let e = s.execute("script.run", &json!({"source": "#strict on\nvc.log(1);"})).unwrap_err().to_string(); + assert!(e.contains("unsupported strict checking: #strict on"), "{e}"); + assert_eq!(s.script_report().unwrap()["kind"], "syntax"); +} + +#[test] +fn script_eval_file_returns_last_value_and_shares_globals() { + let base = folder("evalfile"); + write(&base.join("main/sub/e.jsx"), "var inner = $.fileName;\nfunction h(x) { return x * g; }\n#include \"f.jsxinc\"\ng + f;"); + write(&base.join("main/sub/f.jsxinc"), "var f = 40;"); + write(&base.join("main/empty.jsx"), ""); + let r = run( + &base, + "var g = 2;\nvar r = $.evalFile('sub/e.jsx', 1000);\nvc.log(r);\nvc.log(h(3));\nvc.log(/e\\.jsx$/.test(inner));\nvc.log(/main\\.jsx$/.test($.fileName));\nvc.log($.evalFile('empty.jsx') === undefined);", + ) + .unwrap(); + assert_eq!(r["log"], json!([42, 6, true, true, true])); + // Its errors can be caught; a file is evaluated again after it has returned. + let r = run( + &base, + "var g = 1; var out = [];\ntry { $.evalFile(''); } catch (e) { out.push(e.name); }\ntry { $.evalFile('nope.jsx'); } catch (e) { out.push(String(e)); }\ntry { $.evalFile('sub/e.jsx', -1); } catch (e) { out.push(e.name); }\ntry { $.evalFile('sub/e.jsx', Infinity); } catch (e) { out.push(e.name); }\nout.push($.evalFile('sub/e.jsx') + $.evalFile('sub/e.jsx'));\nvc.log(out);", + ) + .unwrap(); + assert_eq!(r["log"], json!([["TypeError", "Error: include file not found: nope.jsx", "RangeError", "TypeError", 82]])); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_eval_file_outside_scope_refused() { + let base = folder("evaloutside"); + write(&base.join("outside.jsx"), "1;"); + let absolute = base.join("outside.jsx").to_string_lossy().replace('\\', "/"); + let script = format!( + "var out = [];\ntry {{ $.evalFile('../outside.jsx'); }} catch (e) {{ out.push(String(e)); }}\ntry {{ $.evalFile('{absolute}'); }} catch (e) {{ out.push(String(e)); }}\nvc.log(out);" + ); + let r = run(&base, &script).unwrap(); + let out = r["log"][0].as_array().unwrap(); + assert_eq!(out.len(), 2); + for e in out { + assert!(e.as_str().unwrap().starts_with("Error: script file read refused"), "{e}"); + } + // A script given as text has no folder to read from. + let e = Session::new().execute("script.run", &json!({"source": "$.evalFile('x.jsx');"})).unwrap_err().to_string(); + assert!(e.contains("script file read refused: x.jsx"), "{e}"); + let _ = std::fs::remove_dir_all(base); +} + +#[test] +fn script_directives_in_strings_and_block_comments_are_inert() { + let mut s = Session::new(); + let source = "#target x\nvar a = '#include \"nope.jsxinc\"';\n/*\n#include \"nope.jsxinc\"\n//@include \"nope.jsxinc\"\n*/\nvar b = 1; // #include \"nope.jsxinc\"\nvar c = 2; #include\nvc.log(a.length + b);"; + let e = s.execute("script.run", &json!({"source": source})).unwrap_err().to_string(); + // `#` after code on a line is not a preprocessor line: the script has a syntax error there. + assert!(e.contains("line 8"), "{e}"); + let source = source.replace("var c = 2; #include\n", ""); + assert_eq!(s.execute("script.run", &json!({"source": source})).unwrap()["log"], json!([23])); +} + +#[test] +fn script_included_error_has_local_file_and_line() { + let base = folder("errors"); + write(&base.join("main/lib.jsxinc"), "var x = 1;\n\n throw new Error('boom');\n"); + write(&base.join("main/bad.jsxinc"), "var y = 1;\nvar = 2;\n"); + write(&base.join("main/five.jsxinc"), "1;\n2;\n3;\n4;\n5;"); + write(&base.join("main/sub/thrower.jsx"), "\n\nthrow new Error('deep');"); + let mut s = Session::new(); + let report = failed(&base, "var z = 0;\n#include \"lib.jsxinc\""); + assert_eq!((report["line"].as_u64(), report["column"].as_u64()), (Some(3), Some(5)), "{report}"); + assert!(report["file"].as_str().unwrap().ends_with("lib.jsxinc"), "{report}"); + let report = failed(&base, "#include \"bad.jsxinc\""); + assert_eq!((report["kind"].as_str(), report["line"].as_u64()), (Some("syntax"), Some(2)), "{report}"); + assert!(report["file"].as_str().unwrap().ends_with("bad.jsxinc"), "{report}"); + // A line of the script after an include keeps its own number. + let report = failed(&base, "#include \"five.jsxinc\"\n throw new Error('after');"); + assert_eq!((report["line"].as_u64(), report["column"].as_u64()), (Some(2), Some(3)), "{report}"); + assert!(report.get("file").is_none(), "{report}"); + let report = failed(&base, "#include \"five.jsxinc\"\nvar = 1;"); + assert_eq!((report["kind"].as_str(), report["line"].as_u64()), (Some("syntax"), Some(2)), "{report}"); + // An included file that ends in a carriage return does not join the script's next line ending. + write(&base.join("main/cr.jsxinc"), "1;\r2;\r"); + let report = failed(&base, "#include \"cr.jsxinc\"\n\n throw new Error('after');"); + assert_eq!((report["line"].as_u64(), report["column"].as_u64()), (Some(3), Some(3)), "{report}"); + assert!(report.get("file").is_none(), "{report}"); + let report = failed(&base, "#include \"cr.jsxinc\"\r\nvar = 1;"); + assert_eq!((report["kind"].as_str(), report["line"].as_u64()), (Some("syntax"), Some(2)), "{report}"); + // An uncaught error in a file $.evalFile ran is at that file's line. + let report = failed(&base, "var a = 1;\n$.evalFile('sub/thrower.jsx');"); + assert_eq!(report["line"], 3, "{report}"); + assert!(report["file"].as_str().unwrap().ends_with("thrower.jsx"), "{report}"); + // So is a file $.evalFile could not run: a syntax error, or a bad preprocessor line. + write(&base.join("main/sub/broken.jsx"), "var a = 1;\n\nvar = 2;"); + write(&base.join("main/sub/directive.jsx"), "\n#frobnicate 1"); + for (file, line) in [("broken.jsx", 3), ("directive.jsx", 2)] { + let report = failed(&base, &format!("var a = 1;\n$.evalFile('sub/{file}');")); + assert_eq!(report["line"], line, "{report}"); + assert!(report["file"].as_str().unwrap().ends_with(file), "{report}"); + } + // The console's error line names the file. + let main = base.join("main/main.jsx"); + write(&main, "#include \"lib.jsxinc\""); + assert!(s.execute("script.run", &json!({"path": main.to_string_lossy()})).is_err()); + let console = s.execute("script.console", &json!({})).unwrap(); + let error = console["lines"].as_array().unwrap().iter().find(|l| l["kind"] == "error").unwrap()["text"].as_str().unwrap().to_string(); + assert!(error.starts_with("main.jsx: ") && error.contains("lib.jsxinc:3:5: Error: boom"), "{error}"); + let _ = std::fs::remove_dir_all(base); +} diff --git a/crates/engine/src/tests/script_run.rs b/crates/engine/src/tests/script_run.rs index eba254116..8294aa638 100644 --- a/crates/engine/src/tests/script_run.rs +++ b/crates/engine/src/tests/script_run.rs @@ -166,7 +166,7 @@ fn an_error_names_the_script_its_place_and_kind() { let e = run(&mut s, "var = 1;").unwrap_err().to_string(); assert!(e.starts_with("t.js: line 1,") && e.ends_with("(syntax)"), "{e}"); assert!(run(&mut s, "app.activeDocument;").unwrap_err().to_string().contains("`vc`")); - assert!(run(&mut s, "#target x\n1").unwrap_err().to_string().contains("#target")); + assert!(run(&mut s, "#targetengine x\n1").unwrap_err().to_string().contains("unsupported script directive: #targetengine")); assert_eq!(s.script_report().unwrap()["line"], 1); } @@ -989,6 +989,15 @@ fn source_text() -> impl proptest::strategy::Strategy { Just("{".to_string()), Just("try {".to_string()), Just("} catch (e) {}".to_string()), + Just("\n#include \"x.jsxinc\"\n".to_string()), + Just("\n//@include 'x\n".to_string()), + Just("\n#includepath \";;..;\"\n".to_string()), + Just("\n#script \"\"\n".to_string()), + Just("\n#target\n".to_string()), + Just("\n#strict on\n".to_string()), + Just("\n#\u{0}\n".to_string()), + Just("try { $.evalFile('x.jsx', NaN); } catch (e) {}".to_string()), + "\n#[a-z]{0,12}[ -~]{0,20}\n", "[ -~]{0,30}", "\\PC{0,10}", ]; diff --git a/crates/js/src/builtins.rs b/crates/js/src/builtins.rs index d420b94b3..62f6ade7a 100644 --- a/crates/js/src/builtins.rs +++ b/crates/js/src/builtins.rs @@ -472,7 +472,7 @@ fn function_ctor(interp: &mut Interp, _: Value, args: &[Value], _: bool) -> JsRe Ok(p) => p, Err(e) => return Err(interp.throw(ErrorType::Syntax, &e.message)), }; - let source = Rc::new(Source { lines: program.lines.clone(), text: units.into() }); + let source = Rc::new(Source { lines: program.lines.clone(), text: units.into(), tag: interp.source_tag }); let code = match compile_constructed_function(&program, &source) { Ok(Some(code)) => code, Ok(None) => return Err(interp.throw(ErrorType::Syntax, "the parameters or body do not form one function")), @@ -501,7 +501,7 @@ fn function_to_string(interp: &mut Interp, this: Value, _: &[Value], _: bool) -> return Ok(Value::String(Str::from_units(units.to_vec()))); } Some(Func::Native { name, .. }) => format!("function {name}() {{ [native code] }}"), - Some(Func::Host { name, .. } | Func::Data { name, .. }) => format!("function {name}() {{ [native code] }}"), + Some(Func::Host { name, .. } | Func::Data { name, .. } | Func::Eval { name, .. }) => format!("function {name}() {{ [native code] }}"), Some(Func::Bound { .. }) => "function () { [native code] }".to_string(), None => return Err(interp.type_error("Function.prototype.toString was called on a value that is not a function")), }; diff --git a/crates/js/src/bytecode.rs b/crates/js/src/bytecode.rs index b23d9a525..0a666d1f2 100644 --- a/crates/js/src/bytecode.rs +++ b/crates/js/src/bytecode.rs @@ -171,6 +171,8 @@ pub(crate) struct Handler { pub(crate) struct Source { pub(crate) lines: LineTable, pub(crate) text: Rc<[u16]>, + /// Which source this is, as [`crate::Realm::error_source`] gives it. + pub(crate) tag: u32, } #[derive(Debug)] diff --git a/crates/js/src/heap.rs b/crates/js/src/heap.rs index 1399c0570..b229ce6e6 100644 --- a/crates/js/src/heap.rs +++ b/crates/js/src/heap.rs @@ -176,13 +176,15 @@ pub(crate) enum NativeKind { /// What a function object runs. `Bound` is a function made by `Function.prototype.bind` (section /// 15.3.4.5): calling it calls `target` with `this` and `args` before the arguments it is given. /// `Data` is a host function given [`crate::Data`]; with `text`, each argument arrives as the -/// string `String(value)` gives. +/// string `String(value)` gives. `Eval` is a host function that gives source to run as global code +/// ([`crate::HostItem::Evaluator`]); `end` is called when that code ends. #[derive(Clone)] pub(crate) enum Func { Closure { code: Rc, env: EnvId }, Native { f: NativeFn, kind: NativeKind, constructor: bool, name: &'static str }, Host { f: Rc, name: Rc }, Data { f: DataFn, name: Rc, text: bool }, + Eval { f: DataFn, end: Rc, name: Rc }, Bound { target: ObjId, this: Value, args: Rc<[Value]> }, } diff --git a/crates/js/src/host.rs b/crates/js/src/host.rs index 888fb3d95..b725245ce 100644 --- a/crates/js/src/host.rs +++ b/crates/js/src/host.rs @@ -55,6 +55,8 @@ pub enum HostError { Type(String), /// Thrown to the script as a `RangeError`. Range(String), + /// Thrown to the script as a `SyntaxError`. + Syntax(String), /// Ends the run with [`LimitKind::Host`] and this message. The script cannot catch it. Abort(String), } @@ -74,6 +76,16 @@ pub enum HostItem { Getter(HostFunction), /// A frozen object with these members. Object(Vec<(String, HostItem)>), + /// A function whose host function gives JavaScript source as a string, which then runs as + /// global code in the realm, as an indirect `eval` runs it, with the run's fuel and limits: + /// the call gives that code's completion value, and its declarations become globals. A result + /// that is not a string is the call's result. `end` is called when the source has run, + /// normally or not. Each source given has the next tag of [`crate::Realm::error_source`], + /// from 1. + Evaluator { + source: HostFunction, + end: Rc, + }, } enum Step { @@ -280,6 +292,39 @@ impl Interp { /// Calls host function `f`: reads its arguments (with `text`, each as the string `String(value)` /// gives), runs it, and gives its result to the script. pub(crate) fn call_data(&mut self, f: &crate::heap::DataFn, name: &Rc, args: &[Value], text: bool) -> JsResult { + let data = self.read_args(name, args, text)?; + match f(data) { + Ok(d) => self.write_data(d, 0, false), + Err(e) => Err(self.host_error(e)?), + } + } + + /// Calls evaluator `f` ([`HostItem::Evaluator`]): reads its arguments, runs it, and runs the + /// source it gives as global code, as an indirect `eval` does, giving that code's completion + /// value. A result that is not a string is given to the script as it is. `end` is called once + /// the source has run, normally or not. + pub(crate) fn call_evaluator(&mut self, f: &crate::heap::DataFn, end: &Rc, name: &Rc, args: &[Value]) -> JsResult { + let data = self.read_args(name, args, false)?; + let source = match f(data) { + Ok(Data::String(source)) => source, + Ok(d) => return self.write_data(d, 0, false), + Err(e) => return Err(self.host_error(e)?), + }; + self.evaluated = self.evaluated.saturating_add(1); + let tag = std::mem::replace(&mut self.source_tag, self.evaluated); + let code = self.encode_source(&source).and_then(|units| self.compile_source(&units, false, crate::bytecode::CodeKind::Eval)); + self.source_tag = tag; + let r = match code { + Ok(code) => self.run_global(code, true), + Err(e) => Err(self.compile_error_to_abrupt(e)), + }; + end(); + r + } + + /// The arguments of host function `name` as [`Data`] (with `text`, each as the string + /// `String(value)` gives). + fn read_args(&mut self, name: &Rc, args: &[Value], text: bool) -> JsResult> { let mut reading = Reading { limits: self.data_limits, bytes: 0, entries: 0, ancestors: Vec::new(), path: Vec::new(), function: name.clone() }; let mut data = Vec::with_capacity(args.len()); // A conversion can run the script's own `toString`, which may collect garbage. @@ -302,15 +347,20 @@ impl Interp { } self.temp_roots.truncate(mark); read?; - match f(data) { - Ok(d) => self.write_data(d, 0, false), - Err(HostError::Abort(message)) => { + Ok(data) + } + + /// What host error `e` throws, or the limit it ends the run with. + fn host_error(&mut self, e: HostError) -> JsResult { + match e { + HostError::Abort(message) => { self.host_abort = Some(message); - Err(Abrupt::Limit(LimitKind::Host)) + Ok(Abrupt::Limit(LimitKind::Host)) } - Err(HostError::Error(message)) => Err(self.throw_host(ErrorType::Error, &message)?), - Err(HostError::Type(message)) => Err(self.throw_host(ErrorType::Type, &message)?), - Err(HostError::Range(message)) => Err(self.throw_host(ErrorType::Range, &message)?), + HostError::Error(message) => self.throw_host(ErrorType::Error, &message), + HostError::Type(message) => self.throw_host(ErrorType::Type, &message), + HostError::Range(message) => self.throw_host(ErrorType::Range, &message), + HostError::Syntax(message) => self.throw_host(ErrorType::Syntax, &message), } } @@ -361,6 +411,11 @@ impl Interp { HostItem::TextFunction(f) => Prop::Data { value: Value::Object(self.host_function(f, &full, true)), flags: ENUMERABLE }, HostItem::Getter(f) => Prop::Accessor { get: Some(self.host_function(f, &full, false)), set: None, flags: ENUMERABLE }, HostItem::Object(items) => Prop::Data { value: Value::Object(self.host_object(&full, items)?), flags: ENUMERABLE }, + HostItem::Evaluator { source, end } => { + let proto = self.intrinsics.function_proto; + let f = self.heap.alloc_object(Object::new(Class::Function(Func::Eval { f: source, end, name: full.into() }), Some(proto))); + Prop::Data { value: Value::Object(f), flags: ENUMERABLE } + } }; self.define(o, key(&name), prop) } diff --git a/crates/js/src/interp.rs b/crates/js/src/interp.rs index ee711ffb3..22bbb6c0d 100644 --- a/crates/js/src/interp.rs +++ b/crates/js/src/interp.rs @@ -185,6 +185,12 @@ pub(crate) struct Interp { pub(crate) host_abort: Option, /// How much a value passed to or from a host function may carry. pub(crate) data_limits: crate::DataLimits, + /// The tag the source compiled next gets ([`crate::Realm::error_source`]). + pub(crate) source_tag: u32, + /// The sources [`crate::HostItem::Evaluator`] functions have given in this realm. + pub(crate) evaluated: u32, + /// The tag of the source the last run's error was raised in. + pub(crate) error_tag: u32, } /// The bytes all live [`HeldBytes`] hold, and the bytes dropped holders released that the heap's @@ -376,6 +382,9 @@ impl Interp { pause: Rc::default(), host_abort: None, data_limits: crate::DataLimits::default(), + source_tag: crate::realm::UNKNOWN_SOURCE, + evaluated: 0, + error_tag: crate::realm::UNKNOWN_SOURCE, }; crate::builtins::install(&mut interp); interp.fuel = limits.fuel; @@ -1851,6 +1860,7 @@ impl Interp { // keeps itself reachable. return Ok(Prepared::Done(self.call_data(&f, &name, &args, text)?)); } + Func::Eval { f, end, name } => return Ok(Prepared::Done(self.call_evaluator(&f, &end, &name, &args)?)), Func::Host { f: host, .. } => { let js_args: Vec = args.iter().map(|v| self.to_js_value(v)).collect(); return match host(&js_args) { @@ -2123,7 +2133,7 @@ impl Interp { let remaining = self.options.max_source_units.saturating_sub(self.source_units.saturating_sub(units.len())); let options = Options { strict, max_source_units: remaining, ..self.options.clone() }; let program = crate::parse_utf16(units, &options).map_err(CompileError::Syntax)?; - let source = Rc::new(Source { lines: program.lines.clone(), text: units.into() }); + let source = Rc::new(Source { lines: program.lines.clone(), text: units.into(), tag: self.source_tag }); compile_program(&program, &source, kind).map_err(|e| match e { CompileFail::Syntax(s) => CompileError::Syntax(s), CompileFail::Internal(m) => CompileError::Internal(m), diff --git a/crates/js/src/interp_tests.rs b/crates/js/src/interp_tests.rs index 03d9e6639..c7f69ec9b 100644 --- a/crates/js/src/interp_tests.rs +++ b/crates/js/src/interp_tests.rs @@ -774,7 +774,7 @@ fn deep_nesting_compiles_on_a_small_stack() { // ---- the verifier ---- fn code(ops: Vec) -> Code { - let source = Rc::new(Source { lines: crate::LineTable::new(&[]), text: Rc::from(Vec::::new()) }); + let source = Rc::new(Source { lines: crate::LineTable::new(&[]), text: Rc::from(Vec::::new()), tag: 0 }); Code { positions: vec![0; ops.len()], ops, diff --git a/crates/js/src/lib.rs b/crates/js/src/lib.rs index a95a4b884..67ee2712c 100644 --- a/crates/js/src/lib.rs +++ b/crates/js/src/lib.rs @@ -69,7 +69,7 @@ pub use error::{SyntaxError, TOO_DEEP, TOO_LONG}; pub use host::{Data, DataLimits, HostError, HostFunction, HostItem}; pub use lexer::{Goal, Lexer}; pub use number::to_string as number_to_string; -pub use realm::{Interrupt, JsValue, LimitKind, Limits, MAX_UNBROKEN_FUEL, Progress, Realm, RunError, RunErrorKind}; +pub use realm::{Interrupt, JsValue, LimitKind, Limits, MAX_UNBROKEN_FUEL, Progress, Realm, RunError, RunErrorKind, UNKNOWN_SOURCE}; pub use token::{Directive, DirectiveName, Keyword, Punct, Span, Token, TokenKind}; /// How a script is read. diff --git a/crates/js/src/realm.rs b/crates/js/src/realm.rs index a28b150a5..beaad2557 100644 --- a/crates/js/src/realm.rs +++ b/crates/js/src/realm.rs @@ -58,6 +58,10 @@ pub enum LimitKind { Host, } +/// The tag [`Realm::error_source`] gives for a source that is neither the realm's script nor an +/// evaluator's: code that `eval` or `Function` compiled, or an error with no place. +pub const UNKNOWN_SOURCE: u32 = u32::MAX; + /// The fuel [`Realm::advance`] lets a slice spend, past the slice it was asked for, in a nested run /// it cannot pause. pub const MAX_UNBROKEN_FUEL: u64 = 200_000_000; @@ -218,10 +222,17 @@ impl Realm { interp.throw_site = None; interp.last_thrown = Value::Undefined; interp.host_abort = None; + interp.error_tag = UNKNOWN_SOURCE; let strict = interp.options.strict; - match interp.encode_source(source).and_then(|units| interp.compile_source(&units, strict, CodeKind::Global)) { + interp.source_tag = 0; + let code = interp.encode_source(source).and_then(|units| interp.compile_source(&units, strict, CodeKind::Global)); + interp.source_tag = UNKNOWN_SOURCE; + match code { Ok(code) => Ok(code), - Err(CompileError::Syntax(e)) => Err(e.into()), + Err(CompileError::Syntax(e)) => { + interp.error_tag = 0; + Err(e.into()) + } Err(CompileError::Internal(m)) => Err(RunError { kind: RunErrorKind::Internal, message: m, line: 0, column: 0 }), } } @@ -229,7 +240,9 @@ impl Realm { /// The end of a run: its completion value or error, with the interpreter's stacks emptied. fn outcome(&mut self, result: Result) -> Result { let interp = &mut self.interp; - let (line, column) = interp.throw_site.take().map_or((0, 0), |(code, pc)| code.line_column(pc)); + let site = interp.throw_site.take(); + interp.error_tag = site.as_ref().map_or(UNKNOWN_SOURCE, |(code, _)| code.source.tag); + let (line, column) = site.map_or((0, 0), |(code, pc)| code.line_column(pc)); interp.stack.clear(); interp.frames.clear(); interp.temp_roots.clear(); @@ -299,6 +312,13 @@ impl Realm { self.interp.interrupt = interrupt.0.clone(); } + /// Which source the last error's `line` and `column` are in: 0 for the script given to + /// [`Realm::run`] or [`Realm::start`], `n` for the `n`th source a + /// [`crate::HostItem::Evaluator`] gave in this realm, and [`UNKNOWN_SOURCE`] otherwise. + pub fn error_source(&self) -> u32 { + self.interp.error_tag + } + /// Fuel the run has left. pub fn fuel_left(&self) -> u64 { self.interp.fuel diff --git a/docs/mcp.md b/docs/mcp.md index 5f9f29d3d..363f5494a 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -1549,7 +1549,7 @@ describes that menu item. Pass exactly one of `path` and `source`. A `path` must case. A `.jsxbin` file or text that starts with `@JSXBIN@` is compiled and ends with an error, as does text that is not UTF-8. The language is ES5.1, except that identifiers take the characters Unicode's `ID_Start` and `ID_Continue` properties allow, as later editions define them, and may use `\u{...}` escapes. The script reaches the -app through the global object `vc`, and writes to the Script Console through `$`: +app through the global object `vc`, and writes to the Script Console and runs other files through `$`: - `vc.run(id, params)` runs a command and returns its result. A command that fails throws an error the script can catch. The documents and the session state the command changed, preferences included, go back, and any file it @@ -1561,9 +1561,41 @@ app through the global object `vc`, and writes to the Script Console through `$` - `vc.log(value)` adds a value to the report and a line to the Script Console. - `$.write(...values)` writes the values to the Script Console with no separator, each as `String(value)` gives it, and `$.writeln(...values)` does the same and ends the line. Each call counts as a host call. +- `$.evalFile(path, timeout?)` runs the file `path` names, relative to the file being run, in the same global + environment, so its functions and variables stay defined, and returns its last value (`undefined` for an empty + file). Its preprocessor lines are read as the script's are. `timeout` is a number of milliseconds from 0; the file + is read at once and `timeout` is not waited on. The run's fuel and limits apply to the file. An empty path, a + missing file and a refused path throw errors the script can catch. +- `$.fileName` is the file being run, which is the file `$.evalFile` is running while it runs, and is empty for a + script given as `source`. `$.includePath` is the current `#includepath` list, separated by `;`. - `vc.args` is the `args` object. - `vc.api.version` is `"1.0"`, and `vc.api.has(feature)` says whether the API has a feature. +Before any of the script runs, its preprocessor lines are read. The help page +https://extendscript.docsforadobe.dev/extendscript-tools-features/preprocessor-directives/ describes these lines, and +https://extendscript.docsforadobe.dev/extendscript-tools-features/dollar-object/ describes `$.evalFile`, `$.fileName` +and `$.includePath`. A line counts when it starts with `#` or `//@`, after blanks only, outside strings and comments. +VectorCraft reads them by these rules of its own: + +- `#include "file"` and `//@include "file"` put the text of that file in place of the line. A value with spaces or + punctuation goes in single or double quotes. The file is looked for beside the file that includes it, then in each + `#includepath` folder in turn, then in each folder of the `JSINCLUDE` environment variable (absolute folders + separated by `;`). The included file must be a whole script on its own. A file may be included again after its + earlier inclusion has ended, and a file that includes itself, directly or through others, is an error. +- `#includepath "a;b"` sets the folders `#include` looks in, separated by `;`. A relative folder is relative to the + folder of the file that includes. A later `#includepath` replaces the list. +- `#script "name"` names the run in the report and the Script Console, unless `name` is given. A `#script` line in an + included file does not rename the run. +- `#target` takes any value and does not change how the script runs. `#strict off` is taken. +- `#strict on`, `#targetengine` and `#engine` end the run with an unsupported-directive error, and any other `#name` + with an unknown-directive error. A `//@` line with another name is a comment. +- A script run from a file reads its included files and the files `$.evalFile` runs only in that file's folder and + the folders below it. A path with a `..` name, a path that leads out of the folder through a link or a junction, a + Windows device name or stream, and a computer on the network are refused, and so is every path for a script given as + `source`. A `JSINCLUDE` folder outside the script's folder is passed over. +- A missing or refused included file ends the run before any of it runs. An error in an included file or a file `$.evalFile` + ran is reported at that file's line and column, and the report's `file` names the file. + The run is one transaction. When the script ends, each document it changed gets one undo step named Run Script. A run that changes no document adds no undo step. The journal records each command the run finished, as it records commands run one at a time. A preference the run changed takes the run's value, and a preference changed in the app @@ -1590,7 +1622,8 @@ VectorCraft sets these limits itself: | Limit | Value | |-------|-------| -| Source text | 8 MiB | +| Source text | 8 MiB a file, 32 MiB in all | +| Included and evaluated files | 64 in all, 16 levels deep, 1,024 paths looked up | | Heap | 256 MiB (128 MiB on the web) | | Host calls | 2,000,000 | | `maxFuel`, instructions in all | 1 to 100,000,000,000; 1,000,000,000 by default | @@ -1607,7 +1640,8 @@ In the app, a script past the growth limit stops at a question to continue or to with an error at that limit. The headless reply is `{ok: true, name, changed, written, log, fuelUsed, hostCalls, runTimeMs, timingAvailable, hostCallCounts, slowestCommands, console}`. In the app the reply is `{running: true, name}`, and the report appears later as `scriptReport` in `ui.inspect`. The report of a failed run has the same -fields after `kind`, `message`, `line`, `column` and `fuelLeft`. `answers` takes only an empty object, since scripts +fields after `kind`, `message`, `line`, `column`, `file` (when the error is in another file than the script) and +`fuelLeft`. `answers` takes only an empty object, since scripts have no prompts yet. - `runTimeMs` is the time the script ran, its commands included. Waits between frames and for a person's answer are @@ -1626,7 +1660,8 @@ vectorcraft-cli run --cmd script.run --params '{"path": "Tidy Layers.js", "args" ``` Window › Script Console shows the session's script lines, oldest first: each run's start, its `$.write` and -`$.writeln` output, its `vc.log` entries, its uncaught error as `name:line:column: message`, its end, and a report of +`$.writeln` output, its `vc.log` entries, its uncaught error as `name:line:column: message` (or `name: file:line:column: message` when the error is in another +file), its end, and a report of its time, host calls and slowest commands. Each line shows the run's number and the script's name. Line feeds, carriage returns and the two together break lines, and other control characters show as their Unicode Control Pictures. The console keeps at most 2,048 lines and 1 MiB of text and drops the oldest lines past that. A line longer diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index 5ea1b3a4a..384b7a40b 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -729,6 +729,7 @@ Moved from the ROADMAP's "Shipped so far" on 2026-10-10. Update it when a featur - The control server writes each reply through a 64 KiB buffer. On Windows 11 it sends the 17 MB `document.inspect` reply of a document with 50,000 objects on one layer to a client over loopback in 0.09 s instead of 34 s. - Scripts: File › Scripts › Other Script… and `script.run` run a JavaScript file against the open documents as one transaction with one undo step for each changed document. A failed or stopped run puts every document back. `crates/engine/src/tests/script_run.rs`. - Scripts: Window › Script Console shows each run's `$.write` and `$.writeln` output, `vc.log` entries, uncaught error with the script's name, line and column, and a report of its time, host calls and slowest commands, keeping at most 2,048 lines and counting the lines it dropped. `script.run`'s report carries the same data, and `script.console`, `script.console.clear` and `ui.copyScriptConsole` read, clear and copy it. `crates/engine/src/tests/script_console.rs`. + - Scripts: a run reads its preprocessor lines before any of it runs: `#include` and `//@include` put another file in place of the line, looked for beside the including file, in the `#includepath` folders and in `JSINCLUDE`; `#script` names the run; `#target` and `#strict off` are taken, and `#strict on`, `#targetengine` and `#engine` are refused. `$.evalFile` runs a file with the same globals and returns its last value, and `$.fileName` and `$.includePath` give the file being run and the include folders. A script run from a file reads these files only in its folder and below, without `..` names or links that lead out, and an error in one is reported at its own file, line and column ([preprocessor directives](https://extendscript.docsforadobe.dev/extendscript-tools-features/preprocessor-directives/), [the `$` object](https://extendscript.docsforadobe.dev/extendscript-tools-features/dollar-object/); `crates/engine/src/tests/script_include.rs`). - Scripts folder: File › Scripts lists the `.js` and `.jsx` files of the `Scripts` folder next to the preferences, each named after its file without the extension, with each subfolder as a submenu, above Check Script… and Other Script… (https://helpx.adobe.com/illustrator/desktop/automate-visualize-data/automate-actions/install-and-run-scripts.html). VectorCraft's own rules: the folder is read again when the window becomes active and when the File menu opens, links are not followed, and one read stops at 2,048 entries, 512 scripts, 128 subfolders, 4 levels or 250 ms. `script.listInstalled` returns the listing. `crates/engine/src/tests/script_folder.rs`, `crates/ui-egui/src/tests/scripts_menu.rs`. - Parameter keys: the engine refuses a key that a command's params description doesn't name before the command runs, in every build and for every caller, including the app's own calls, the commands a command runs and the steps of `command.batch` and `action.batch`. An action recorded by an older version still plays from the Actions panel and the Batch dialog (`Session::replay`). `crates/engine/tests/it/param_keys.rs`. - MCP server with 25 tools (drawing, text, effects, Pathfinder, transforms, graphs, text wrap, export, screenshots, any command), attached to the running app or headless. Tool descriptions and replies agree with the engine: format lists come from its format table, `save_file` says it keeps the document's own format, `inspect_document` reports the paint type's characters show, and selection exports report the encoder's warnings. `--automation-read-root` / `--automation-write-root` (on `vectorcraft-cli mcp` and on `vectorcraft --control`) confine the files commands read and write to chosen folders, links followed, with the flags PhotoCraft takes (#832); each flag may be given more than once, a path inside any of its folders being allowed (#3546, `file_access::tests::several_roots_of_a_kind_each_grant_their_access`, `cli::mcp_automation_roots_take_several_folders`). A long export can be cancelled (`notifications/cancelled` stops it at the next page or group and writes nothing), reports page progress to a client that sent a progress token, and does not make other requests wait: engine-free ones answer at once, the rest are refused as busy after two seconds (#2165, #2171, #2172).