Commit Graph
151 Commits
Author SHA1 Message Date
dotCooCoo 8a007b03ad Merge main into feat/script-file-folder-write 2026-11-14 19:35:28 -08:00
dotCooCoo a750c997be Scripts: a property name converted from an object stays counted while its getter or setter runs
A computed name such as o[key], where key's toString returns a rope,
is flattened into a copy that only native code held. A getter or
setter that started a collection dropped that copy from the heap
estimate, so the live heap could pass the limit by one name per
nested access. Indexed reads, ++ and -- on an indexed property, and
Object.defineProperty now keep the converted name reachable while the
accessor runs.
2026-11-14 13:51:09 -08:00
dotCooCoo 64662b435b Scripts: a file opened under a withdrawn grant is not read
When a run evaluated code that withdrew a kept grant, a File opened under that grant still gave the bytes it held to read, readln and readch, and seek to the end showed its length. Those calls now give Script file access denied and set error, and eof is true. docs/mcp.md says so.
2026-11-13 20:34:27 -08:00
dotCooCoo 0300f70950 Scripts: file and folder dialogs are native functions that pause the script
File.openDialog, File.saveDialog, Folder.selectDialog and the openDlg, saveDlg and selectDlg methods are now built-ins written in Rust instead of JavaScript source compiled when File and Folder are installed. The interpreter gains a native function form that asks the host for a value: it gives the host call to make and a function that takes the answer. When the host waits, the run pauses at the call, and the answer goes to that function before the next instruction, so a filter function or an exception it throws runs at the dialog's call. Arguments, filters, multiSelect, the start folder and the null and error answers work as before.
2026-11-13 14:45:05 -08:00
dotCooCoo 6644dc4686 Merge main into feat/script-file-folder-write 2026-11-13 11:00:25 -08:00
dotCooCoo b153a3d7a6 Scripts: a file whose close could not write it stays open, so a later close tries again 2026-11-13 10:31:59 -08:00
dotCooCoo dbfffc8ecb Scripts: commands read through the gate in runs the person starts, and the grants file is not written over when it cannot be read
- A command a script runs reads only where the script may, also in a run the person started; only
  reads of the app's own folders inside the command are exempt.
- Always and script.grants.revoke leave a grants file they cannot read as it is, and fail.
- script.grants names a path outside the automation read roots by its file name alone.
- copy reads the source once, within the 8 MiB limit, writes those bytes, and gives the copy the
  source's permissions.
- write joins its values as UTF-16 before making them text, so a surrogate pair split between two
  values is one character.
2026-11-13 10:09:27 -08:00
BFlat c7adea5518 Merge pull request #7144 from dotCooCoo/fix/script-line-numbers
Scripts: $.line and $.stack give the line of the file the code is in
2026-11-14 00:51:19 +07:00
dotCooCoo 13514841c1 Merge main into feat/script-file-folder-write 2026-11-13 08:15:22 -08:00
dotCooCoo c764652469 Merge main into fix/script-line-numbers 2026-11-13 08:14:59 -08:00
dotCooCoo 1db47f6488 Merge main into fix/script-namespace-heap-limit 2026-11-13 08:11:30 -08:00
BFlat f525bd742e Merge pull request #7147 from dotCooCoo/fix/script-file-name
Scripts: $.fileName names the file of the code that reads it
2026-11-13 23:04:04 +07:00
dotCooCoo b1c684e912 Scripts: E4X operations count their temporary storage against the heap limit
Assigning to a named child counts its list of the matching children, and
assigning through an empty XMLList counts the chain of targets it climbs,
before either changes the tree. copy(), toXMLString(), normalize(), the
`..` selector and the depth check on insertion hold the stack they keep
for each open level of the tree. Inserting an XMLList, and assigning one
to an item of an XMLList, hold their copy of the list's items.

copy() reads attributes in place, and toXMLString() reads attributes and
namespace declarations in place, instead of copying the lists. copy()
charges the declarations it copies before it copies them. Deleting by
name or assigning an attribute compacts the remaining links in place
instead of building a second list.

toXMLString() holds the set of prefixes a namespace lookup passes, and
picks a new nsN prefix in one pass over the declarations in scope
instead of rescanning them for each candidate.

The parser holds its open elements and their names, the text it gathers,
each tag's attribute values, the sets that check declarations and
attribute names, and its one copy of an element's namespaces, and it
charges the strings of the namespace declarations it makes. The
allowance it holds from the source's length covers the copy a value
takes as it becomes a string.

Comparing XML values, converting them to strings, assigning an XMLList
to an attribute, and reading a name, a qualifier, a namespace URI or
embedded text from a string built by joining strings flatten it with
flat_charged. An attribute name read from a string reserves its copy,
and an element name shares the string.

Docs: docs/target-app-parity.md.
2026-11-13 06:54:23 -08:00
dotCooCoo 4091f110b1 Scripts: $.line and $.stack give the line of the file the code is in
$.line and $.stack counted lines in the run's expanded source. In code
that comes after an #include they gave lines that match no file, and
this included code in a file $.evalFile ran and code a named engine
kept from an earlier run.

Both now map each frame through the run's source maps, as uncaught
errors already were. $.stack writes each frame as name@file:line,
outermost first, with the run's name as the file of a script given as
source. Code that eval or Function compiled keeps the line of the
source they compiled, in [eval]; code whose place a named engine no
longer keeps is at line 0 of [unknown]. The 64-frame limit, [global]
and [anonymous] are unchanged.

Docs: docs/mcp.md, docs/target-app-parity.md.
2026-11-13 06:19:54 -08:00
dotCooCoo 75c74557bb Scripts: $.fileName names the file of the code that reads it
$.fileName gave the file being run, so a function defined in a file
that #include inserted, in a file $.evalFile ran, or in an earlier run
of a named engine reported the calling file instead of its own.

- vectorcraft_js: HostItem::SiteGetter gives a getter the CallSite
  (source tag, line, column) of the code that reads it.
- $.fileName maps that place through the run's source maps, or the
  maps a named engine kept, to the file the code is in. It is empty in
  code from a script given as source and in kept code whose map was
  dropped. Code that eval or Function compiled still gets the file
  being run.

Docs: docs/mcp.md, docs/target-app-parity.md.
2026-11-13 06:19:23 -08:00
dotCooCoo 3ca5a75d87 Scripts: namespace scopes and edits count against the heap limit
The namespace list that inScopeNamespaces(), namespace(),
namespaceDeclarations(), the XPath namespace axis, XPath prefixes and
XPath name() collect from a node and its ancestors was built in Rust
buffers the heap limit did not see, so a node with many namespaces in
scope let a script grow past its limit.

The scope now holds the bytes of its list and prefix table until it is
dropped, and namespaceDeclarations() also holds its ancestor table and
the declarations it returns. The queried node stays rooted while the
scope is built, and each declaration is read in place instead of
copying every node's namespace list.

Other copies in the same files are held against the limit too:
- addNamespace(), setNamespace(), setName() and removeNamespace() read
  an element's attributes one at a time instead of copying the list,
  and removeNamespace() holds its walk stack.
- child(index), comments(), text(), elements(), processingInstructions()
  and normalize() on an XMLList hold the copy of its items, and
  replace() holds the copy of the children it matched.
- XPath name() holds the units of a prefixed name it builds.
- The XML, XMLList, Namespace and QName constructors, isXMLName(),
  attribute(), namespace(), propertyIsEnumerable(), setName(),
  setLocalName(), addNamespace(), removeNamespace() and setNamespace()
  reserve the bytes of the flat copy of a long (rope) string argument
  before they make it, and attribute() reserves the copy of a name
  without its leading @.
- QName.prototype.toString() reserves its result before it builds it.
2026-11-13 05:55:22 -08:00
dotCooCoo 4716646c6b Scripts: File and Folder objects write files inside a script's scope, with per-script grants and pickers
A script's File and Folder objects now open files for writing, appending
and editing, and copy, rename, remove and create files and folders. A
script reads and writes its run's temporary folder (Folder.temp) and its
own data folder (Folder.userData), kept between runs for a script run
from a file and inside Folder.temp for a script given as text. It reads
its own folder, and reaches other places only through File.openDialog,
File.saveDialog, Folder.selectDialog and the objects' openDlg, saveDlg and
selectDlg.

- A file dialog asks first. The person picks the file or folder with
  Choose..., then allows it once or, for a script run from a file, always.
  A kept grant is bound to the script's path, the SHA-256 of its text and
  every source the run evaluated. It is withdrawn when a later run of the
  script evaluates other code, including code an earlier run left in the
  same named engine. It stays on the real path that was granted, so a
  link put in that place later does not carry it elsewhere.
- A dialog's filter is a string of file types or a function that is
  called with each file picked; multiSelect gives an array of the files
  picked, at most 2,048, and the dialog says when more were left out.
- A folder's selectDlg opens the host's folder dialog in that folder,
  also where the host shows file dialogs in line. Enter in the file
  dialog denies, unless it goes to the focused Choose... button.
  script.grants.revoke is refused with the busy error while a script
  runs, as the run keeps the grants it started with.
- The paths a file dialog allows come only from the host's file dialog:
  ui.dialog.set cannot set them. Only the person at the keyboard allows
  access; Allow Once and Always Allow sent by automation are refused. The
  dialog describes the access the run asked for, whatever its fields say.
- No grant reaches the settings folder, also when the script's own folder
  holds it.
- A file left open is not written through a link put in place of its
  folder since it was opened, and Folder.userData is null when the data
  folder or the script's folder in it is a link.
- copy onto another link to the same file (a hard link) leaves the file's
  bytes as they are, after the source's open objects have written what
  they hold. Files are compared by identity (device and inode on Unix,
  volume and file index on Windows).
- Folder.temp and Folder.userData check each folder they make against the
  automation write roots first, and Folder.create() makes nothing when a
  missing folder above the target is outside the places the script may
  write.
- An open file holds at most 8 MiB, a run writes at most 32 MiB and keeps
  at most 256 files open, and the files a run left open are written in
  the order they were opened. Setting length and copying a file count the
  bytes every open object of that file takes, and write checks the joined
  text against the limit while joining it. The report names at most 4,096
  written files and sets writtenMore when there were more.
- Errors a script sees are translated in every complete message catalog.
2026-11-13 05:42:56 -08:00
BFlat 24905d721c Merge pull request #6839 from dotCooCoo/fix/script-targetengine-error-site
Scripts: an error in a function from an earlier #targetengine run names that run's file and line
2026-11-12 18:18:31 +07:00
BFlat 8448e5c18f Merge pull request #6788 from dotCooCoo/feat/e4x-xml-xpath
JS: XML and XMLList xpath evaluates XPath 1.0 expressions
2026-11-12 16:01:37 +07:00
dotCooCoo 044c8047f6 Scripts: an error in code an earlier #targetengine run compiled names that run's file and line
A function an earlier run of a named engine defined was reported without
a line, column or file when it threw in a later run. The realm now gives
each source a tag that stays the same from run to run (Realm::error_tag,
Realm::script_tag), and no two sources share one once the tags run out.
Each named engine keeps the source maps of its runs' scripts, included
files and $.evalFile files, within 8 MiB, dropping the oldest first, and
an error in kept code is reported at that file, line and column. The
report's name and the console line keep the calling run's name. For an
earlier script given as source, file is that run's name.

Docs: docs/mcp.md (Script engines), script.run params text,
docs/target-app-parity.md.
2026-11-12 00:39:41 -08:00
BFlat 6121418e0c Merge pull request #6783 from dotCooCoo/fix/js-flat-charges-heap
Scripts: operators, JSON.stringify and join count a flattened string against the heap limit
2026-11-12 15:12:46 +07:00
dotCooCoo 07c6f11abc Merge main into feat/e4x-xml-xpath 2026-11-11 23:31:02 -08:00
BFlat 9a4c36dfed Merge pull request #6771 from dotCooCoo/fix/e4x-scope-name-assign
JavaScript: a bare E4X name inside with or a filter can be assigned
2026-11-12 14:17:08 +07:00
dotCooCoo 49c8a380a1 Scripts: flattening a rope keeps it rooted while its copy is reserved
flat_charged and the property-key path now root the rope itself, so a string held only by
native code (a getter's result for ++o.x, a valueOf result, a toString key) still counts
what it holds when the reservation collects. The per-call roots for these cases are removed.
2026-11-11 23:02:45 -08:00
dotCooCoo c2c560eeaa Scripts: a String object's character is read along its rope when the copy does not fit
When the flattened copy of a String object's concatenated string would pass the heap limit
before a collection can run, the indexed read walks the rope to the unit instead, taking a
unit of fuel for each node it visits, so no copy is made.
2026-11-11 22:57:35 -08:00
dotCooCoo 40283f4a38 Scripts: loose equality and the JSON space string keep a converted string rooted while it is flattened 2026-11-11 22:49:15 -08:00
dotCooCoo 1f991536d6 Merge main into fix/js-flat-charges-heap 2026-11-11 22:44:59 -08:00
dotCooCoo 46efc4a651 Scripts: a String object counts its rope's copy when a character is read
- wrapping a string no longer flattens it, so Object(s).length and new String(s) on a large concatenated string make no copy and cannot collect while a call's operands are off the stack
- reading a String object's character by index, `in` and hasOwnProperty count the flattened copy at the next limit check, and the run ends before the copy when the fuel for it is not left
- unary plus on an object whose valueOf returns a concatenated string, and a property key from toString, keep that string rooted while its copy is reserved
- the unshift comment cites step 6 of section 15.4.4.13
2026-11-11 22:39:04 -08:00
BFlat af22fd6d5b Merge pull request #6776 from dotCooCoo/fix/eval-source-takes-fuel
JavaScript: source that eval and Function compile takes fuel before it is parsed
2026-11-12 13:17:47 +07:00
dotCooCoo 7222f43ecb Merge main into feat/e4x-xml-xpath 2026-11-11 22:03:32 -08:00
dotCooCoo 56f9b6181e Merge main into fix/e4x-scope-name-assign 2026-11-11 22:03:32 -08:00
dotCooCoo 3dc597f259 Scripts: a call and new String keep their operands rooted while a rope is flattened
Boxing the string receiver of a non-strict function flattens it, and reserving the copy can
collect. The callee, receiver and arguments are already off the stack at that point, so they
stay in the temporary roots until the frame holds them; before, an argument object could be
freed and its cell reused by the wrapper. new String(value) roots the converted string the
same way, so what the rope holds is still counted while its copy is reserved.
2026-11-11 21:53:50 -08:00
BFlat 105536fff4 Merge pull request #6765 from dotCooCoo/fix/js-finally-sites-gc-test
Scripts: a finally block left early drops its saved throw site
2026-11-12 12:40:29 +07:00
BFlat df41593465 Merge pull request #6772 from dotCooCoo/fix/e4x-xml-declaration-check
Scripts: E4X checks the XML declaration and takes fuel for every run the parser scans
2026-11-12 12:40:21 +07:00
dotCooCoo 44eb3be713 Scripts: a flattened string stays counted while the next copy is made
- the relational operators root both strings while flattening them, so the first copy still counts when the second one starts a collection
- join roots its separator for the whole join and each element while it is written, and JSON.stringify roots a string while it writes it
- a String object flattens its string when it is made, with the copy counted, so reading one of its characters copies nothing
2026-11-11 21:21:45 -08:00
dotCooCoo f526c5310c JS: XML and XMLList xpath evaluates XPath 1.0 expressions
xml.xpath(expression[, variables]) evaluates an XPath 1.0 expression
(https://www.w3.org/TR/1999/REC-xpath-19991116/) with the XML value as
the context node and returns the selected nodes as an XMLList, in
document order and without repeats. On an XMLList the expression runs
on each element in turn, and the results are joined.

- All 13 axes and the 27 core functions. id() selects nothing, since
  E4X trees carry no DTD data.
- Each property of `variables` binds the variable of its name. A
  prefix in a name test resolves through the context element's
  in-scope namespaces; `xml` is always bound.
- An expression that breaks the grammar gives a SyntaxError. An
  unknown function, a wrong argument count, an unbound prefix or
  variable, a result that is not a node-set, and a result that holds
  the root or a namespace node give a TypeError.
- The parser and the evaluator keep their state on their own stacks,
  so expressions of any depth use no native stack; nesting past
  Options::max_depth gives "nesting too deep". Traversal stacks, node
  lists, strings and the values that string functions build count
  against the heap limit, and every node visited and string read takes
  fuel.

Docs: docs/target-app-parity.md (E4X line), crate doc in lib.rs.
2026-11-11 21:01:32 -08:00
dotCooCoo 0996268f87 JavaScript: a host evaluator's whole source must fit in the heap before it is encoded
The fuel and heap checks for a host evaluator's source were split into a third
before encoding and the rest after, so a source whose whole UTF-16 text did not
fit in the heap could still be encoded and parsed. The whole encoded size is now
reserved before encoding. A source over the source limit is still refused as a
syntax error.
2026-11-11 20:54:00 -08:00
BFlat b945662f75 Merge pull request #6700 from dotCooCoo/fix/script-files-followups
Scripts: File and Folder exists checks the kind, encode keeps slashes, every call is counted
2026-11-12 11:42:26 +07:00
BFlat cd92345707 Merge pull request #6759 from dotCooCoo/fix/js-tosource-property-flags
Scripts: toSource refuses read-only, non-configurable and non-extensible data it cannot write back
2026-11-12 11:42:09 +07:00
dotCooCoo b172e3672d JavaScript: for each evaluates its target before reading the value
A for each loop read the property's value before evaluating its target,
so a target such as (o.p = 2, t).v, t[(o.p = 3, 'w')] or a bare
@[(o.p = 2, 'k')] stored the value from before the target ran. The
target's object and name are now evaluated first and the value is read
after them (ECMA-357 section 12.3).
2026-11-11 20:39:02 -08:00
dotCooCoo e2eb364f83 Scripts: operators, JSON.stringify and join count a flattened string against the heap limit
Flattening a concatenated string copies its units into a new buffer. The
string built-ins reserve that copy with flat_charged, but several other
paths flattened without it, so a script could build copies the heap limit
never saw:

- the relational operators (`<`, `>`, `<=`, `>=`) on two strings, `==`
  between a string and a number, unary `+` and every other ToNumber of a
  string, and reading a character by index (`s[0]`)
- a property name made from an object whose toString returns a long
  string, and a long string used as a property name, which now reserves
  its copy before it is made instead of after
- direct `eval` of a concatenated source
- `JSON.stringify` of a string value and its `space` argument
- `Array.prototype.join`'s separator and each string element it writes

Property names and eval source reserve the copy's heap bytes without
taking fuel.
2026-11-11 20:35:33 -08:00
dotCooCoo d151f3f5d0 JavaScript: source that eval and Function compile takes fuel before it is parsed
Direct and indirect eval, Function and host evaluators parsed and compiled
their source before taking any fuel or checking the heap, so a script with a
small fuel limit could make the interpreter parse megabytes of source in a
handful of instructions. Interp::prepare_source now takes one unit of fuel for
each UTF-16 code unit and checks that the source text fits in the heap before
the source is flattened, parsed or compiled. A host evaluator's string takes
the fuel for a third of its bytes before it is encoded, and the rest after.

Limits::fuel and the maxFuel row in docs/mcp.md say so.
2026-11-11 19:52:06 -08:00
dotCooCoo db97cc1a8f JavaScript: a bare E4X name inside with or a filter can be assigned
With Options::e4x set, an E4X name standing alone (@a, @ns::a, @[expr],
ns::a, *) is now a valid target for =, compound assignment, ++, --,
delete and for/for each loops, as well as a read. The target is the
nearest XML or XMLList value in scope whose [[HasProperty]] accepts the
name (ECMA-357 section 11.1). The name is evaluated once, and the
target resolves before the right-hand side runs.

When no XML value in scope holds the name, a read or write throws a
ReferenceError and creates no child or attribute. A read no longer
falls back to the first XML value in scope, so a filter such as
list.(@id == '1') throws when an item has no @id and no outer XML
value holds one, instead of treating that item as not matching.
2026-11-11 19:35:59 -08:00
dotCooCoo 781357921e Scripts: E4X checks the XML declaration and takes fuel for every run the parser scans
An XML declaration at the start of XML or XMLList text was skipped up to
the next "?>" without being read. It is now checked against XML 1.0's
grammar (https://www.w3.org/TR/xml/#NT-XMLDecl): a version of "1." and
digits, then an optional encoding name and an optional standalone "yes"
or "no", in that order, with optional white space before "?>". Any other
declaration gives "SyntaxError: invalid XML declaration", and a character
XML does not allow gives "invalid character in XML text". White space
before the declaration is still accepted.

The parser now takes fuel for the length of the declaration and the
white space before it, element and attribute names, white space in open
and closing tags, and the text it searched when an attribute value, CDATA
section, comment or processing instruction is not closed, or when "]]>"
or "<" stops a text run or attribute value. A long run in any of these
used the same fuel as a short one.
2026-11-11 19:35:39 -08:00
dotCooCoo dea30d6328 Scripts: a finally block left early drops its saved throw site
A finally block that break or continue leaves no longer keeps the saved
site of the exception it ran for. A Jump now removes the sites at or
above the stack's length, as a return already does, so a collection
after the stack grows back over that slot does not root and count the
old code.

New tests in crates/js/src/host_tests.rs: an error passed on by a
finally block that collects garbage under a 1 MiB heap is reported at
its throw; the saved site keeps direct eval code live through a
collection; a host abort inside a finally block leaves no saved site;
and break, continue and return out of a finally block drop the site.
2026-11-11 19:14:05 -08:00
BFlat 433a3009b7 Merge pull request #6701 from dotCooCoo/fix/xml-namespace-scope-fuel
JavaScript: XML namespace lookups take fuel for each ancestor they climb
2026-11-12 10:03:17 +07:00
dotCooCoo 19ed0889b1 Merge main into fix/script-files-followups
# Conflicts:
#	crates/engine/src/script/files.rs
2026-11-11 18:48:18 -08:00
dotCooCoo 98ad53e799 Merge main into fix/js-tosource-property-flags 2026-11-11 18:34:29 -08:00
dotCooCoo d28b72a358 Scripts: toSource refuses read-only, non-configurable and non-extensible data it cannot write back
A literal makes an extensible object whose properties are writable and
configurable, and an array whose length is writable. toSource wrote a
property defined with writable or configurable false, a sealed, frozen or
non-extensible object or array, and an array with a read-only length as an
ordinary literal, so evaluating the text gave an open value.

toSource now throws a TypeError for each of these, at any depth, as it
already does for getters, hidden properties and other prototypes.
Properties defined with every flag on are written as before.
2026-11-11 18:33:42 -08:00
BFlat 8719585cc6 Merge pull request #6663 from dotCooCoo/feat/script-target-engine
Scripts: #targetengine runs a script in a named engine that keeps its globals
2026-11-12 09:22:45 +07:00