Check Spelling: Ignore and Change All reach every word past the check limit

text.spelling.check listed at most 10,000 words and always started each
type object at its first byte, so the dialog could not go on inside an
object. With 10,001 unknown words in one object, Ignore ended the check
after the first 10,000. Change All stopped when a round changed nothing,
so a target after 10,000 other unknown words was never reached. Follow-up
to #7744; part of #4936.

- text.spelling.check takes from {id, byte}, which goes on from that
  byte of that type object and then the objects after it, and max
  (1 to 10,000). The paragraphs before the one that holds from are not
  checked again.
- The dialog keeps the words a check lists after the one shown, and
  Ignore and Ignore All take the next word from them. A new check goes
  on from the last word listed once they run out. Every word is reached,
  in a selected group's type too.
- text.spelling.change {to, words} replaces every listed word in one
  undo step and lays each object out once. Change All finds every
  occurrence with one check after another, then changes them with it.
- Change and Change All to the word shown make no edit and no undo step.
- When another command changes the document or the custom dictionary
  while the dialog is open, the next step checks the scope again from
  its start. An ignored occurrence stays skipped while the same word is
  at the same bytes.
This commit is contained in:
dotCooCoo
2026-11-15 16:34:23 -08:00
parent bb218101c9
commit ca822f41f7
4 changed files with 935 additions and 261 deletions
+131 -6
View File
@@ -7,17 +7,20 @@
//! `removeWord`, `changeWord`), which every document's check takes as right; Ignore All
//! (`text.spelling.ignoreAll`) skips a word in the active document only.
use std::collections::BTreeSet;
use std::collections::{BTreeMap, BTreeSet};
use std::ops::Range;
use std::sync::atomic::{AtomicU64, Ordering};
use craft_proof::{IssueKind, Options, ProofLang};
use serde_json::{Value, json};
use vectorcraft_doc::{Document, NodeId, NodeKind, TextObject};
use vectorcraft_text::edit;
use super::menucmds::whole_or;
use super::textedit::text_mut;
use super::typecmd::refresh_bounds;
use super::typemenu::{all_texts, text_ids};
use super::{CommandSpec, Result, Session, always, bad, checked_ids_param, cmd, has_doc, str_param};
use super::{CommandSpec, EngineError, Result, Session, always, bad, checked_ids_param, cmd, has_doc, str_param};
/// Most unknown words one check lists; past it the result says `more`.
pub const MAX_WORDS: usize = 10_000;
@@ -36,10 +39,19 @@ pub fn specs() -> Vec<CommandSpec> {
"Check Spelling",
[],
None,
"{ids?: [id…] (type objects, or groups holding them; default: every visible, unlocked type object)} the words the dictionaries don't know, each in its run's language (English when it has none; none in a language without a dictionary), each paragraph on its own, in paint order (given ids too) → {words: [{id, start: byte, end: byte, word, lang}…], more: bool (more than 10000 words, the rest left out)}. text.editRange {id, start, end, insert} changes one",
"{ids?: [id…] (type objects, or groups holding them; default: every visible, unlocked type object), from?: {id, byte} (go on from there: the words of that type object that start at byte or later, then the objects after it; id must be one of the objects checked), max?: 1..10000 (default 10000)} the words the dictionaries don't know, each in its run's language (English when it has none; none in a language without a dictionary), each paragraph on its own, in paint order (given ids too) → {words: [{id, start: byte, end: byte, word, lang}…], more: bool (more than max words, the rest left out: check again with from {id, byte: end} of the last word)}. text.editRange {id, start, end, insert} changes one",
has_doc,
check
),
cmd!(
"text.spelling.change",
"Change All",
[],
None,
"{to: string, words: [{id, start: byte, end: byte}…] (words of type objects, as text.spelling.check lists them; none overlapping)} replaces each word with to, all in one undo step, each object laid out once (Check Spelling's Change All); an error, with nothing changed, when a word's bytes are not in its object's text → {changed: count}",
has_doc,
change_words
),
cmd!(
query "text.spelling.suggest",
"Spelling Suggestions",
@@ -107,22 +119,42 @@ pub fn specs() -> Vec<CommandSpec> {
}
/// `text.spelling.check`: the unknown words of the given type objects (or every visible, unlocked
/// one), in paint order, each paragraph checked on its own, at most [`MAX_WORDS`] of them.
/// one), in paint order, each paragraph checked on its own, at most `max` of them, starting at
/// `from` when it is given.
fn check(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "text.spelling.check";
// Given `ids` are checked like Find and Replace's, so a malformed one is an error rather than
// a check of the whole document.
let ids = match p.get("ids").filter(|v| !v.is_null()) {
let mut ids = match p.get("ids").filter(|v| !v.is_null()) {
Some(_) => in_paint_order(&s.doc()?.doc, text_ids(&s.doc()?.doc, &checked_ids_param(s, p, "ids", C)?)),
None => all_texts(&s.doc()?.doc),
};
let max = whole_or(p, "max", MAX_WORDS as u64, C)?;
let max = usize::try_from(max).ok().filter(|m| (1..=MAX_WORDS).contains(m)).ok_or_else(|| bad(C, format!("`max` must be 1 to {MAX_WORDS}")))?;
let st = s.doc()?;
// Where the last check stopped: the objects before it are skipped, and the bytes of its object
// before `byte`.
let mut from_byte = 0;
if let Some(from) = p.get("from").filter(|v| !v.is_null()) {
let (id, byte) = from_param(from, C)?;
let Some(pos) = ids.iter().position(|i| i.0 == id) else {
return Err(bad(C, format!("`from.id` {id} is not one of the type objects checked")));
};
let Some(NodeKind::Text(t)) = st.doc.node(NodeId(id)).map(|n| &n.kind) else {
return Err(bad(C, format!("`from.id` {id} is not a type object")));
};
let len = t.plain_text().len();
from_byte = usize::try_from(byte).ok().filter(|b| *b <= len).ok_or_else(|| bad(C, format!("`from.byte` is past the end of {id}'s text")))?;
ids.drain(..pos);
}
let mut words = vec![];
let mut more = false;
for id in ids {
let Some(NodeKind::Text(t)) = st.doc.node(id).map(|n| &n.kind) else { continue };
let text = t.plain_text();
let (found, cut) = misspellings(t, &st.spelling_ignored, MAX_WORDS - words.len());
// Only the first object listed starts part way.
let from = std::mem::take(&mut from_byte);
let (found, cut) = misspellings_from(t, &st.spelling_ignored, from, max - words.len());
for m in found {
let Some(word) = text.get(m.range.clone()) else { continue };
words.push(json!({ "id": id.0, "start": m.range.start, "end": m.range.end, "word": word, "lang": m.lang.tag() }));
@@ -135,6 +167,84 @@ fn check(s: &mut Session, p: &Value) -> Result<Value> {
Ok(json!({ "words": words, "more": more }))
}
/// `text.spelling.check`'s `from`: `{id, byte}`, both whole numbers.
fn from_param(from: &Value, command: &str) -> Result<(u64, u64)> {
let Some(o) = from.as_object() else { return Err(bad(command, "`from` must be {id, byte}")) };
if let Some(k) = o.keys().find(|k| !matches!(k.as_str(), "id" | "byte")) {
return Err(bad(command, format!("`from` takes id and byte, not `{k}`")));
}
let id = o.get("id").and_then(Value::as_u64).ok_or_else(|| bad(command, "`from.id` must be a type object's id"))?;
let byte = o.get("byte").and_then(Value::as_u64).ok_or_else(|| bad(command, "`from.byte` must be a whole number"))?;
Ok((id, byte))
}
/// `text.spelling.change`: Change All, every listed word replaced in one edit. Each object's words
/// are changed back to front, so the bytes of the ones before stay true, and the object is laid out
/// once at the end rather than once for each word.
fn change_words(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "text.spelling.change";
let to = str_param(p, "to").ok_or_else(|| bad(C, "missing `to`"))?.to_string();
let list = p.get("words").and_then(Value::as_array).ok_or_else(|| bad(C, "`words` must be a list of {id, start, end}"))?;
let mut by_id: BTreeMap<NodeId, Vec<(usize, usize)>> = BTreeMap::new();
for w in list {
let (id, start, end) = word_param_range(w, C)?;
by_id.entry(NodeId(id)).or_default().push((start, end));
}
// Every word checked against its object before anything changes.
let st = s.doc()?;
for (id, ranges) in &mut by_id {
let Some(NodeKind::Text(t)) = st.doc.node(*id).map(|n| &n.kind) else {
return Err(bad(C, format!("{} is not a type object", id.0)));
};
let text = t.plain_text();
ranges.sort_unstable();
let mut last_end = 0;
for &(a, b) in ranges.iter() {
if a >= b || text.get(a..b).is_none() {
return Err(bad(C, format!("{a}..{b} is not a word of {}'s text", id.0)));
}
if a < last_end {
return Err(bad(C, format!("words of {} overlap at byte {a}", id.0)));
}
last_end = b;
}
}
let changed: usize = by_id.values().map(Vec::len).sum();
if changed == 0 {
return Ok(json!({ "changed": 0 }));
}
s.edit("Change All", |d, _| {
for (id, ranges) in &by_id {
let t = text_mut(d, *id).ok_or(EngineError::NoNode(*id))?;
for &(a, b) in ranges.iter().rev() {
// Paragraph styles and ruby follow each edit, as in `text.editRange`.
t.splice_paras(a, b, &to);
edit::replace_range(&mut t.runs, a, b, &to);
t.ruby_after_replace(a, b, to.len());
}
refresh_bounds(t);
}
Ok(())
})?;
Ok(json!({ "changed": changed }))
}
/// One of `text.spelling.change`'s `words`: `{id, start, end}`, all whole numbers.
fn word_param_range(w: &Value, command: &str) -> Result<(u64, usize, usize)> {
let Some(o) = w.as_object() else { return Err(bad(command, "each of `words` must be {id, start, end}")) };
if let Some(k) = o.keys().find(|k| !matches!(k.as_str(), "id" | "start" | "end")) {
return Err(bad(command, format!("each of `words` takes id, start and end, not `{k}`")));
}
let id = o.get("id").and_then(Value::as_u64).ok_or_else(|| bad(command, "a word's `id` must be a type object's id"))?;
let byte = |k: &str| {
o.get(k)
.and_then(Value::as_u64)
.and_then(|b| usize::try_from(b).ok())
.ok_or_else(|| bad(command, format!("a word's `{k}` must be a whole number")))
};
Ok((id, byte("start")?, byte("end")?))
}
/// One word of a type object its dictionaries don't know ([`misspellings`]).
#[derive(Clone, Debug, PartialEq)]
pub struct Misspelling {
@@ -149,6 +259,13 @@ pub struct Misspelling {
/// All words). At most `max` of them; the flag says the list was cut short there. What
/// `text.spelling.check` lists for one object, and what Auto Spell Check underlines.
pub fn misspellings(t: &TextObject, ignored: &BTreeSet<String>, max: usize) -> (Vec<Misspelling>, bool) {
misspellings_from(t, ignored, 0, max)
}
/// [`misspellings`] of the words that start at byte `from` or later. The paragraphs that end at
/// or before `from` are not checked; the paragraph that holds `from` is checked whole, since
/// craft-proof reads a paragraph's language from all of it.
pub fn misspellings_from(t: &TextObject, ignored: &BTreeSet<String>, from: usize, max: usize) -> (Vec<Misspelling>, bool) {
let text = t.plain_text();
// Each run in its own language, at its bytes of the plain text.
let mut spans: Vec<(Range<usize>, ProofLang)> = Vec::with_capacity(t.runs.len());
@@ -166,6 +283,11 @@ pub fn misspellings(t: &TextObject, ignored: &BTreeSet<String>, max: usize) -> (
let mut para_start = 0;
for para in text.split('\n') {
let para_end = para_start + para.len();
// No word of a paragraph that ends at `from` starts there or later.
if para_end <= from {
para_start = para_end + 1;
continue;
}
let para_spans: Vec<(Range<usize>, ProofLang)> = spans
.iter()
.filter_map(|(r, lang)| {
@@ -178,6 +300,9 @@ pub fn misspellings(t: &TextObject, ignored: &BTreeSet<String>, max: usize) -> (
continue;
}
let (start, end) = (para_start + issue.start, para_start + issue.end);
if start < from {
continue;
}
let Some(word) = text.get(start..end) else { continue };
if craft_proof::custom_word(word).is_some_and(|w| ignored.contains(&w)) {
continue;
+107
View File
@@ -345,6 +345,113 @@ fn misspellings_of_one_object_are_what_the_check_lists() {
assert_eq!(found.iter().map(|m| m.range.clone()).collect::<Vec<_>>(), [listed[0].clone(), listed[2].clone()]);
}
/// The (id, start, word) of each word a check lists, and its `more`.
fn listed(s: &mut Session, p: Value) -> (Vec<(u64, u64, String)>, bool) {
let r = run(s, "text.spelling.check", p);
let words = r["words"]
.as_array()
.unwrap()
.iter()
.map(|w| (w["id"].as_u64().unwrap(), w["start"].as_u64().unwrap(), w["word"].as_str().unwrap().to_string()))
.collect();
(words, r["more"].as_bool().unwrap())
}
/// A check cut short at `max` goes on with `from` at the end of the last word listed: the rest of
/// that type object first, then the objects after it, and nothing listed twice.
#[test]
fn a_check_goes_on_from_where_the_last_one_stopped() {
let mut s = session();
let a = text(&mut s, 50.0, "qzxvw one wvxzq two\nthree qqzzx");
let b = text(&mut s, 100.0, "zzqqx");
let (first, more) = listed(&mut s, json!({"max": 2}));
assert_eq!((first.clone(), more), (vec![(a.0, 0, "qzxvw".into()), (a.0, 10, "wvxzq".into())], true));
// From the end of the second word: the third, in the next paragraph of the same object, not
// the start of the next object.
let (rest, more) = listed(&mut s, json!({"from": {"id": a.0, "byte": 15}, "max": 2}));
assert_eq!((rest, more), (vec![(a.0, 26, "qqzzx".into()), (b.0, 0, "zzqqx".into())], false));
// A byte inside a word starts from the next word.
let (rest, _) = listed(&mut s, json!({"from": {"id": a.0, "byte": 1}}));
assert_eq!(rest.first(), Some(&(a.0, 10, "wvxzq".to_string())));
// `from` follows the ids given, groups' type included.
run(&mut s, "select.set", json!({"ids": [a.0, b.0]}));
let g = run(&mut s, "object.group", json!({}))["id"].as_u64().unwrap();
let (rest, more) = listed(&mut s, json!({"ids": [g], "from": {"id": a.0, "byte": 26}, "max": 1}));
assert_eq!((rest, more), (vec![(a.0, 26, "qqzzx".into())], true));
let (rest, more) = listed(&mut s, json!({"ids": [g], "from": {"id": a.0, "byte": 31}, "max": 1}));
assert_eq!((rest, more), (vec![(b.0, 0, "zzqqx".into())], false));
let bad = [
json!({"from": {"id": a.0}}),
json!({"from": {"id": a.0, "byte": -1}}),
json!({"from": {"id": a.0, "byte": 1000}}),
json!({"from": {"id": a.0, "byte": 0, "x": 1}}),
json!({"from": a.0}),
json!({"ids": [b.0], "from": {"id": a.0, "byte": 0}}),
json!({"max": 0}),
json!({"max": 10_001}),
json!({"max": "2"}),
];
for p in bad {
assert!(s.execute("text.spelling.check", &p).is_err(), "{p} is refused");
}
}
/// One type object with one unknown word more than a check lists: going on from the last word
/// listed reaches the last one.
#[test]
fn a_check_past_the_limit_in_one_object_goes_on_to_its_last_word() {
use crate::cmd::spelling::MAX_WORDS;
let mut s = session();
let t = text(&mut s, 50.0, &vec!["qzxvw"; MAX_WORDS + 1].join(" "));
let r = run(&mut s, "text.spelling.check", json!({}));
let words = r["words"].as_array().unwrap();
assert_eq!((words.len(), &r["more"]), (MAX_WORDS, &json!(true)));
let end = words.last().unwrap()["end"].as_u64().unwrap();
let (rest, more) = listed(&mut s, json!({"from": {"id": t.0, "byte": end}}));
assert_eq!((rest, more), (vec![(t.0, end + 1, "qzxvw".into())], false));
}
/// The plain text of type object `id`.
fn plain(s: &Session, id: NodeId) -> String {
match &s.doc().unwrap().doc.node(id).unwrap().kind {
vectorcraft_doc::NodeKind::Text(t) => t.plain_text(),
_ => panic!("not type"),
}
}
/// Change All's command replaces every word listed, in several objects, as one undo step, and
/// refuses a list with any word not in its object's text, changing nothing.
#[test]
fn change_all_replaces_every_word_listed_in_one_undo_step() {
let mut s = session();
let a = text(&mut s, 50.0, "qzxvw one qzxvw\nqzxvw");
let b = text(&mut s, 100.0, "é qzxvw");
let depth = s.doc().unwrap().history.undo.len();
let words = json!([{"id": a.0, "start": 16, "end": 21}, {"id": a.0, "start": 0, "end": 5}, {"id": b.0, "start": 3, "end": 8}, {"id": a.0, "start": 10, "end": 15}]);
let r = run(&mut s, "text.spelling.change", json!({"to": "fixed", "words": words}));
assert_eq!(r["changed"], 4);
assert_eq!((plain(&s, a), plain(&s, b)), ("fixed one fixed\nfixed".to_string(), "é fixed".to_string()));
assert_eq!(s.doc().unwrap().history.undo.len(), depth + 1);
run(&mut s, "edit.undo", json!({}));
assert_eq!((plain(&s, a), plain(&s, b)), ("qzxvw one qzxvw\nqzxvw".to_string(), "é qzxvw".to_string()));
let bad = [
json!({"to": "x", "words": [{"id": a.0, "start": 0, "end": 5}, {"id": a.0, "start": 3, "end": 8}]}),
json!({"to": "x", "words": [{"id": a.0, "start": 0, "end": 5}, {"id": a.0, "start": 20, "end": 99}]}),
json!({"to": "x", "words": [{"id": b.0, "start": 1, "end": 4}]}),
json!({"to": "x", "words": [{"id": a.0, "start": 5, "end": 5}]}),
json!({"to": "x", "words": [{"id": a.0, "start": 0, "end": 5, "x": 1}]}),
json!({"to": "x", "words": [{"id": a.0, "start": -1, "end": 5}]}),
json!({"to": "x", "words": [{"id": 999_999, "start": 0, "end": 5}]}),
json!({"to": "x", "words": {"id": a.0}}),
json!({"words": []}),
];
for p in bad {
assert!(s.execute("text.spelling.change", &p).is_err(), "{p} is refused");
}
assert_eq!(plain(&s, a), "qzxvw one qzxvw\nqzxvw", "nothing changed by a refused list");
assert_eq!(run(&mut s, "text.spelling.change", json!({"to": "x", "words": []}))["changed"], 0);
}
/// Each change to the custom dictionary moves its generation on, whether it is saved or not, so
/// what Auto Spell Check found before is checked again.
#[test]
File diff suppressed because it is too large Load Diff
+13 -4
View File
@@ -9302,13 +9302,22 @@ a spelling dictionary marks nothing. English words come from the first dictionar
dictionary in the user's dictionaries folder, the SCOWL dictionary for that English built in from craft-i18n, a
system Hunspell dictionary, and last the public-domain Moby word list built into craft-proof. Words in capitals, words with digits and web addresses are taken as correct. The result is
`{words: [{id, start, end, word, lang}], more}`: `start` and `end` are byte offsets in the type object's text, the
ones `text.editRange` takes, so `text.editRange {id, start, end, insert}` changes a word as one undo step. At most
10,000 words are listed; `more` is `true` when there were more. `text.spelling.suggest {word, max?, lang?}` gives
the words the dictionary of `lang` (default English) suggests instead of `word`, best first, at most `max` (1 to 20,
default 5). Both only read the document.
ones `text.editRange` takes, so `text.editRange {id, start, end, insert}` changes a word as one undo step, and
`text.spelling.change {to, words: [{id, start, end}…]}` changes every listed word to `to` as one undo step (Check
Spelling's Change All; an error, with nothing changed, when a word's bytes are not in its object's text or two words
overlap). At most
`max` words are listed (1 to 10,000, default 10,000); `more` is `true` when there were more. To get the rest, check
again with `from: {id, byte}` set to the last word's `id` and `end`: that check lists the words of that type object
that start at `byte` or later, then those of the objects after it, so a type object with more unknown words than one
check lists is read to its end. `from.id` must be one of the objects checked. The paragraphs before the one that
holds `from.byte` are not checked again; that paragraph is checked whole. `text.spelling.suggest {word, max?, lang?}`
gives the words the dictionary of `lang` (default English) suggests instead of `word`, best first, at most `max` (1 to
20, default 5). `text.spelling.check` and `text.spelling.suggest` only read the document.
```json
{"name":"run_command","arguments":{"command":"text.spelling.check","params":{}}}
{"name":"run_command","arguments":{"command":"text.spelling.check","params":{"from":{"id":12,"byte":340},"max":100}}}
{"name":"run_command","arguments":{"command":"text.spelling.change","params":{"to":"drawing","words":[{"id":12,"start":4,"end":10}]}}}
{"name":"run_command","arguments":{"command":"text.spelling.suggest","params":{"word":"drawng","lang":"en"}}}
```