Commit Graph
10 Commits
Author SHA1 Message Date
Martin BallandClaude Opus 5.5 820695e951 FU.2: craft-proof — Australian English reads SCOWL's own en-AU dictionary (ProofLang::EnAu), with en-GB's spelling policy; en-NZ stays on en-GB
SCOWL en-AU has 238 words en-GB lacks (billabong, brumby, larrikinism, Australian place
names) and spells program, inquiry, livable, licorice, verandah the Australian way (no
programme, enquiry, liveable, dialog or veranda, as Hunspell en_AU says); -ise/-ize policy and
affix file are the same as en-GB's. craft-i18n already ships language-data/en-au.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:18:42 +00:00
Martin BallandClaude Opus 5.5 5bd21c0509 CP6.13: craft-libs#482/#485/#487 review — en.UTF-8 and pl@euro resolve to their language; the German, Bulgarian and no-data English tests hold on a machine with Hunspell dictionaries installed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:27:37 +00:00
Martin BallandClaude Opus 5.5 8f16fe5a13 CP4.2: review — test dictionaries installed in memory hold the dictionary-tests lock until the test ends, so a pack or folder reload in another test can't drop them mid-test (the with-data ru suggestion flake)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 ee3be01b73 CP4.2: review — English tests hold the dictionaries lock; another region's pack never beats the system's exact region; bounded pack hyphenation lookups; NFC hyphenation; licences budgeted before reading; pack languages report like the others
P2:
- Every test whose English results depend on installed dictionaries takes DICTIONARY_TESTS (the en-GB goitre CI failure: tiny en_US fixtures raced unlocked en-GB tests).
- hunspell::candidates/find order: dictionaries folder, the first name's region pack (or the language alone), the system's exact-region dictionary, any other region's pack, the system's other names. pt_BR system beats a pt-pt pack; de_DE / de-ch, nl_NL / nl-be, es_ES / es-mx, en_US / en-gb likewise.

P3:
- Pack hyphenation lookups: a map, at most 256 languages (cleared when full); 3000 tags stay bounded and fast.
- A decomposed word (a + U+0301) is hyphenated composed (NFC) and the breaks mapped back; combining marks stay in their word.
- licence_texts checks each file's size against what is left before reading it; never reads past 16 MB.
- German, Bulgarian and pack languages: spelling message in the language where we have it; missing_dictionary/unreadable_dictionary report them like the others.
- build.rs: a missing file no longer says nothing is embedded; ru ignore message says CRAFT_I18N_DIR is enough. Built-in English list notice names the whole Moby list (en-words.dic).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 22:34:33 +00:00
Martin BallandClaude Opus 5.5 0675d58341 CP2.2: review — each English checks only its own SCOWL dictionary; English suggestions drop junk splits and names
craft-libs#483 review (P2): English spelling checked the union of the three
SCOWL dictionaries, so dialects crossed (en-US took colourblind, greyscale;
en-GB took colorblind, esophagi). Each English now checks its own dictionary
only. British English also takes the -ize spellings of its -ise words
(SCOWL's en-GB is -ise only; Word accepts both), and en-variants.txt widens
each dialect by its pairs; judgment/judgement come from craft-i18n#114, which
CI now pins. English with no dialect (is_correct) still takes a word any
English knows. Test: scowl_dialects_dont_cross samples the stems one
dialect has and another lacks, for every pair.

P3s:
- English suggestions (a Tuning of their own): a missing space only between
  two lower-case words of three letters or more, ranked after every one-word
  candidate (publically: publicly first; no "Thea tr", "Che Que"), and no
  word the dictionary only capitalizes (Col, Wis) for a lower-case word.
- A FORBIDDENWORD whole word is never rebuilt from compound parts.
- to_bytes writes ONLYINCOMPOUND without compound rules too (round-trip test).
- Failed lookups: the compound part table is filled lazily, and a word that
  doesn't start like a compound part (SCOWL: a digit) skips compounding;
  20,000 failed long words take 83 ms, as without compounding (1.09 s before).
- README: en-CA is not a British-spelling English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 21:44:18 +00:00
Martin BallandClaude Opus 5.5 e6e948f7d5 CP2.2: English spelling reads SCOWL 2026 through the affix engine
craft-i18n's SCOWL rel-2026.02.25 dictionaries (en-us, en-gb -ise, en-ca)
are embedded by build.rs and read per English: en-US, the British-spelling
Englishes (en-AU, en-NZ, en-IE...) through en-GB, en-CA. Suggestions come from
the run's own dictionary; a word is right when any of the three (or
en-extra.txt) knows it, unless en-variants.txt marks it as the other dialect's.
en-variants.txt stays: SCOWL's size-60 lists keep one spelling per dialect and
Word accepts more (organize and organise in en-GB, judgement in en-US).

English lookup order: dictionaries folder (the run's own English only),
[installed English pack, craft-libs#485], embedded SCOWL, system Hunspell
(own English only; not searched while SCOWL is embedded), the built-in Moby
list. The embedded Moby list is no
longer a spelling source (the built-in list is the same list); it stays for
hyphenation. The dialect policy applies to every source.

The affix engine gains COMPOUNDRULE (one-character flags, * and ?),
ONLYINCOMPOUND and COMPOUNDMIN, which SCOWL uses for ordinals (21st, 112th).
Hostile patterns are capped (64 rules, 32 tokens, 48-byte words) and checked
by a memoised walk. load_english now reads them instead of ignoring them (and
keeps ONLYINCOMPOUND stems as compound parts); only flag-based compounding and
OCONV are still ignored for English.

scowl::notice() exposes the SCOWL/Ispell/Titze notice for About. Embedded
Language data grows by 1,542,396 bytes. CI pins craft-i18n#109's merge commit efb1127.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 21:06:00 +00:00
Martin BallandClaude Opus 5.5 9e5862a1c1 CP1.2: each English uses only its own Hunspell dictionary, else the built-in Moby list
Re-review of craft-libs#482:
- en-GB and en-CA never fall back to an en_US Hunspell dictionary, which marked colour,
  organise, centre… (CI's runner has one); without their own they use the built-in Moby list,
  which has both spellings. en-US looks for en_US/en-US only.
- hunspell::candidates goes name by name across every folder, so a system en_GB beats an
  en_US in the dictionaries folder.
- Built-in suggestions rank swaps, then common words, neighbouring keys, the same first letter;
  no one-letter suggestions but a and i (hte -> the, recieve -> receive, wprd -> word).
- The generator leaves out Moby entries that are far more often typos: teh, abd, nad, ur, u, r
  (wont and cant stay: words). en-words.dic regenerated (343,988 bytes).
- A load that started before a reload no longer notes its load error; reload clears the
  English load errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:45:01 +00:00
Martin BallandClaude Opus 5.5 c80b6225f1 CP1.2: English without Language data uses the whole Moby list and every installed English dictionary
Review of craft-libs#482:
- The built-in fallback is now the full public-domain Moby word list (data/en-words.dic,
  words only, 343,992 bytes, from craft-i18n's en-us.dic by examples/en_words.rs) plus
  WordCraft's own en-extra.txt (MIT OR Apache-2.0). An unknown word is misspelled; the
  near-miss scan (which flagged fiber, theater, organize… and took 14 s on a 2,000-word
  token) is gone. Suggestions are the words one edit away, looked up, for words of up to 32
  letters. spelling_scales_over_one_long_token runs without the data, with a time bound.
- English Hunspell dictionaries ignore compounding (COMPOUNDMIN, COMPOUNDRULE…; their
  ONLYINCOMPOUND stems are left out) and OCONV instead of refusing the file, and a file that
  can't be read is skipped for the next name or folder.
- A load that started before english::reload() isn't kept (reloads counter, as slot.rs);
  Englishes that use the same .dic share one copy. en-GB/en-CA falling back to en_US is
  documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 18:37:02 +00:00
Martin BallandClaude Opus 5.5 23151270ab CP1.2: English never goes unchecked — embedded data, then system Hunspell, then a built-in core word list
- No embedded Moby list: English (en-US, en-GB, en-CA) uses an English
  Hunspell .aff/.dic from the dictionaries folder or the system (the run's
  English first, then another English), else data/en-core.txt.
- en-core.txt: 11,367 public-domain Moby Hyphenator II words (Grady Ward),
  ranked by lower-case frequency in 50 public-domain Gutenberg books
  (examples/en_core.rs, book ids recorded). 94 KB. Marks only near misses
  of common words (swapped letters; one edit for lower-case words of 4+).
- Unreadable English dictionary (encoding, not UTF-8, no words, size):
  error via dictionary_load_error, falls through to the core list; a bad
  embedded list is reported the same way. Never a panic.
- Non-English text is never checked against English.
- New: english_source(lang) -> Option<EnglishSource>.
- Embedded-data path unchanged; tests with data unchanged.
- Tests: core list flags/accepts, folder Hunspell preferred over core list,
  unreadable dictionaries reported, no_data English now marks misspellings;
  two data-ignored tests now run without data. README, ATTRIBUTION, NOTICE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 10:55:57 +00:00
Martin BallandClaude Opus 5.5 b758ba6943 CP1.2: craft-proof embeds Language data from CRAFT_I18N_DIR; works without it
- Package renamed wordcraft-proof -> craft-proof; [workspace.dependencies]
  entry; public API unchanged.
- build.rs embeds craft-i18n language-data/ (en-us, de, pl, pt, it, ru)
  instead of include_str!/include_bytes! of WordCraft's assets/: warns when
  CRAFT_I18N_DIR is unset, errors under CRAFT_I18N_REQUIRED=1.
- No data: English spelling marks and suggests nothing, hyphenation that
  needs data is off, grammar and rule-based hyphenation work. no_data.rs runs
  every public entry point without data; data-pinning tests are ignored then
  (cfg craft_proof_data).
- es timing bound 75 -> 150 ms: craft-libs tests run at opt-level 0.
- proof.yml (fmt, clippy -D warnings, tests with and without data, release
  guard, wasm32), README, CODEOWNERS for crates/proof.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 10:02:10 +00:00