Commit Graph
5 Commits
Author SHA1 Message Date
NaxvogandClaude Opus 5.5 b732f2abc7 proof: cap .aff REP replacements at parse time (storytold/wordcraft#3655)
Both readers kept every REP line: an .aff with hundreds of thousands of
entries, or megabyte-long patterns and replacements, made each
suggestion round hang and allocate hundreds of MB. push_rep keeps at
most 4096 entries, drops patterns longer than MAX_LETTERS letters (no
suggester takes a longer word, so they never match) and replacements
longer than MAX_WORD_BYTES.

Ported from storytold/wordcraft#4632, which can't land now that
proofing lives here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 07:10:24 +01:00
Martin BallandClaude Opus 5.5 9b6ef1199a CP7.8: suggestion steps charge the work done; big homonym aliases load once
A step now counts a rule, a condition class, a split's suffix lookup,
16 flags of a set looked through and a group visit, so hostile rules
whose steps were cheap to count but costly to run hit the budget in
time. Retuned: 2^13 steps per candidate, 2^25 per suggest call; the
81 packs' typo sets suggest exactly as at d0c341c8.

Condition character sets are sorted and looked up by binary search.
A stem's sets are worked out once per distinct entry list when its
entries hold 256+ flags (16 kept-apart homonyms of 8,190 flags on
17,576 stems: 55 s load, now 57 ms). A near-word candidate whose
steps ran out counts as a word, so suggest never offers a compound
check() would refuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 05:55:02 +00:00
Martin BallandClaude Opus 5.5 b8efc5add9 CP7.4: craft-libs#497 re-review — hu prefixed suffix chains, homonyms kept apart, all-caps mixed-case stems; compound suggestions cheapest-first in one budget
- A prefix with two chained suffixes (hu `meg`+`épít`+`és`+`e`, eo `enjugo`): every rule
  cross, the prefix flag from the stem or a continuation class (CLI answers).
- Homonyms with different flags are kept apart (and written apart in the compact form):
  a prefix and a suffix, or a compound flag and a suffix, must come from one homonym
  (nl `afstopje`, `schuurpoederden` wrong in the CLI). nl now 100% in both comparisons.
- Hungarian syllable rule leaves out a last suffix with no continuation classes;
  COMPOUNDROOT counts on the stem only.
- Near-word check: a word listed as an ONLYINCOMPOUND/NEEDAFFIX stem counts only with a
  suffix, not a prefix (de `Fahrradverkehr` right).
- A word in capitals may be a mixed-case stem (`KB`/`KBS` from `kB/S`, hu `DNY`, nl `BSC`).
- Mixed-case retry keeps KEEPCASE for the first part; KEEPCASE kept apart from the other
  exclusions (NOSUGGEST sharing its flag is no longer cleared); overflow of the homonym
  groups counts as dropped.
- Suggestions: candidates are checked as simple words first; the rest get the compound
  check cheapest first, those changed where the word stops being parts before others, in
  one step budget per call, with piece analyses shared between candidates. The suggestions'
  stem scan makes forms in one shared budget (mn).
- affix_check example: `suggest` command; AffixDict::suggest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 03:18:31 +00:00
Martin BallandClaude Opus 5.5 3eaa276786 CP7.3: craft-libs#497 review — compounds as the hunspell CLI answers: ONLYINCOMPOUND homonyms keep their flags, triples after SIMPLIFIEDTRIPLE, KEEPCASE and capitals in parts, CHECKCOMPOUNDPATTERN 0, a forbidden prefix, FORCEUCASE; suggestions and many-way pieces stay cheap
- ONLYINCOMPOUND homonyms beside others are kept apart (compound parts only), so their suffixes
  make no standalone words (de verfilmungs, sv norges); round-trips as a second stem line.
- CHECKCOMPOUNDTRIPLE also checks a SIMPLIFIEDTRIPLE join (busssen, booob wrong).
- KEEPCASE holds for the first part only; a capitalized stem is a part (FooQux).
- CHECKCOMPOUNDPATTERN `0`: no suffix (a prefix doesn't count): unfoobar wrong.
- COMPOUNDFORBIDFLAG on a prefix: no part anywhere (bardefoo wrong).
- FORCEUCASE: from the stem only; wrong only without any capital (fooQuxbar right).
- Suggestion candidates: 2,048 compound steps each, compounds for the first 128 only.
- A piece's ways are hashed, capped at 256 and counted (with the patterns) against the budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 02:08:35 +00:00
Martin BallandClaude Opus 5.5 87c7d1874a CP7.1: craft-proof reads flag-based compounding (COMPOUNDFLAG/BEGIN/MIDDLE/END, PERMITFLAG, FORBIDFLAG, ROOT, WORDMAX, SYLLABLE, CHECKCOMPOUND*, SIMPLIFIEDTRIPLE, FORCEUCASE), as the hunspell CLI answers
Black-box: every rule pinned to hunspell 1.7.5's answers on synthetic dictionaries and on
the Language packs' own files. Bounded work per word (16 parts, 65,536 steps). Also:
ONLYINCOMPOUND belongs to its homonym like NEEDAFFIX; COMPOUNDRULE reads the Swedish `)k`
and the Mongolian `[..]`, `%?`, `.` as the CLI does; COMPOUNDMORESUFFIXES is refused only
with flag-based compounding (Korean keeps loading).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:43:54 +00:00