I18n::new keeps the embedded text and parses nothing; a locale's rows
are parsed on the first lookup that reaches it (also as a fallback) and
its bad rows logged with log::warn!. new returns only the locale-level
problems. check_all parses every locale and returns all problems.
name() scans for the language.name row instead of parsing the locale.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A value with an embedding, override, isolate or mark (LRE RLE PDF LRO
RLO LRI RLI FSI PDI LRM RLM ALM) makes fmt isolate, even in text with no
right-to-left letter, so an ASCII file name with an RLO can't reorder
the sentence. Embeddings, overrides and isolates a value opens are
closed inside its isolate, and stray PDF and PDI are dropped.
han(zh-US), zh-AU and the other regions CLDR writes in Traditional
characters give Traditional, and pa-PK picks a pa-arab catalog, never
Gurmukhi pa, as pa-Arab-PK does.
Format::parse reads what a Format writes: its digits, its decimal and
grouping separators (the system's included) and the locale's Latin
marks in an Arabic-Indic format, plus every digit set. parse_number is
Format::from_tag(locale).parse, so the decimal marks come from CLDR's
data alone and the hand-made table is gone.
A lone . in a comma locale is the decimal mark when it can't be
grouping (1.5, 0.250, 1234.5, .5); 1.234 and 1.234.567 stay grouped in
German. The same holds for , in a point locale. Percent signs and units
are documented as the caller's to strip.
Plural rules are looked up by language alone: CLDR's parent of sr-Latn
and zh-Hant is the root, whose only category is other, so sr-latn rows
were rejected. The baked data covers every CLDR locale (10 KB): plural
rules for he, ro, nb, ca, sk, th, ms and the rest, and decimal symbols
for nb-NO, sk-SK, bn-BD, ar-MA and English in Europe. Format applies a
Region's separators only where the system set them, and an empty
grouping separator writes none.
trn_plain writes counts in 0-9 without grouping, for documents and file
names. reset_format goes back to the system region. trn_decimal and
Format write infinities and NaN as CLDR does and take the other form.
Tests that don't set a format use en-US, not the machine's region. The
ICU4X crates are pinned to ~2.3, the minor version the data is for.
Collator::new takes sv_SE.UTF-8 and sr_RS@latin. Folding maps every
script's decimal digits to ASCII and, ignoring accents, Persian keheh,
Farsi yeh, alef maksura, teh marbuta and heh with hamza to the Arabic
letters. The collation data grows 4.8 KB (465 bytes gzipped).
Region::decimal and grouping are None unless Windows' or macOS's own
settings set them, so the number locale's CLDR data decides otherwise.
A cleared Windows sThousand is no grouping. LC_ALL=C and LC_NUMERIC=C
(and the other LC_* variables) mean the C locale; only LANG=C gives way
to the preferred language.
fmt and trn add isolates only when the template or an inserted value has
a strong RTL character (bidi class R or AL). Text with none comes out as
before, and a value's own isolate from ltr or isolate is taken off.
Format writes integers, decimals and percentages for a Region on ICU4X's
decimal formatter: the locale's digits (Arabic-Indic in Egypt, Persian in
Iran) or Latin ones on request, its separators and grouping sizes (Indian
12,34,567), CLDR's percent patterns, and the region's own separators where
the system sets them. I18n::format holds the one trn, file_size and
measure use, the system region's until set_format.
Plural rules move from intl_pluralrules to ICU4X, and trn_decimal picks
the form from the digits shown (1.0 files, 1,5 файла).
The CLDR data (decimal symbols, digits, plural rules, percent patterns)
covers the shipped languages and their regional variants; data/generate.sh
rebuilds it.
`Folder` folds text for case- and accent-insensitive search (Turkish and
Azerbaijani i by locale, ß, final sigma, ligatures, full-width forms) and
maps matches back to the original text for highlighting. `Collator`
(feature `collate`) sorts lists in the locale's order with digits compared
by value, on ICU4X with collation data for the shipped locales only.
The Turkish i and sigma handling come from PdfCraft's `fold_case`.
Part of #198.
Co-authored-by: snipeTR <31975916+snipeTR@users.noreply.github.com>
Co-authored-by: nevicia2010-hub <nevicia2010@gmail.com>
I18n::file_size writes a byte count in steps of 1024 (KB or KiB) or 1000
(KB), with decimals per unit, no decimals from a given value up, a
smallest and largest unit, and bytes as B or in words. I18n::measure
writes any value with a unit template such as "{n} px". Unit names are
catalog rows in context `unit`, keyed by the English template, so a
translation can change the symbol and its spacing.
Reads any Unicode decimal digits, the locale's decimal mark plus ٫,
grouping by the locale's other mark, spaces, apostrophes or ٬ (threes
or Indian 2-2-3), and -/−/+ signs, ignoring bidi marks. Input that
fits no single reading is an error, not a guess.
Region holds the number and time locales, metric or US measurement, A4 or
Letter, the decimal separator, and the grouping separator and clock when
the system sets them. Region::system reads Windows' regional format with
the user's changes, Apple's current locale, the POSIX LC_* variables each
on their own, and the preferred language on the web and Android.
Region::from_setting takes an app's saved choice ("auto" or a tag).
Ported from WordCraft's Region (storytold/wordcraft#857, #1494).
Co-authored-by: Dmitrii Khlapov <dkhlapov@pm.me>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
fmt wraps each value in FSI…PDI and trn wraps {n} in LRI…PDI, so a
value's direction can't reorder the message around it. ltr() isolates
numbers and paths left to right; a value that already is one isolate
goes in unchanged, and a stray PDI can't close the isolate early.
fmt_plain and strip_isolates are for text that leaves the interface.
scripts(tag) lists the ISO 15924 scripts a language's interface text is
written in, direction(tag) follows the main script, and han(tag) gives
the regional Han forms (SC, TC, HK, JP, KR) for picking a CJK face.
I18n::rtl uses direction.
Skylar's review of 886a0b3:
1. tc/tid go locale by locale: in each locale the context or @id row,
then the plain row, so pt-br's plain row beats pt's context row.
2. pt (and pt-* other than pt-br) uses pt-PT plural rules: 0 is plural.
3. A script subtag (sr-Latn-RS, sr_RS@latin, be_BY@latin) never picks a
code of another script; sr-Cyrl-RS still picks sr.
4. A row defined twice is reported and the first kept, as in the checker.
5. The pseudo locale is debug-only via from_code, get marks too, and
looked_up() lists what was looked up for an app's audit.
6. intl_pluralrules pinned to =7.0.2; plural_form_counts copied from the
checker, plus pt.
7. I18n::new takes the app's offered codes: languages(), from_code,
match_tag and system_lang only return those; others stay fallbacks.
Smaller: en/pseudo/repeated codes are skipped and reported,
system_lang is cached, zh-hant-hk falls back to zh-hant, rtl adds ckb,
dv, ps, sd, ug and yi.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review fixes (CodeRabbit): a glibc @latin/@cyrillic modifier becomes the
script subtag and only codes with that script are tried. sys-locale
drops the modifier, so on desktop Unix system_lang reads LANGUAGE,
LC_ALL, LC_MESSAGES and LANG raw, in its order. README: tl! qualifies
CURRENT with $crate; get follows the locale chain but never the plain
row or English.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Final review fixes: a European Portuguese locale wins over pt-br when both
are embedded; sr-latn no longer falls back to Cyrillic sr; get() looks up
one context with no plain or English fallback (key-cap names).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9fM3mv1PUyKpk5rdwUQrW
aiongg