pauljd/vectorcraft
1
0
Fork 0
mirror of https://github.com/storytold/vectorcraft.git synced 2026-11-14 00:44:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

JavaScript: characters assigned in Unicode 17 are identifier characters

Browse Source 
vectorcraft-js and Check Script took identifier categories from
unicode-general-category 1.1.0, which carries Unicode 16.0, so letters
and marks assigned in Unicode 17.0 (U+088F at the start of a name,
U+1ACF after its first character) were refused. Both lexers now read
general categories from unicode-properties 0.1.4 (Unicode 17.0), which
Cargo.lock already holds. The version is exact because 0.1.5 adds the
Unicode-3.0 license, which is not on the license allowlist. The
Other_ID_Start, Other_ID_Continue and Pattern_Syntax lists are the same
in PropList 17.0.

The exhaustive identifier test now expects the DerivedCoreProperties
17.0 totals (145,916 ID_Start, 149,240 ID_Continue), and new tests read
U+088F and U+1ACF written out and as escapes in both lexers.
This commit is contained in:
dotCooCoo
2026-11-11 13:57:54 -08:00
parent 566f720868
commit 283d127e2a
9 changed files with 59 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/scriptcheck/src/tests.rs
+6 -1
View File
@@ -278,12 +278,14 @@ fn unescaped(src: &str) -> String {
fn names_are_made_of_the_unicode_identifier_characters() {
// U+2118 (Other_ID_Start) starts a name, and U+203F (connector punctuation), U+00B7
// (Other_ID_Continue) and the zero width non-joiner and joiner continue one.
for name in ["\u{2118}a", "a\u{203F}b", "a\u{B7}b", "a\u{200C}b", "a\u{200D}b", "_\u{2118}", "$_a1", "\u{E9}t\u{E9}"] {
// Unicode 17.0 assigned the letter U+088F and the combining mark U+1ACF.
for name in ["\u{2118}a", "a\u{203F}b", "a\u{B7}b", "a\u{200C}b", "a\u{200D}b", "_\u{2118}", "$_a1", "\u{E9}t\u{E9}", "\u{88F}", "_\u{1ACF}"] {
assert_eq!(tokens(name), [(T::Ident, name)], "{name:?}");
}
for (src, want) in [
// Characters that only continue a name, and digits, don't start one.
("\u{203F}a", [(T::Other, "\u{203F}"), (T::Ident, "a")]),
("\u{1ACF}a", [(T::Other, "\u{1ACF}"), (T::Ident, "a")]),
("\u{B7}a", [(T::Other, "\u{B7}"), (T::Ident, "a")]),
("\u{200C}a", [(T::Other, "\u{200C}"), (T::Ident, "a")]),
("0a", [(T::Number, "0"), (T::Ident, "a")]),
@@ -318,8 +320,11 @@ fn an_escape_stands_for_a_character_only_where_a_name_may_hold_it() {
("a\\u200Db", "a\u{200D}b"),
("a\\u0030", "a0"),
("\\u0024\\u005F", "$_"),
("\\u088F", "\u{88F}"),
("_\\u1ACF", "_\u{1ACF}"),
// Not at the start of a name.
("\\u0030abc", "\\u0030abc"),
("\\u1ACFa", "\\u1ACFa"),
("\\u203Fa", "\\u203Fa"),
("\\u00B7a", "\\u00B7a"),
// In no name.