diff --git a/Cargo.lock b/Cargo.lock index cec8e7aef7..cb0c72cc0c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -6775,6 +6775,7 @@ dependencies = [ "craft-stream-codecs", "craft-xml", "crc32fast", + "encoding_rs", "flate2", "hayro-ccitt", "heic-rs", diff --git a/crates/engine/Cargo.toml b/crates/engine/Cargo.toml index 9f6ce7b710..a8209b9b42 100644 --- a/crates/engine/Cargo.toml +++ b/crates/engine/Cargo.toml @@ -59,6 +59,9 @@ chrono = { workspace = true } craft-xml = { workspace = true } # The compound file that holds a Word 97-2003 document placed as type (place/doc.rs). cfb = "0.15" +# The Text Import Options character sets other than Unicode and ANSI (place/text.rs). Already in +# Cargo.lock (vectorcraft-eps), so this adds no new crate. +encoding_rs = { workspace = true } kurbo = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } diff --git a/crates/engine/src/cmd/place.rs b/crates/engine/src/cmd/place.rs index 486a5633a7..096f90bf36 100644 --- a/crates/engine/src/cmd/place.rs +++ b/crates/engine/src/cmd/place.rs @@ -82,7 +82,7 @@ pub fn specs() -> Vec { "Place…", ["File"], Some("Cmd+Shift+P"), - "{path | name | dataBase64 (a file's path, or its name with its bytes), link?: true (a raster image keeps its file's path; a VectorCraft document at path becomes one locked placed document showing artboard `page`, linked to the file and read again when it changes (links.update; saving it in this app updates it at once), drawn as vectors in every output; so does a PDF/.ai page at path, showing its `crop` box, unless password or textAs: \"outlines\" is given, and an EPS file, a PostScript file or a PostScript .ai at path, showing the art false gives; false: an editable copy of its art, as from dataBase64; other files are embedded), text?: {characterSet?: \"unicode\" (UTF-8, or UTF-16 with a byte-order mark, or without one by dataset.import's rule: an even length, at least half of its two-byte units holding a NUL byte, the byte order that reads a line break, valid UTF-16 with no NUL character; other bytes as the platform's 8-bit set) | \"ansi\" (the platform's 8-bit set), platform?: \"windows\" (Windows-1252) | \"mac\" (Mac Roman), removeLineReturns?: false (each block of lines becomes one paragraph; blank lines end paragraphs), removeParagraphReturns?: false (drop blank lines), replaceSpaces?: n (runs of n ≥ 2 spaces become a tab)} (a .txt file, placed as area type filling rect, the replaced object's bounds, or else the artboard less a 36 pt margin), template?: false (onto a new locked template layer below the current layer), replace?: false (swap the one selected object, keeping its stacking place and transform; no at/rect; refused while an object of a symbol instance's art is selected), at?: [x, y] centre (default: the first artboard's centre), rect?: [x, y, width, height] fit inside; aspect kept (wins over at), page?: 1 (PDF/.ai page, or a native document's artboard), crop?: \"crop\" (clipped to that page or artboard, default; an SVG, only when given: framed and clipped by its viewBox, else by its art's bounds) | \"bounding\" (the bounds of the art that shows, with its strokes and effects, as an export measures them: hidden objects and guides left out) | \"art\" | \"trim\" | \"bleed\" | \"media\" (a PDF page's boxes), password? (an encrypted PDF), layers?: false (a PSD or PSB file: true embeds its layers as one group named after the file, read as document.open reads them: an image for each layer and a group for each layer group, with the layer's name, visibility, opacity, blend mode and opacity mask, at the place its merged image would take and clipped to the canvas when layers, hidden ones included, reach past it; never linked: link is then false unless given, and link: true is refused; a file without layers is embedded as its merged image, and so are one whose layers are all left out and one whose layers can't be read, with a warning), hiddenLayers?: false (with layers: true, the hidden layers come in hidden instead of being left out; refused without it), importSlices?: bool (a PSD or PSB file: its slices become user slices, moved and scaled with the placed art, in the same undo step; default true when the file is embedded, false when it is linked, and true with link is refused; slices the file generated automatically are left out), dxf?: {…the DXF options of document.open} (a .dxf drawing: fit fills the artboard under at, else the first; fitted or uncentred art lands with its drawing's artboard on that artboard, else it is centred on at), showImportOptions?: true (in the app, a text file without text, a DXF drawing without dxf, a PSD or PSB file embedded without layers (with two layers or more, or with slices) and a PDF with several pages or a password and no page ask for those options first, and an encrypted PDF given page and no password asks for its password; false places them with the options last confirmed, a PDF on page 1 (or page), and asks only an encrypted PDF's password)} → {ids, name, format, linked, width, height, warnings, image?: the id of the image of a DCS file or PSD/PSB file placed in a clip group, which the Links commands take, slices?: [the ids of the user slices a PSD file brought in]}. A PSD or PSB file that names a clipping path places in a clip group of that path (even-odd), as does its merged image in document.open. Raster images come in at 100% of their physical size (the file's ppi, else 72); SVG, DXF, HP-GL (.plt, .hpgl, .hgl, .hpg), PDF/.ai, EMF/WMF and PICT (.pct, .pic, .pict; clipped to the picture's frame), EPS (clipped to its bounding box; as document.open reads it), a .ai saved without PDF compatibility (from its editing data, as document.open reads it: the art of the layers that show on artboard `page`, clipped to that artboard unless crop is \"bounding\") and native documents as one group, with the images, symbols, patterns, swatches and brushes they use. A DCS file (an EPS file with one program per printing plate, DCS 1.0 or 2.0) is placed only from its path with link (else an error, as are document.open and links.embed): one linked image of its plates mixed into CMYK, clipped by the clipping path its composite is drawn through, its spot plates added as spot swatches in the process equivalents the file gives (a swatch of the same name is kept, with a warning); print separations and Separations Preview show each plate on its own ink, and plate files of their own are read from its folder. One undo step; selects what it placed (unless on a template layer); never touches the clipboard. A PNG, JPEG, GIF or WebP image whose pixels can't be read (a file cut short) is an error, and nothing is placed", + "{path | name | dataBase64 (a file's path, or its name with its bytes), link?: true (a raster image keeps its file's path; a VectorCraft document at path becomes one locked placed document showing artboard `page`, linked to the file and read again when it changes (links.update; saving it in this app updates it at once), drawn as vectors in every output; so does a PDF/.ai page at path, showing its `crop` box, unless password or textAs: \"outlines\" is given, and an EPS file, a PostScript file or a PostScript .ai at path, showing the art false gives; false: an editable copy of its art, as from dataBase64; other files are embedded), text?: {characterSet?: \"unicode\" (UTF-8, or UTF-16 with a byte-order mark, or without one by dataset.import's rule: an even length, at least half of its two-byte units holding a NUL byte, the byte order that reads a line break, valid UTF-16 with no NUL character; other bytes as the platform's 8-bit set) | \"ansi\" (the platform's 8-bit set) | \"shiftJis\" | \"gb2312\" (read as GBK) | \"chineseBig5\" | \"cyrillic\" (windows-1251, x-mac-cyrillic on mac) | \"gb18030\" | \"greek\" (windows-1253) | \"turkish\" (windows-1254) | \"baltic\" (windows-1257) | \"centralEuropean\" (windows-1250) (the WHATWG Encoding Standard's decoders: a byte-order mark is read as ordinary bytes, a malformed sequence as U+FFFD), platform?: \"windows\" (ANSI is Windows-1252) | \"mac\" (ANSI is Mac Roman), removeLineReturns?: false (each block of lines becomes one paragraph; blank lines end paragraphs), removeParagraphReturns?: false (drop blank lines), replaceSpaces?: n (runs of n ≥ 2 spaces become a tab)} (a .txt file, placed as area type filling rect, the replaced object's bounds, or else the artboard less a 36 pt margin), template?: false (onto a new locked template layer below the current layer), replace?: false (swap the one selected object, keeping its stacking place and transform; no at/rect; refused while an object of a symbol instance's art is selected), at?: [x, y] centre (default: the first artboard's centre), rect?: [x, y, width, height] fit inside; aspect kept (wins over at), page?: 1 (PDF/.ai page, or a native document's artboard), crop?: \"crop\" (clipped to that page or artboard, default; an SVG, only when given: framed and clipped by its viewBox, else by its art's bounds) | \"bounding\" (the bounds of the art that shows, with its strokes and effects, as an export measures them: hidden objects and guides left out) | \"art\" | \"trim\" | \"bleed\" | \"media\" (a PDF page's boxes), password? (an encrypted PDF), layers?: false (a PSD or PSB file: true embeds its layers as one group named after the file, read as document.open reads them: an image for each layer and a group for each layer group, with the layer's name, visibility, opacity, blend mode and opacity mask, at the place its merged image would take and clipped to the canvas when layers, hidden ones included, reach past it; never linked: link is then false unless given, and link: true is refused; a file without layers is embedded as its merged image, and so are one whose layers are all left out and one whose layers can't be read, with a warning), hiddenLayers?: false (with layers: true, the hidden layers come in hidden instead of being left out; refused without it), importSlices?: bool (a PSD or PSB file: its slices become user slices, moved and scaled with the placed art, in the same undo step; default true when the file is embedded, false when it is linked, and true with link is refused; slices the file generated automatically are left out), dxf?: {…the DXF options of document.open} (a .dxf drawing: fit fills the artboard under at, else the first; fitted or uncentred art lands with its drawing's artboard on that artboard, else it is centred on at), showImportOptions?: true (in the app, a text file without text, a DXF drawing without dxf, a PSD or PSB file embedded without layers (with two layers or more, or with slices) and a PDF with several pages or a password and no page ask for those options first, and an encrypted PDF given page and no password asks for its password; false places them with the options last confirmed, a PDF on page 1 (or page), and asks only an encrypted PDF's password)} → {ids, name, format, linked, width, height, warnings, image?: the id of the image of a DCS file or PSD/PSB file placed in a clip group, which the Links commands take, slices?: [the ids of the user slices a PSD file brought in]}. A PSD or PSB file that names a clipping path places in a clip group of that path (even-odd), as does its merged image in document.open. Raster images come in at 100% of their physical size (the file's ppi, else 72); SVG, DXF, HP-GL (.plt, .hpgl, .hgl, .hpg), PDF/.ai, EMF/WMF and PICT (.pct, .pic, .pict; clipped to the picture's frame), EPS (clipped to its bounding box; as document.open reads it), a .ai saved without PDF compatibility (from its editing data, as document.open reads it: the art of the layers that show on artboard `page`, clipped to that artboard unless crop is \"bounding\") and native documents as one group, with the images, symbols, patterns, swatches and brushes they use. A DCS file (an EPS file with one program per printing plate, DCS 1.0 or 2.0) is placed only from its path with link (else an error, as are document.open and links.embed): one linked image of its plates mixed into CMYK, clipped by the clipping path its composite is drawn through, its spot plates added as spot swatches in the process equivalents the file gives (a swatch of the same name is kept, with a warning); print separations and Separations Preview show each plate on its own ink, and plate files of their own are read from its folder. One undo step; selects what it placed (unless on a template layer); never touches the clipboard. A PNG, JPEG, GIF or WebP image whose pixels can't be read (a file cut short) is an error, and nothing is placed", has_doc, place ), diff --git a/crates/engine/src/cmd/place/text.rs b/crates/engine/src/cmd/place/text.rs index 7d8afc92d4..78261a8804 100644 --- a/crates/engine/src/cmd/place/text.rs +++ b/crates/engine/src/cmd/place/text.rs @@ -3,8 +3,8 @@ use serde_json::Value; -use super::super::utf16_order_without_mark; -use super::{Point, Rect, Result, bad, bool_or, fileio, shapes, str_param}; +use super::super::{text_param, utf16_order_without_mark}; +use super::{Point, Rect, Result, bad, fileio, shapes, switch_or}; /// Text files as `file.place` reports them (not a format `document.open` reads). pub(super) const FORMAT: fileio::Format = fileio::Format { @@ -22,13 +22,68 @@ pub(super) const FORMAT: fileio::Format = fileio::Format { /// document may give. pub(super) const MAX_TEXT: usize = 4 << 20; +/// A character set of Text Import Options (`text.characterSet`). +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +enum CharacterSet { + /// UTF-8, or UTF-16 with or without a byte-order mark ([`decode`]). + #[default] + Unicode, + /// The platform's 8-bit character set: Windows-1252, or Mac Roman on a Mac. + Ansi, + ShiftJis, + Gb2312, + ChineseBig5, + Cyrillic, + Gb18030, + Greek, + Turkish, + Baltic, + CentralEuropean, +} + +/// Each character set by its `text.characterSet` value. +const CHARACTER_SETS: [(&str, CharacterSet); 11] = [ + ("unicode", CharacterSet::Unicode), + ("ansi", CharacterSet::Ansi), + ("shiftJis", CharacterSet::ShiftJis), + ("gb2312", CharacterSet::Gb2312), + ("chineseBig5", CharacterSet::ChineseBig5), + ("cyrillic", CharacterSet::Cyrillic), + ("gb18030", CharacterSet::Gb18030), + ("greek", CharacterSet::Greek), + ("turkish", CharacterSet::Turkish), + ("baltic", CharacterSet::Baltic), + ("centralEuropean", CharacterSet::CentralEuropean), +]; + +impl CharacterSet { + /// The Encoding Standard decoder of a set other than Unicode and ANSI, for a file from a Mac + /// when `mac`: GB2312 reads as GBK, Cyrillic as windows-1251 (x-mac-cyrillic on a Mac), and + /// Greek, Turkish, Baltic and Central European as windows-1253, 1254, 1257 and 1250 on both + /// platforms. + fn encoding(self, mac: bool) -> Option<&'static encoding_rs::Encoding> { + Some(match self { + Self::Unicode | Self::Ansi => return None, + Self::ShiftJis => encoding_rs::SHIFT_JIS, + Self::Gb2312 => encoding_rs::GBK, + Self::ChineseBig5 => encoding_rs::BIG5, + Self::Cyrillic if mac => encoding_rs::X_MAC_CYRILLIC, + Self::Cyrillic => encoding_rs::WINDOWS_1251, + Self::Gb18030 => encoding_rs::GB18030, + Self::Greek => encoding_rs::WINDOWS_1253, + Self::Turkish => encoding_rs::WINDOWS_1254, + Self::Baltic => encoding_rs::WINDOWS_1257, + Self::CentralEuropean => encoding_rs::WINDOWS_1250, + }) + } +} + /// Text Import Options (`file.place`'s `text` param). #[derive(Clone, Copy, Debug, Default, PartialEq)] pub(super) struct TextOptions { - /// Decode as the platform's 8-bit character set (Windows-1252 or Mac Roman) instead of - /// Unicode. - ansi: bool, - /// The file comes from a Mac (its 8-bit character set is Mac Roman). + /// The character set the file's bytes are decoded as. + character_set: CharacterSet, + /// The file comes from a Mac (its ANSI set is Mac Roman, its Cyrillic set x-mac-cyrillic). mac: bool, /// Join the lines of each block of text (blank lines end paragraphs). remove_line_returns: bool, @@ -45,11 +100,13 @@ impl TextOptions { if !t.is_object() { return Err(bad(cmd, "text must be {characterSet?, platform?, removeLineReturns?, removeParagraphReturns?, replaceSpaces?}")); } - let choice = |key: &str, values: [&str; 2]| match str_param(t, key) { - None => Ok(false), - Some(v) if v == values[0] => Ok(false), - Some(v) if v == values[1] => Ok(true), - Some(v) => Err(bad(cmd, format!("text.{key} `{v}`: use \"{}\" or \"{}\"", values[0], values[1]))), + // The index of `text[key]` in `values`; 0 when it is omitted or null. + let choice = |key: &str, values: &[&str]| -> Result { + let Some(v) = text_param(t, key, cmd)? else { return Ok(0) }; + values.iter().position(|x| *x == v).ok_or_else(|| { + let (last, rest) = values.split_last().map_or(("", &[][..]), |(last, rest)| (*last, rest)); + bad(cmd, format!("text.{key} `{v}`: use \"{}\" or \"{}\"", rest.join("\", \""), last)) + }) }; let spaces_to_tab = match t.get("replaceSpaces") { None | Some(Value::Null) => None, @@ -58,11 +115,12 @@ impl TextOptions { as usize, ), }; + let set = choice("characterSet", &CHARACTER_SETS.map(|(name, _)| name))?; Ok(Self { - ansi: choice("characterSet", ["unicode", "ansi"])?, - mac: choice("platform", ["windows", "mac"])?, - remove_line_returns: bool_or(t, "removeLineReturns", false), - remove_paragraph_returns: bool_or(t, "removeParagraphReturns", false), + character_set: CHARACTER_SETS.get(set).map_or(CharacterSet::Unicode, |(_, s)| *s), + mac: choice("platform", &["windows", "mac"])? == 1, + remove_line_returns: switch_or(t, "removeLineReturns", false, cmd)?, + remove_paragraph_returns: switch_or(t, "removeParagraphReturns", false, cmd)?, spaces_to_tab, }) } @@ -80,10 +138,15 @@ pub(super) fn import(bytes: &[u8], o: TextOptions, cmd: &str) -> Result /// `bytes` as text: Unicode (UTF-8, or UTF-16 with a byte-order mark, or without one when the /// file has an even length, reads as valid UTF-16 with no NUL character in the byte order /// [`utf16_order_without_mark`] gives; bytes that aren't UTF-8 are read as the platform's 8-bit -/// set), or the platform's 8-bit character set. +/// set), the platform's 8-bit character set, or another set's decoder +/// ([`CharacterSet::encoding`]), which reads a byte-order mark as ordinary bytes and a malformed +/// sequence as U+FFFD. fn decode(bytes: &[u8], o: TextOptions) -> String { + if let Some(encoding) = o.character_set.encoding(o.mac) { + return encoding.decode_without_bom_handling(bytes).0.into_owned(); + } let eight_bit = |b: &[u8]| -> String { b.iter().map(|&c| if o.mac { mac_roman(c) } else { windows_1252(c) }).collect() }; - if o.ansi { + if o.character_set == CharacterSet::Ansi { return eight_bit(bytes); } let utf16 = |rest: &[u8], le: bool| { @@ -278,22 +341,76 @@ mod tests { assert_eq!(decode(&[0xFF, 0xFE, b'h', 0, 0xE9, 0], unicode), "hé", "UTF-16 LE"); assert_eq!(decode(&[0xFE, 0xFF, 0, b'h', 0x20, 0xAC], unicode), "h€", "UTF-16 BE"); assert_eq!(decode(&[b'c', b'a', b'f', 0xE9, 0x80], unicode), "café€", "not UTF-8: Windows-1252"); - let ansi = TextOptions { ansi: true, ..TextOptions::default() }; + let ansi = TextOptions { character_set: CharacterSet::Ansi, ..TextOptions::default() }; assert_eq!(decode(&[0x93, b'q', 0x94, 0xE9], ansi), "“q”é"); - let mac = TextOptions { ansi: true, mac: true, ..TextOptions::default() }; + let mac = TextOptions { character_set: CharacterSet::Ansi, mac: true, ..TextOptions::default() }; assert_eq!(decode(&[0xD2, b'q', 0xD3, 0x8E, 0xA5, 0xDB], mac), "“q”é•€"); } + /// The other character sets decode by their Encoding Standard decoders: the platform matters + /// for Cyrillic alone, a byte-order mark is read as ordinary bytes, and a malformed sequence + /// reads as U+FFFD. + #[test] + fn decodes_the_other_character_sets() { + let cases: [(CharacterSet, &[u8], &str); 9] = [ + (CharacterSet::ShiftJis, &[0x93, 0xFA, 0x96, 0x7B], "日本"), + (CharacterSet::Gb2312, &[0xD6, 0xD0, 0xCE, 0xC4], "中文"), + (CharacterSet::ChineseBig5, &[0xA4, 0xA4, 0xA4, 0xE5], "中文"), + (CharacterSet::Cyrillic, &[0xCF, 0xF0, 0xE8, 0xE2, 0xE5, 0xF2], "Привет"), + (CharacterSet::Gb18030, &[0x95, 0x32, 0x82, 0x36], "\u{20000}"), + (CharacterSet::Greek, &[0xC5, 0xEB, 0xEB, 0xDC, 0xE4, 0xE1], "Ελλάδα"), + (CharacterSet::Turkish, &[0xDD, 0x73, 0x74], "İst"), + (CharacterSet::Baltic, &[0x52, 0xEE, 0x67, 0x61], "Rīga"), + (CharacterSet::CentralEuropean, &[0xA3, 0xF3, 0x64, 0x9F], "Łódź"), + ]; + for (set, bytes, text) in cases { + let windows = TextOptions { character_set: set, ..TextOptions::default() }; + assert_eq!(decode(bytes, windows), text, "{set:?}"); + if set != CharacterSet::Cyrillic { + assert_eq!(decode(bytes, TextOptions { mac: true, ..windows }), text, "{set:?} from a Mac"); + } + } + let mac_cyrillic = TextOptions { character_set: CharacterSet::Cyrillic, mac: true, ..TextOptions::default() }; + assert_eq!(decode(&[0x8F, 0xF0, 0xE8, 0xE2, 0xE5, 0xF2], mac_cyrillic), "Привет"); + let shift_jis = TextOptions { character_set: CharacterSet::ShiftJis, ..TextOptions::default() }; + assert_eq!(decode(&[0x82, 0x22], shift_jis), "\u{FFFD}\"", "a lead byte before ASCII"); + assert_eq!(decode(&[b'a', 0x82], shift_jis), "a\u{FFFD}", "a lead byte at the end"); + let central = TextOptions { character_set: CharacterSet::CentralEuropean, ..TextOptions::default() }; + assert_eq!(decode(&[0xEF, 0xBB, 0xBF, b'A'], central), "A", "a UTF-8 byte-order mark is read as windows-1250"); + assert_eq!(decode(&[0xFF, 0xFE, b'A', 0], shift_jis), "\u{FFFD}\u{FFFD}A\0", "a UTF-16 byte-order mark is read as Shift JIS"); + } + #[test] fn options_parse_and_reject_bad_values() { let p = serde_json::json!({"text": {"characterSet": "ansi", "platform": "mac", "removeLineReturns": true, "replaceSpaces": 4}}); let o = TextOptions::parse(&p, "file.place").unwrap(); - assert_eq!(o, TextOptions { ansi: true, mac: true, remove_line_returns: true, remove_paragraph_returns: false, spaces_to_tab: Some(4) }); + assert_eq!( + o, + TextOptions { + character_set: CharacterSet::Ansi, + mac: true, + remove_line_returns: true, + remove_paragraph_returns: false, + spaces_to_tab: Some(4) + } + ); + for (name, set) in CHARACTER_SETS { + let o = TextOptions::parse(&serde_json::json!({"text": {"characterSet": name}}), "file.place").unwrap(); + assert_eq!(o.character_set, set, "{name}"); + } assert_eq!(TextOptions::parse(&serde_json::json!({}), "file.place").unwrap(), TextOptions::default()); - for bad in - [serde_json::json!({"text": 3}), serde_json::json!({"text": {"platform": "amiga"}}), serde_json::json!({"text": {"replaceSpaces": 1}})] - { + for bad in [ + serde_json::json!({"text": 3}), + serde_json::json!({"text": {"platform": "amiga"}}), + serde_json::json!({"text": {"replaceSpaces": 1}}), + serde_json::json!({"text": {"characterSet": 3}}), + serde_json::json!({"text": {"platform": true}}), + serde_json::json!({"text": {"removeLineReturns": "yes"}}), + serde_json::json!({"text": {"removeParagraphReturns": 1}}), + ] { assert!(TextOptions::parse(&bad, "file.place").is_err(), "{bad}"); } + let null = serde_json::json!({"text": {"characterSet": null, "platform": null, "removeLineReturns": null}}); + assert_eq!(TextOptions::parse(&null, "file.place").unwrap(), TextOptions::default(), "null leaves the default"); } } diff --git a/crates/engine/src/tests/textimport.rs b/crates/engine/src/tests/textimport.rs index cc89a8261b..980682f57e 100644 --- a/crates/engine/src/tests/textimport.rs +++ b/crates/engine/src/tests/textimport.rs @@ -135,6 +135,36 @@ fn text_files_report_and_queue_with_their_options() { assert!(close(frame, Rect::new(50.0, 60.0, 130.0, 100.0)), "{frame:?}"); } +/// Text Import Options' other character sets place their text as area type, and an unknown +/// character set is refused with the list of sets and places nothing. +#[test] +fn text_files_place_in_the_other_character_sets() { + let mut s = session(); + let cases: [(&str, &str, &[u8], &str); 11] = [ + ("shiftJis", "windows", &[0x93, 0xFA, 0x96, 0x7B], "日本"), + ("gb2312", "windows", &[0xD6, 0xD0, 0xCE, 0xC4], "中文"), + ("chineseBig5", "mac", &[0xA4, 0xA4, 0xA4, 0xE5], "中文"), + ("cyrillic", "windows", &[0xCF, 0xF0, 0xE8, 0xE2, 0xE5, 0xF2], "Привет"), + ("cyrillic", "mac", &[0x8F, 0xF0, 0xE8, 0xE2, 0xE5, 0xF2], "Привет"), + ("gb18030", "windows", &[0x95, 0x32, 0x82, 0x36], "\u{20000}"), + ("greek", "windows", &[0xC5, 0xEB, 0xEB, 0xDC, 0xE4, 0xE1], "Ελλάδα"), + ("turkish", "mac", &[0xDD, 0x73, 0x74], "İst"), + ("baltic", "windows", &[0x52, 0xEE, 0x67, 0x61], "Rīga"), + ("centralEuropean", "windows", &[0xA3, 0xF3, 0x64, 0x9F, b'\r', b'\n', 0xA3], "Łódź\nŁ"), + ("shiftJis", "windows", &[0x82, 0x22], "\u{FFFD}\""), + ]; + for (set, platform, bytes, text) in cases { + let r = place_text(&mut s, bytes, json!({"text": {"characterSet": set, "platform": platform}})); + assert_eq!(placed(&s, &r).0, text, "{set} on {platform}"); + } + let undo = s.doc().unwrap().history.undo.len(); + let bad = json!({"name": "a.txt", "dataBase64": "YQ==", "text": {"characterSet": "latin9"}}); + let err = s.execute("file.place", &bad).unwrap_err().to_string(); + let list = r#""unicode", "ansi", "shiftJis", "gb2312", "chineseBig5", "cyrillic", "gb18030", "greek", "turkish", "baltic" or "centralEuropean""#; + assert!(err.contains(&format!("text.characterSet `latin9`: use {list}")), "{err}"); + assert_eq!(s.doc().unwrap().history.undo.len(), undo, "nothing placed"); +} + /// An RTF file with a font table, a color table, a bold run, an italic run, a superscript and a /// centered, indented second paragraph. const LETTER_RTF: &str = concat!( diff --git a/crates/engine/tests/text_import_fuzz.rs b/crates/engine/tests/text_import_fuzz.rs index 1ea656a158..c8417b73fc 100644 --- a/crates/engine/tests/text_import_fuzz.rs +++ b/crates/engine/tests/text_import_fuzz.rs @@ -17,7 +17,9 @@ proptest! { fn any_bytes_place_as_text_without_panics( bytes in prop::collection::vec(prop_oneof![any::(), Just(b'\r'), Just(b'\n'), Just(b' ')], 0..400), bom in prop::sample::select(vec![&b""[..], b"\xEF\xBB\xBF", b"\xFF\xFE", b"\xFE\xFF"]), - ansi in any::(), + set in prop::sample::select(vec![ + "unicode", "ansi", "shiftJis", "gb2312", "chineseBig5", "cyrillic", "gb18030", "greek", "turkish", "baltic", "centralEuropean", + ]), mac in any::(), lines in any::(), paras in any::(), @@ -31,7 +33,7 @@ proptest! { let mut s = vectorcraft_engine::Session::new(); s.execute("file.new", &json!({"width": 100, "height": 100})).unwrap(); let text = json!({ - "characterSet": if ansi { "ansi" } else { "unicode" }, + "characterSet": set, "platform": if mac { "mac" } else { "windows" }, "removeLineReturns": lines, "removeParagraphReturns": paras, diff --git a/crates/ui-egui/src/dialogs/text_import.rs b/crates/ui-egui/src/dialogs/text_import.rs index 99128cc321..f0fe62b40d 100644 --- a/crates/ui-egui/src/dialogs/text_import.rs +++ b/crates/ui-egui/src/dialogs/text_import.rs @@ -3,7 +3,7 @@ //! places it as area type (`file.place` with `text`) and remembers the options: the next text file //! starts from them, and is placed with them when Show Import Options is off. //! -//! Fields: `platform` (`windows` | `mac`), `characterSet` (`unicode` | `ansi`), +//! Fields: `platform` (`windows` | `mac`), `characterSet` (one of [`CHARACTER_SETS`]), //! `removeLineReturns`, `removeParagraphReturns`, `replaceSpaces` (on or off), `spaces` (how many //! spaces make a tab, 2–100), and `__params` (the `file.place` params naming the file). @@ -22,6 +22,21 @@ pub(super) const SPEC: DialogSpec = /// The label column of the Encoding rows. const LABEL: f32 = 104.0; +/// The Character Set choices: `file.place`'s `text.characterSet` value and the menu label. +const CHARACTER_SETS: [(&str, &str); 11] = [ + ("unicode", "Unicode"), + ("ansi", "ANSI"), + ("shiftJis", "Shift JIS"), + ("gb2312", "GB2312"), + ("chineseBig5", "Chinese Big 5"), + ("cyrillic", "Cyrillic"), + ("gb18030", "GB18030"), + ("greek", "Greek"), + ("turkish", "Turkish"), + ("baltic", "Baltic"), + ("centralEuropean", "Central European"), +]; + /// Is the file `p` names (`path` or `name`) a text file? pub fn is_text(p: &Value) -> bool { let name = p.get("path").or_else(|| p.get("name")).and_then(Value::as_str).unwrap_or_default(); @@ -41,7 +56,8 @@ pub fn open(app: &mut VectorcraftApp, params: &Value) -> Value { "__params": params, }); let last = &app.ui.text_import; - for (k, wanted) in [("platform", ["windows", "mac"]), ("characterSet", ["unicode", "ansi"])] { + let sets = CHARACTER_SETS.map(|(value, _)| value); + for (k, wanted) in [("platform", &["windows", "mac"][..]), ("characterSet", &sets[..])] { if let Some(v) = last.get(k).and_then(Value::as_str).filter(|v| wanted.contains(v)) { fields[k] = json!(v); } @@ -68,7 +84,7 @@ pub fn last_options(app: &VectorcraftApp) -> Value { fn body(_: &mut VectorcraftApp, ui: &mut egui::Ui, d: &mut Dialog) -> bool { widgets::subheader(ui, tl!("Encoding")); form::choice(ui, d, "platform", tl!("Platform:"), (LABEL, 180.0), &[("windows", "Windows"), ("mac", "Mac")]); - form::choice(ui, d, "characterSet", tl!("Character Set:"), (LABEL, 180.0), &[("unicode", "Unicode"), ("ansi", "ANSI")]); + form::choice(ui, d, "characterSet", tl!("Character Set:"), (LABEL, 180.0), &CHARACTER_SETS); ui.add_space(10.0); widgets::subheader(ui, tl!("Line Breaks")); for (key, label) in @@ -166,6 +182,39 @@ mod tests { assert_eq!(st.history.undo.len(), 1); } + /// Every character set is a choice: one confirmed last comes back on the next text file and + /// decodes it, an unknown remembered value falls back to Unicode, and with Show Import Options + /// off the remembered set is the one the file is placed with. + #[test] + fn a_remembered_character_set_comes_back_and_places_the_file() { + let mut app = VectorcraftApp::new(Session::new(), Default::default()); + app.run("file.new", json!({"width": 300, "height": 200})).unwrap(); + let file = json!({"name": "notes.txt", "dataBase64": vectorcraft_format::base64_encode(&[0x93, 0xFA, 0x96, 0x7B])}); + app.ui.text_import = json!({"characterSet": "shiftJis", "platform": "windows"}); + app.run("file.place", file.clone()).unwrap(); + assert_eq!(app.ui.dialog.as_ref().map(|d| d.str("characterSet")).as_deref(), Some("shiftJis")); + assert!(frame(&mut app).contains("Shift JIS"), "the remembered set is shown"); + let r = super::super::confirm(&mut app).unwrap(); + let st = app.session.doc().unwrap(); + let n = st.doc.node(vectorcraft_engine::doc::NodeId(r["ids"][0].as_u64().unwrap())).unwrap(); + let NodeKind::Text(t) = &n.kind else { panic!("type") }; + assert_eq!(t.plain_text(), "日本"); + assert_eq!(last_options(&app)["characterSet"], "shiftJis", "remembered for Show Import Options off"); + app.ui.text_import = json!({"characterSet": "latin9"}); + app.run("file.place", file).unwrap(); + assert_eq!(app.ui.dialog.as_ref().map(|d| d.str("characterSet")).as_deref(), Some("unicode")); + let values: Vec<&str> = CHARACTER_SETS.iter().map(|(v, _)| *v).collect(); + let labels: Vec<&str> = CHARACTER_SETS.iter().map(|(_, l)| *l).collect(); + assert_eq!( + values, + ["unicode", "ansi", "shiftJis", "gb2312", "chineseBig5", "cyrillic", "gb18030", "greek", "turkish", "baltic", "centralEuropean"] + ); + assert_eq!( + labels, + ["Unicode", "ANSI", "Shift JIS", "GB2312", "Chinese Big 5", "Cyrillic", "GB18030", "Greek", "Turkish", "Baltic", "Central European"] + ); + } + /// A text file dropped on a canvas is placed by the dialog's OK in the document it was dropped /// on, even when another document became active meanwhile. #[test] diff --git a/crates/ui-egui/src/i18n/bg.tsv b/crates/ui-egui/src/i18n/bg.tsv index 92713fa6d1..3d3811c403 100644 --- a/crates/ui-egui/src/i18n/bg.tsv +++ b/crates/ui-egui/src/i18n/bg.tsv @@ -408,6 +408,12 @@ font-technology Variable Променливи Character Alignment Подравняване на знаците Character / Paragraph Знак / Абзац Character Rotation Въртене на знак + Chinese Big 5 Китайски Big 5 + Cyrillic Кирилица + Greek Гръцки + Turkish Турски + Baltic Балтийски + Central European Централноевропейски Character Set: Набор от знаци: Character Style Стил на знак Character Style Options Опции за стил на знак diff --git a/crates/ui-egui/src/i18n/bs.tsv b/crates/ui-egui/src/i18n/bs.tsv index b2eb1e76ab..b053827273 100644 --- a/crates/ui-egui/src/i18n/bs.tsv +++ b/crates/ui-egui/src/i18n/bs.tsv @@ -3326,6 +3326,12 @@ antiAlias Sharp Oštro LF (macOS, Linux) LF (macOS, Linux) CRLF (Windows) CRLF (Windows) Text Export Options Opcije izvoza teksta + Chinese Big 5 Kineski Big 5 + Cyrillic Ćirilica + Greek Grčki + Turkish Turski + Baltic Baltički + Central European Srednjoevropski Character Set: Skup znakova: Join the lines of each paragraph Spoji redove svakog pasusa Line Breaks Prelomi reda diff --git a/crates/ui-egui/src/i18n/ca.tsv b/crates/ui-egui/src/i18n/ca.tsv index 59686a4d60..126ae03109 100644 --- a/crates/ui-egui/src/i18n/ca.tsv +++ b/crates/ui-egui/src/i18n/ca.tsv @@ -685,6 +685,12 @@ antiAlias Sharp Nítid Character options Opcions de caràcter Paragraph options Opcions de paràgraf Character Rotation Rotació de caràcters + Chinese Big 5 Xinès Big 5 + Cyrillic Ciríl·lic + Greek Grec + Turkish Turc + Baltic Bàltic + Central European Centreeuropeu Character Set: Joc de caràcters: Character Style Estil de caràcter Character Style Options Opcions de l’estil de caràcter @@ -8258,3 +8264,4 @@ breakText Glyphs Glifs @msg A joint does not close by {gap} pt. Una unió no es tanca per {gap} pt. @msg Two panels pass through each other. Dos panells es travessen l’un a l’altre. @msg composer must be "singleLine", "everyLine", "standardSingleLine" or "standardEveryLine" composer ha de ser "singleLine", "everyLine", "standardSingleLine" o "standardEveryLine" +@msg text.{key} `{v}`: use "{_1}" or "{_2}" text.{key} `{v}`: utilitzeu "{_1}" o "{_2}" diff --git a/crates/ui-egui/src/i18n/cs.tsv b/crates/ui-egui/src/i18n/cs.tsv index 0778326c31..086b9b2f90 100644 --- a/crates/ui-egui/src/i18n/cs.tsv +++ b/crates/ui-egui/src/i18n/cs.tsv @@ -3414,6 +3414,12 @@ antiAlias Sharp Ostré LF (macOS, Linux) LF (macOS, Linux) CRLF (Windows) CRLF (Windows) Text Export Options Možnosti exportu textu + Chinese Big 5 Čínština Big 5 + Cyrillic Cyrilice + Greek Řečtina + Turkish Turečtina + Baltic Baltské jazyky + Central European Středoevropské jazyky Character Set: Znaková sada: Join the lines of each paragraph Spojit řádky každého odstavce Line Breaks Zalomení řádků diff --git a/crates/ui-egui/src/i18n/da.tsv b/crates/ui-egui/src/i18n/da.tsv index 1342c2a6a7..eb7fa71fdc 100644 --- a/crates/ui-egui/src/i18n/da.tsv +++ b/crates/ui-egui/src/i18n/da.tsv @@ -5873,6 +5873,12 @@ antiAlias Sharp Skarp LF (macOS, Linux) LF (macOS, Linux) CRLF (Windows) CRLF (Windows) Text Export Options Indstillinger for teksteksport + Chinese Big 5 Kinesisk Big 5 + Cyrillic Kyrillisk + Greek Græsk + Turkish Tyrkisk + Baltic Baltisk + Central European Centraleuropæisk Character Set: Tegnsæt: Join the lines of each paragraph Forbind linjerne i hvert afsnit Line Breaks Linjeskift diff --git a/crates/ui-egui/src/i18n/de.tsv b/crates/ui-egui/src/i18n/de.tsv index f5b0d11a1a..84c9ca0788 100644 --- a/crates/ui-egui/src/i18n/de.tsv +++ b/crates/ui-egui/src/i18n/de.tsv @@ -402,6 +402,12 @@ font-technology Variable Variabel Character Alignment Zeichenausrichtung Character / Paragraph Zeichen / Absatz Character Rotation Zeichendrehung + Chinese Big 5 Chinesisch Big 5 + Cyrillic Kyrillisch + Greek Griechisch + Turkish Türkisch + Baltic Baltisch + Central European Mitteleuropäisch Character Set: Zeichensatz: Character Style Zeichenformat Character Style Options Zeichenformatoptionen diff --git a/crates/ui-egui/src/i18n/el.tsv b/crates/ui-egui/src/i18n/el.tsv index b75f133cbf..74b1a19dec 100644 --- a/crates/ui-egui/src/i18n/el.tsv +++ b/crates/ui-egui/src/i18n/el.tsv @@ -398,6 +398,12 @@ font-technology Variable Μεταβλητές Character Alignment Στοίχιση χαρακτήρων Character / Paragraph Χαρακτήρας / Παράγραφος Character Rotation Περιστροφή χαρακτήρων + Chinese Big 5 Κινεζικά Big 5 + Cyrillic Κυριλλικά + Greek Ελληνικά + Turkish Τουρκικά + Baltic Βαλτικά + Central European Κεντρικής Ευρώπης Character Set: Σύνολο χαρακτήρων: Character Style Στυλ χαρακτήρων Character Style Options Επιλογές στυλ χαρακτήρων @@ -8232,3 +8238,4 @@ breakText Glyphs Γλύφοι @msg A joint does not close by {gap} pt. Μια ένωση δεν κλείνει κατά {gap} pt. @msg Two panels pass through each other. Δύο πάνελ διαπερνούν το ένα το άλλο. @msg composer must be "singleLine", "everyLine", "standardSingleLine" or "standardEveryLine" το composer πρέπει να είναι "singleLine", "everyLine", "standardSingleLine" ή "standardEveryLine" +@msg text.{key} `{v}`: use "{_1}" or "{_2}" text.{key} `{v}`: χρησιμοποιήστε "{_1}" ή "{_2}" diff --git a/crates/ui-egui/src/i18n/es.tsv b/crates/ui-egui/src/i18n/es.tsv index 832be7b2c7..f0244278fd 100644 --- a/crates/ui-egui/src/i18n/es.tsv +++ b/crates/ui-egui/src/i18n/es.tsv @@ -425,6 +425,12 @@ font-technology Variable Variables Character Alignment Alineación de caracteres Character / Paragraph Carácter / Párrafo Character Rotation Rotación de caracteres + Chinese Big 5 Chino Big 5 + Cyrillic Cirílico + Greek Griego + Turkish Turco + Baltic Báltico + Central European Centroeuropeo Character Set: Juego de caracteres: Character Style Estilo de carácter Character Style Options Opciones de estilo de carácter @@ -8219,3 +8225,4 @@ breakText Glyphs Glifos @msg A joint does not close by {gap} pt. Una unión no cierra por {gap} pt. @msg Two panels pass through each other. Dos paneles se atraviesan entre sí. @msg composer must be "singleLine", "everyLine", "standardSingleLine" or "standardEveryLine" composer debe ser "singleLine", "everyLine", "standardSingleLine" o "standardEveryLine" +@msg text.{key} `{v}`: use "{_1}" or "{_2}" text.{key} `{v}`: use "{_1}" o "{_2}" diff --git a/crates/ui-egui/src/i18n/fi.tsv b/crates/ui-egui/src/i18n/fi.tsv index 049d3b9816..66ac99ffb1 100644 --- a/crates/ui-egui/src/i18n/fi.tsv +++ b/crates/ui-egui/src/i18n/fi.tsv @@ -3361,6 +3361,12 @@ antiAlias Sharp Terävä LF (macOS, Linux) LF (macOS, Linux) CRLF (Windows) CRLF (Windows) Text Export Options Tekstin vientiasetukset + Chinese Big 5 Kiina Big 5 + Cyrillic Kyrillinen + Greek Kreikka + Turkish Turkki + Baltic Balttilainen + Central European Keskieurooppalainen Character Set: Merkistö: Join the lines of each paragraph Yhdistä kunkin kappaleen rivit Line Breaks Rivinvaihdot @@ -8315,3 +8321,4 @@ breakText Glyphs Merkit @msg A joint does not close by {gap} pt. Liitos ei sulkeudu, rako {gap} pt. @msg Two panels pass through each other. Kaksi paneelia kulkee toistensa läpi. @msg composer must be "singleLine", "everyLine", "standardSingleLine" or "standardEveryLine" composer on oltava "singleLine", "everyLine", "standardSingleLine" tai "standardEveryLine" +@msg text.{key} `{v}`: use "{_1}" or "{_2}" text.{key} `{v}`: käytä arvoa "{_1}" tai "{_2}" diff --git a/crates/ui-egui/src/i18n/fr.tsv b/crates/ui-egui/src/i18n/fr.tsv index 6fbee86686..b551e3cbe2 100644 --- a/crates/ui-egui/src/i18n/fr.tsv +++ b/crates/ui-egui/src/i18n/fr.tsv @@ -426,6 +426,12 @@ font-technology Variable Variables Character Alignment Alignement des caractères Character / Paragraph Caractère / Paragraphe Character Rotation Rotation des caractères + Chinese Big 5 Chinois Big 5 + Cyrillic Cyrillique + Greek Grec + Turkish Turc + Baltic Balte + Central European Europe centrale Character Set: Jeu de caractères : Character Style Style de caractère Character Style Options Options de style de caractère diff --git a/crates/ui-egui/src/i18n/gu.tsv b/crates/ui-egui/src/i18n/gu.tsv index e7a21d56bc..9f6ce6b664 100644 --- a/crates/ui-egui/src/i18n/gu.tsv +++ b/crates/ui-egui/src/i18n/gu.tsv @@ -542,6 +542,11 @@ Character Color અક્ષર રંગ Character Direction અક્ષર દિશા Character Rotation અક્ષર પરિભ્રમણ + Chinese Big 5 ચાઇનીઝ Big 5 + Cyrillic સિરિલિક + Greek ગ્રીક + Baltic બાલ્ટિક + Central European મધ્ય યુરોપિયન Character Set: અક્ષર સેટ: Character Style Options અક્ષર સ્ટાઇલ વિકલ્પો Character Styles અક્ષર સ્ટાઇલ diff --git a/crates/ui-egui/src/i18n/hi.tsv b/crates/ui-egui/src/i18n/hi.tsv index a49f570dff..3e144d045a 100644 --- a/crates/ui-egui/src/i18n/hi.tsv +++ b/crates/ui-egui/src/i18n/hi.tsv @@ -801,6 +801,11 @@ antiAlias Sharp शार्प Character Direction कैरेक्टर दिशा Character options कैरेक्टर ऑप्शन्स Character Rotation कैरेक्टर रोटेशन + Chinese Big 5 चीनी Big 5 + Cyrillic सिरिलिक + Greek ग्रीक + Baltic बाल्टिक + Central European मध्य यूरोपीय Character Set: कैरेक्टर सेट: Character Style कैरेक्टर स्टाइल Character Style Options कैरेक्टर स्टाइल ऑप्शन्स diff --git a/crates/ui-egui/src/i18n/hu.tsv b/crates/ui-egui/src/i18n/hu.tsv index f40f9eba2e..977b6062b6 100644 --- a/crates/ui-egui/src/i18n/hu.tsv +++ b/crates/ui-egui/src/i18n/hu.tsv @@ -584,6 +584,12 @@ font-technology Variable Változó Character options Karakterbeállítások Paragraph options Bekezdésbeállítások Character Rotation Karakterforgatás + Chinese Big 5 Kínai Big 5 + Cyrillic Cirill + Greek Görög + Turkish Török + Baltic Balti + Central European Közép-európai Character Set: Karakterkészlet: Character Style Karakterstílus Character Style Options Karakterstílus beállításai diff --git a/crates/ui-egui/src/i18n/id.tsv b/crates/ui-egui/src/i18n/id.tsv index 5ca57f3082..97280c99bc 100644 --- a/crates/ui-egui/src/i18n/id.tsv +++ b/crates/ui-egui/src/i18n/id.tsv @@ -3105,6 +3105,12 @@ antiAlias Sharp Tajam LF (macOS, Linux) LF (macOS, Linux) CRLF (Windows) CRLF (Windows) Text Export Options Opsi ekspor teks + Chinese Big 5 Tionghoa Big 5 + Cyrillic Sirilik + Greek Yunani + Turkish Turki + Baltic Baltik + Central European Eropa Tengah Character Set: Set karakter: Join the lines of each paragraph Gabungkan baris setiap paragraf Line Breaks Pemisah baris diff --git a/crates/ui-egui/src/i18n/it.tsv b/crates/ui-egui/src/i18n/it.tsv index 5d7af01fbe..9465b6aca8 100644 --- a/crates/ui-egui/src/i18n/it.tsv +++ b/crates/ui-egui/src/i18n/it.tsv @@ -427,6 +427,12 @@ font-technology Variable Variabili Character Alignment Allineamento caratteri Character / Paragraph Carattere / Paragrafo Character Rotation Rotazione caratteri + Chinese Big 5 Cinese Big 5 + Cyrillic Cirillico + Greek Greco + Turkish Turco + Baltic Baltico + Central European Centroeuropeo Character Set: Set di caratteri: Character Style Stile carattere Character Style Options Opzioni stile carattere diff --git a/crates/ui-egui/src/i18n/ja.tsv b/crates/ui-egui/src/i18n/ja.tsv index b41654cf55..91c85b9e27 100644 --- a/crates/ui-egui/src/i18n/ja.tsv +++ b/crates/ui-egui/src/i18n/ja.tsv @@ -1265,6 +1265,12 @@ antiAlias Sharp シャープ Channel チャンネル Character / Paragraph 文字/段落 Character Rotation 文字回転 + Chinese Big 5 中国語 Big 5 + Cyrillic キリル文字 + Greek ギリシャ語 + Turkish トルコ語 + Baltic バルト言語 + Central European 中央ヨーロッパ言語 Character Set: 文字セット: Character Style Options 文字スタイルオプション Check Links リンクを確認 diff --git a/crates/ui-egui/src/i18n/ka.tsv b/crates/ui-egui/src/i18n/ka.tsv index 2cdfcab3cf..6fa08c2bd6 100644 --- a/crates/ui-egui/src/i18n/ka.tsv +++ b/crates/ui-egui/src/i18n/ka.tsv @@ -5189,6 +5189,11 @@ antiAlias Sharp მკვეთრი Encoding: კოდირება: Line Endings: სტრიქონის დაბოლოებები: Text Export Options ტექსტის ექსპორტის პარამეტრები + Chinese Big 5 ჩინური Big 5 + Cyrillic კირილიცა + Greek ბერძნული + Baltic ბალტიური + Central European ცენტრალურევროპული Character Set: სიმბოლოთა ნაკრები: Join the lines of each paragraph თითოეული აბზაცის სტრიქონების შეერთება Line Breaks სტრიქონის წყვეტები diff --git a/crates/ui-egui/src/i18n/ko.tsv b/crates/ui-egui/src/i18n/ko.tsv index b4052c3ba3..b0182a55e1 100644 --- a/crates/ui-egui/src/i18n/ko.tsv +++ b/crates/ui-egui/src/i18n/ko.tsv @@ -1255,6 +1255,12 @@ antiAlias Sharp 날카롭게 Channel 채널 Character / Paragraph 문자 / 단락 Character Rotation 문자 회전 + Chinese Big 5 중국어 Big 5 + Cyrillic 키릴 자모 + Greek 그리스어 + Turkish 터키어 + Baltic 발트어 + Central European 중앙 유럽어 Character Set: 문자 집합: Character Style Options 문자 스타일 옵션 Check Links 링크 확인 diff --git a/crates/ui-egui/src/i18n/nl.tsv b/crates/ui-egui/src/i18n/nl.tsv index 12bf3a1837..ab8455de30 100644 --- a/crates/ui-egui/src/i18n/nl.tsv +++ b/crates/ui-egui/src/i18n/nl.tsv @@ -3748,6 +3748,12 @@ antiAlias Sharp Scherp LF (macOS, Linux) LF (macOS, Linux) CRLF (Windows) CRLF (Windows) Text Export Options Opties tekstexport + Chinese Big 5 Chinees Big 5 + Cyrillic Cyrillisch + Greek Grieks + Turkish Turks + Baltic Baltisch + Central European Centraal-Europees Character Set: Tekenset: Join the lines of each paragraph De regels van elke alinea samenvoegen Line Breaks Regeleinden diff --git a/crates/ui-egui/src/i18n/pl.tsv b/crates/ui-egui/src/i18n/pl.tsv index f6ce46d0cb..9713139beb 100644 --- a/crates/ui-egui/src/i18n/pl.tsv +++ b/crates/ui-egui/src/i18n/pl.tsv @@ -3295,6 +3295,12 @@ antiAlias Sharp Ostre Encoding: Kodowanie: Line Endings: Znaki końca wiersza: Text Export Options Opcje eksportu tekstu + Chinese Big 5 Chiński Big 5 + Cyrillic Cyrylica + Greek Grecki + Turkish Turecki + Baltic Bałtycki + Central European Środkowoeuropejski Character Set: Zestaw znaków: Join the lines of each paragraph Połącz wiersze każdego akapitu Line Breaks Podziały wierszy diff --git a/crates/ui-egui/src/i18n/pt-br.tsv b/crates/ui-egui/src/i18n/pt-br.tsv index 0cf237ccb2..847946c680 100644 --- a/crates/ui-egui/src/i18n/pt-br.tsv +++ b/crates/ui-egui/src/i18n/pt-br.tsv @@ -394,6 +394,12 @@ antiAlias Sharp Nítido Character Alignment Alinhamento de Caracteres Character / Paragraph Caractere / Parágrafo Character Rotation Rotação de Caractere + Chinese Big 5 Chinês Big 5 + Cyrillic Cirílico + Greek Grego + Turkish Turco + Baltic Báltico + Central European Europa Central Character Set: Conjunto de Caracteres: Character Style Options Opções de Estilo de Caractere Character Styles Estilos de Caractere diff --git a/crates/ui-egui/src/i18n/ru.tsv b/crates/ui-egui/src/i18n/ru.tsv index 2f2d244470..4ff987ee2b 100644 --- a/crates/ui-egui/src/i18n/ru.tsv +++ b/crates/ui-egui/src/i18n/ru.tsv @@ -423,6 +423,12 @@ font-technology Variable Вариативные Character Alignment Выравнивание символов Character / Paragraph Символ / Абзац Character Rotation Поворот символов + Chinese Big 5 Китайская Big 5 + Cyrillic Кириллица + Greek Греческая + Turkish Турецкая + Baltic Балтийская + Central European Центральноевропейская Character Set: Набор символов: Character Style Стиль символа Character Style Options Параметры стиля символа diff --git a/crates/ui-egui/src/i18n/sv.tsv b/crates/ui-egui/src/i18n/sv.tsv index 9c09b26fc8..ca3e9260cb 100644 --- a/crates/ui-egui/src/i18n/sv.tsv +++ b/crates/ui-egui/src/i18n/sv.tsv @@ -415,6 +415,11 @@ font-technology Variable Variabla Character Alignment Teckenjustering Character / Paragraph Tecken / stycke Character Rotation Teckenrotation + Chinese Big 5 Kinesisk Big 5 + Cyrillic Kyrillisk + Greek Grekisk + Baltic Baltisk + Central European Centraleuropeisk Character Set: Teckenuppsättning: Character Style Teckenstil Character Style Options Alternativ för teckenstil diff --git a/crates/ui-egui/src/i18n/tests.rs b/crates/ui-egui/src/i18n/tests.rs index 5d40f121da..0e8f243b49 100644 --- a/crates/ui-egui/src/i18n/tests.rs +++ b/crates/ui-egui/src/i18n/tests.rs @@ -617,6 +617,22 @@ fn monitor_profile_labels_are_in_complete_catalogs() { } } +/// Text Import Options' Character Set choices that name a script or a region (Shift JIS, GB2312 +/// and GB18030 are names of their own), and the error for an unknown character set. +#[test] +fn text_import_character_sets_are_in_complete_catalogs() { + for lang in LANGUAGES.iter().filter(|lang| lang.complete_menus) { + for label in ["Chinese Big 5", "Cyrillic", "Greek", "Turkish", "Baltic", "Central European"] { + assert!(lang.catalog().plain(label).is_some(), "{} is missing {label}", lang.code); + } + } + let unknown = r#"text.characterSet `x`: use "unicode", "ansi", "shiftJis", "gb2312", "chineseBig5", "cyrillic", "gb18030", "greek", "turkish", "baltic" or "centralEuropean""#; + for code in COMPLETE_MESSAGES { + let lang = Lang::from_code(code).expect("registered"); + assert_ne!(message(lang, unknown), unknown, "{code}"); + } +} + /// The Control bar panel menu's items, its groups of options and the edges, and the label of the /// command that turns a group on or off. #[test] diff --git a/crates/ui-egui/src/i18n/tr.tsv b/crates/ui-egui/src/i18n/tr.tsv index a4822e4702..9e67a2571a 100644 --- a/crates/ui-egui/src/i18n/tr.tsv +++ b/crates/ui-egui/src/i18n/tr.tsv @@ -408,6 +408,12 @@ font-technology Variable Değişken Character Alignment Karakter Hizalaması Character / Paragraph Karakter / Paragraf Character Rotation Karakter Rotasyonu + Chinese Big 5 Çince Big 5 + Cyrillic Kiril + Greek Yunanca + Turkish Türkçe + Baltic Baltık + Central European Orta Avrupa Character Set: Karakter Seti: Character Style Karakter Stili Character Style Options Karakter Stili Seçenekleri diff --git a/crates/ui-egui/src/i18n/uk.tsv b/crates/ui-egui/src/i18n/uk.tsv index f7a42837c5..60bc172495 100644 --- a/crates/ui-egui/src/i18n/uk.tsv +++ b/crates/ui-egui/src/i18n/uk.tsv @@ -421,6 +421,12 @@ font-technology Variable Варіативні Character Alignment Вирівнювання символів Character / Paragraph Символ / Абзац Character Rotation Поворот символів + Chinese Big 5 Китайська Big 5 + Cyrillic Кирилиця + Greek Грецька + Turkish Турецька + Baltic Балтійська + Central European Центральноєвропейська Character Set: Набір символів: Character Style Стиль символу Character Style Options Параметри стилю символу diff --git a/crates/ui-egui/src/i18n/vi.tsv b/crates/ui-egui/src/i18n/vi.tsv index bf4b2c7e55..293551aeb8 100644 --- a/crates/ui-egui/src/i18n/vi.tsv +++ b/crates/ui-egui/src/i18n/vi.tsv @@ -500,6 +500,12 @@ font-technology Variable Biến thiên Character options Tùy chọn ký tự Paragraph options Tùy chọn đoạn văn Character Rotation Xoay ký tự + Chinese Big 5 Tiếng Trung Big 5 + Cyrillic Chữ Kirin + Greek Tiếng Hy Lạp + Turkish Tiếng Thổ Nhĩ Kỳ + Baltic Vùng Baltic + Central European Trung Âu Character Set: Bộ ký tự: Character Style Kiểu ký tự Character Style Options Tùy chọn kiểu ký tự @@ -7895,3 +7901,4 @@ breakText Glyphs Glyph @msg {width} × {height} pixels is too large for Targa (at most 65535 pixels a side): lower the resolution {width} × {height} điểm ảnh quá lớn cho Targa (tối đa 65535 điểm ảnh mỗi cạnh): hãy giảm độ phân giải @msg {w} × {h} pixels is too large for PSD (at most {MAX_SIDE} pixels a side): lower the resolution {w} × {h} điểm ảnh quá lớn cho PSD (tối đa {MAX_SIDE} điểm ảnh mỗi cạnh): hãy giảm độ phân giải @msg composer must be "singleLine", "everyLine", "standardSingleLine" or "standardEveryLine" composer phải là "singleLine", "everyLine", "standardSingleLine" hoặc "standardEveryLine" +@msg text.{key} `{v}`: use "{_1}" or "{_2}" text.{key} `{v}`: hãy dùng "{_1}" hoặc "{_2}" diff --git a/crates/ui-egui/src/i18n/zh-hans.tsv b/crates/ui-egui/src/i18n/zh-hans.tsv index 4eb4a9a817..6ac837bf96 100644 --- a/crates/ui-egui/src/i18n/zh-hans.tsv +++ b/crates/ui-egui/src/i18n/zh-hans.tsv @@ -374,6 +374,12 @@ antiAlias Sharp 锐利 Character Alignment 字符对齐 Character / Paragraph 字符/段落 Character Rotation 字符旋转 + Chinese Big 5 繁体中文 Big 5 + Cyrillic 西里尔文 + Greek 希腊文 + Turkish 土耳其文 + Baltic 波罗的海文 + Central European 中欧文 Character Set: 字符集: Character Style 字符样式 Character Style Options 字符样式选项 diff --git a/crates/ui-egui/src/i18n/zh-hant.tsv b/crates/ui-egui/src/i18n/zh-hant.tsv index 37907b4301..fa65e7cd45 100644 --- a/crates/ui-egui/src/i18n/zh-hant.tsv +++ b/crates/ui-egui/src/i18n/zh-hant.tsv @@ -389,6 +389,12 @@ font-technology Variable 可變 Character / Paragraph 字元/段落 Character Alignment 字元對齊 Character Rotation 字元旋轉 + Chinese Big 5 繁體中文 Big 5 + Cyrillic 西里爾文 + Greek 希臘文 + Turkish 土耳其文 + Baltic 波羅的海文 + Central European 中歐文 Character Set: 字元集: Character Style 字元樣式 Character Style Options 字元樣式選項 diff --git a/docs/control-protocol.md b/docs/control-protocol.md index f78d8b375b..7eba21351a 100644 --- a/docs/control-protocol.md +++ b/docs/control-protocol.md @@ -738,7 +738,7 @@ folder, then opens the dialog searching it. The web shows a status line instead disabled there. Text Import Options: placing a `.txt` file through the app (`file.place` with a file and no `text` options, the Place -dialog, a drop) opens the `textImport` dialog (fields `platform`: `windows`/`mac`, `characterSet`: `unicode`/`ansi`, +dialog, a drop) opens the `textImport` dialog (fields `platform`: `windows`/`mac`, `characterSet`: a `text.characterSet` value, `removeLineReturns`, `removeParagraphReturns`, `replaceSpaces`, `spaces`). `ui.dialog.confirm` places the file as area type with those options (`file.place {text: {…}}`). diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 48250c14e6..e1f2b735f8 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -336,7 +336,7 @@ Gradient meshes in the editing data keep each point's own color where a patch's | Pixar PXR | open, place (legacy) | yes (any set of the R, G, B and A channels in 8 or 12 bits, dumped or run-length encoded, in tiles; R alone is gray; alpha matted to black or unassociated) | no | recognized by its magic number; 12-bit samples are clamped to 0.0 to 1.0; null tiles are transparent with A and black without; the offsets aren't read, and the image takes the default resolution; a file cut short, with header values or packets the format doesn't define, or past the pixel allowance is refused with a message; tested on synthetic files only; tests in `crates/engine/src/cmd/fileio/tests_pxrread.rs` and `crates/engine/tests/import_fuzz.rs` | | FreeHand `.fh*` | open (legacy) | no | no | | | CorelDRAW `.cdr` | open | no | no | | -| Text `.txt` | place, export | place (Text Import Options) | yes | | +| Text `.txt` | place, export | place (Text Import Options) | yes | place reads Unicode and ANSI, and Shift JIS, GB2312, Chinese Big 5, Cyrillic, GB18030, Greek, Turkish, Baltic and Central European through the WHATWG Encoding Standard's decoders (VectorCraft's own mapping); `tests/textimport.rs`, `place/text.rs`; fuzzed (`text_import_fuzz.rs`) | | RTF `.rtf` | place | place (area type with fonts, size, bold, italic, underline, strikethrough, caps, superscript and subscript, color, alignment, indents and paragraph spacing) | no | `tests/textimport.rs`, `place/rtf.rs`; fuzzed (`text_import_fuzz.rs`); `\'hh` bytes read as Windows-1252 or Mac Roman only | | Word `.docx` | place | place (area type with the document's styles and its fonts, size, bold, italic, underline, strikethrough, caps, superscript and subscript, color, alignment, indents and paragraph spacing) | no | `tests/textimport.rs`, `place/docx.rs`; fuzzed (`text_import_fuzz.rs`); list numbers, pictures and text boxes are left out | | Word `.doc` | place | place (area type with the same formatting as `.docx`, from the style sheet's styles and direct formatting; Word 97 and later) | no | `tests/textimport.rs`, `place/doc.rs`; fuzzed (`text_import_fuzz.rs`); pictures, headers, footers, notes, comments and text boxes are left out with a warning; encrypted and fast-saved documents are refused | diff --git a/docs/mcp.md b/docs/mcp.md index 2d2489d095..11c3165efa 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -8509,10 +8509,15 @@ the document; so does `object.expand`. ``` Placing a text file (`.txt`) sets it as area type: `file.place {path | name+dataBase64, text?: {characterSet?: -"unicode" | "ansi", platform?: "windows" | "mac", removeLineReturns?, removeParagraphReturns?, replaceSpaces?: n}}` +"unicode" | "ansi" | "shiftJis" | "gb2312" | "chineseBig5" | "cyrillic" | "gb18030" | "greek" | "turkish" | "baltic" | +"centralEuropean", platform?: "windows" | "mac", removeLineReturns?, removeParagraphReturns?, replaceSpaces?: n}}` (Text Import Options). Unicode reads UTF-8, or UTF-16 with a byte-order mark; ANSI reads Windows-1252 (Mac Roman on -`mac`). Unicode reads a file without a byte-order mark as UTF-16 by the rule `dataset.import` uses: at least half of -its two-byte units hold a NUL byte, in the byte order that reads a line break, or for a file with one line, big-endian +`mac`). The other sets use the WHATWG Encoding Standard's decoders: `shiftJis` Shift_JIS, `gb2312` GBK, `chineseBig5` +Big5, `cyrillic` windows-1251 (x-mac-cyrillic on `mac`), `gb18030` gb18030, and `greek`, `turkish`, `baltic` and +`centralEuropean` windows-1253, 1254, 1257 and 1250 on both platforms. They read a byte-order mark as ordinary bytes +and a malformed sequence as U+FFFD. An unknown set, or an option of the wrong type (`characterSet: 3`, +`removeLineReturns: "yes"`), is an error, and nothing is placed. Unicode reads a file without a byte-order mark as +UTF-16 by the rule `dataset.import` uses: at least half of its two-byte units hold a NUL byte, in the byte order that reads a line break, or for a file with one line, big-endian when more of its units start with a NUL byte than end with one, and little-endian otherwise. Any other file is read as UTF-8, else as the platform's 8-bit set: one with fewer NUL bytes, one with an odd length, one where both byte orders read a line break, and one that isn't valid UTF-16 in that order or holds a NUL character. `removeLineReturns` joins diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index 4f14bb6d08..1a4de32fa2 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -2107,6 +2107,7 @@ Moved from the ROADMAP's "Shipped so far" on 2026-10-10. Update it when a featur - The place cursor keeps an encrypted PDF, `.ai` or `.ait` file, at a letter page's size (VectorCraft's own size) and without a thumbnail, and a click or drag on it asks for its password, with the Place dialog's Show Import Options on or off; such a file was left out of the cursor before (`crates/engine/src/tests/place.rs`, `crates/ui-egui/src/tests/place.rs`). - File › Place with Bounding Box (`file.place {crop: "bounding"}`) measures the art that shows, with its strokes and effects, as an export does: hidden objects, guides and template layers do not count, a group whose only part on the page or artboard is hidden is not placed, and symbol instances are measured by their art, for `at`, `rect`, the place cursor and `file.place.info` (`crates/engine/src/tests/place.rs`). - The Place dialog has a Show Import Options checkbox after Link, Template and Replace ([Place multiple files in a document](https://helpx.adobe.com/illustrator/desktop/add-and-import-files/import-other-file-types/place-multiple-files-in-a-document.html)). On (VectorCraft's default, saved with the preferences), each PDF, DXF or text file asks for its import options before it is placed, also in the place cursor; off (`file.place {showImportOptions: false}`), it is placed with the options last confirmed for its format, and only an encrypted PDF asks, for its password. Text Import Options now starts from the options confirmed last (`crates/ui-egui/src/tests/place.rs`). + - Text Import Options' Character Set offers Shift JIS, GB2312, Chinese Big 5, Cyrillic, GB18030, Greek, Turkish, Baltic and Central European beside Unicode and ANSI, the plain-text encodings [the help page](https://helpx.adobe.com/illustrator/desktop/add-and-import-files/import-other-file-types/import-and-export-text.html) lists for placed text. `file.place {text: {characterSet}}` decodes each with its WHATWG Encoding Standard decoder. The mapping is VectorCraft's own: GB2312 reads as GBK, Cyrillic as windows-1251 (x-mac-cyrillic with `platform: "mac"`), and Greek, Turkish, Baltic and Central European as windows-1253, 1254, 1257 and 1250 on both platforms. A byte-order mark is read as ordinary bytes, a malformed sequence as U+FFFD, and the last set confirmed is remembered (`crates/engine/src/tests/textimport.rs`, `crates/ui-egui/src/dialogs/text_import.rs`). - A Shift-click with the loaded place cursor opens the current file's import options (text, DXF, PDF page or embedded PSD) and OK places it at the click, with Show Import Options on or off and for a cursor loaded by `file.place.queue` ([Place multiple files](https://web.archive.org/web/20210601162814/https://helpx.adobe.com/pdf/illustrator_reference.pdf)). VectorCraft's own rules: a file without such a dialog places as a click does, Cancel leaves the file unplaced, a Shift-drag places at the dragged size, and headless the Shift-click places as a click (`crates/tools/src/place.rs`, `crates/engine/src/tests/place.rs`, `crates/ui-egui/src/tests/place.rs`). - Separations Preview shows placed and embedded raster images on the visible plates, as it shows vector art (#4153): an RGB image's pixels are separated into process inks with the proof setup's intent, a CMYK image shows its own inks, a hidden plate's ink leaves the image, and with one plate visible the image shows that plate's ink as a grey. Raster images carry no spot ink, so a spot plate shown alone shows them as paper; a colorized image shows its color's plates. Emoji and other bitmap glyphs follow the same rule. The plate-filtered pixels and their reduced-size copies are kept apart from the image's own pixels, for each set of visible plates, proof setup and color settings, and with the preview off nothing changes. Evidence: `tests::colorize::separations_preview_shows_an_rgb_images_process_inks`, `separations_preview_shows_a_cmyk_images_own_inks`, `separations_preview_keeps_an_images_pixels_per_plate_set`. Print preview fidelity remains the open gap, so scores and hours are unchanged. - Desktop print transport uses `craft-print` for CUPS and modern Windows. Sheet composition, copies, separations and colour management remain in VectorCraft. The Print dialog's Media list also carries the Japanese papers — JIS B4 257 × 364 mm, JIS B5 182 × 257 mm and JIS B6 128 × 182 mm beside the ISO B4/B5 (250 × 353 and 176 × 250 mm, kept so settings and presets saved before them keep their paper), and はがき 100 × 148 mm, 往復はがき 148 × 200 mm, 封筒 長形3号 120 × 235 mm, 長形4号 90 × 225 mm, 角形2号 240 × 332 mm, 角形3号 216 × 277 mm and 四六判 127 × 188 mm; the JSON id or the dialog label names a paper, any case and spacing (`B5 (JIS)` or `b5jis`) (`crates/pdf/src/print/settings.rs`, `crates/pdf/src/tests/print.rs`). The print-preview and physical-device fidelity gaps remain, so print scores and hours are unchanged.