mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-12 07:46:08 +00:00
Dylan M. Taylor
23db9eee7d
craft-rtf and craft-dxf each carried tables for DOS 437, 737, 850, 852 and 857 (and craft-rtf one for 866). They are now one set in craft-codepage, which takes craft-rtf's codepage module (Windows and Mac tables, OEM glyphs, font charsets) and craft-dxf's DOS tables. craft_rtf::codepage re-exports it unchanged; craft-dxf keeps its $DWGCODEPAGE names and Decoder and looks the DOS tables up there. Every byte of every code page either crate decodes is unchanged: the public decoders of both crates were dumped over all 256 bytes of each page (and the East Asian pairs) before and after, and the dumps are identical. craft-mime now reads the DOS pages by their IANA names (IBM437, cp850, ibm-852, 857...), which it read as UTF-8 before.
craft-codepage
Legacy 8-bit code pages, one table each, for every reader of old text in the Craft libraries:
- the single-byte Windows code pages (874, 1250–1258) and Mac OS Roman (10000), byte by byte
(
windows_byte,cp1252); - the DOS (IBM PC, OEM) code pages 437, 720, 737, 775, 850, 852, 855, 857, 860, 861, 863, 864,
865, 866 and 869 (
dos_upper, thedostables;oem_bytewith the glyphs DOS shows for control codes); - document text in any of those (
code_page_byte), Windows' symbol code page, the RTF/Word font charsets (charset_code_page), and the names of the East Asian multibyte code pages forencoding_rs(multibyte_encoding_name).
Everything is keyed by Windows code page number. Readers that name code pages another way map
their names onto these: craft-dxf ($DWGCODEPAGE names such as DOS850), craft-mime (charset
labels such as IBM850), craft-rtf (\ansicpg, \fcharset).
use craft_codepage::{code_page_byte, dos_upper, windows_byte};
assert_eq!(windows_byte(1251, 0xC4), 'Д');
assert_eq!(code_page_byte(850, 0x9B), 'ø');
assert_eq!(dos_upper(437).map(|t| t[0x1B]), Some('¢'));
The Windows and Mac tables match the WHATWG Encoding Standard's indexes, except that a byte a
Windows code page other than 1252 leaves unassigned is U+FFFD here where the standard gives the
C1 control of the same number. A test checks that against encoding_rs.
No dependencies.