A CSV file that holds NUL bytes and has no byte-order mark was refused.
It is now read as UTF-16 when at least half of its two-byte units hold
a NUL byte, in the byte order that reads a line break. A file with fewer
NUL bytes (UTF-8 or Windows-1252 text with stray NUL bytes, or UTF-16
text made mostly of characters above U+00FF) is still refused, and so is
a file that reads a line break in both byte orders, has an odd length,
isn't valid UTF-16 in that order or holds a NUL character. Refused files
get the same message as before.
dataset.import and variable.loadLibrary added an undo step, cleared Redo
and marked a saved document modified even when the variables and data
sets came out as they were, such as the same CSV or library loaded
again. Both now build the new variables on a copy and record a step only
when it differs. variable.loadLibrary's params text and docs/mcp.md say
so.
dataset.import read two header cells that named one variable (City
and City, or Shown and #Shown) into the same column, so each row kept
only the last non-blank cell and the other value was dropped without a
word. The import is now refused with the two column numbers, and the
message has a row in every complete-message catalog.
dataset.import replaced the reason for a refused cell with "value for
`X` must match its variable kind", so a visibility value such as
maybe and a graph value over 262,144 bytes gave the same message. The
error now keeps the line and gives the reason, and both messages have
a row in every complete-message catalog.
dataset.export checked only the 1 MiB size, so a document with more than
1000 data sets or 256 variables exported a CSV that dataset.import then
refused. Export now refuses such a document with a message naming the
limit, and the messages have rows in every complete-message catalog.
Found while testing crates/engine/src/cmd/variablecmds/csv.rs:146 (csv_escape) and :158-163 (cell_to_value)
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dotCooCoo