From 86deb546ada5ffe4555dce53967904113763f6a7 Mon Sep 17 00:00:00 2001 From: dotCooCoo Date: Tue, 3 Nov 2026 16:38:29 +0700 Subject: [PATCH] Variables: Import Data Sets reads every column as a variable and replaces the data sets Squashed from #5151 onto main. --- crates/engine/src/cmd/variablecmds.rs | 14 +- crates/engine/src/cmd/variablecmds/csv.rs | 127 +++---- crates/engine/src/tests/variables.rs | 384 ++++++++++++++-------- crates/ui-egui/src/i18n/tests.rs | 12 +- crates/ui-egui/src/panels/variables.rs | 17 +- crates/ui-egui/src/tests/picks.rs | 10 +- docs/mcp.md | 11 +- docs/target-app-parity.md | 2 +- 8 files changed, 335 insertions(+), 242 deletions(-) diff --git a/crates/engine/src/cmd/variablecmds.rs b/crates/engine/src/cmd/variablecmds.rs index 76d39ea56..4806531cd 100644 --- a/crates/engine/src/cmd/variablecmds.rs +++ b/crates/engine/src/cmd/variablecmds.rs @@ -4,11 +4,11 @@ //! variable, anything to a `visibility` variable, a linked image to an `image` //! variable, or a graph root to a `graph` variable (CSV data), put a row of values in //! each dataset, and `dataset.select` swaps the whole document to that row in one undo -//! step. `dataset.import` / `dataset.export` move those rows as CSV (header: `name` -//! then variable names, which export writes with `#` before a visibility variable and `@` -//! before an image variable); import also reads tab-separated files, files without the -//! `name` column, and header cells that define new variables (`#` for visibility, `@` for -//! image, no prefix for text). `variable.loadLibrary` replaces the variables and data sets with the +//! step. `dataset.import` / `dataset.export` move those rows as CSV (header: one +//! variable name per column, which export writes with `#` before a visibility variable and `@` +//! before an image variable); import replaces the datasets, names the rows Data Set 1, 2…, +//! also reads tab-separated files, and reads header cells that define new variables (`#` for +//! visibility, `@` for image, no prefix for text). `variable.loadLibrary` replaces the variables and data sets with the //! text variables and data sets of a variable library, the XML file of the reference app's Save //! Variable Library, and `variable.saveLibrary` writes the text variables and data sets as one. //! Agents reach every command through `run_command`, like all engine commands. @@ -838,7 +838,7 @@ pub fn specs() -> Vec { "Import Data Sets…", ["Window", "Variables"], None, - "{path? | data? | dataBase64? | csv?} load datasets from CSV, comma- or tab-separated, in UTF-16 with a byte-order mark, UTF-8 or else Windows-1252 (header: name, then variable names; one row per dataset; without the name column every header is a variable and the rows take the first free names Data Set 1, 2…). A header that names no variable defines one, unbound: `#Name` defines a visibility variable, `@Name` defines an image variable, and a header without a prefix defines a text variable. A prefixed header whose name is a variable of another kind, and a `%` header that isn't a variable's exact name (graph data files), are refused, and so are two headers that name one variable and a header that would define a variable while the variables are locked. Rows whose name already exists are replaced → {imported, updated, names, defined}", + "{path? | data? | dataBase64? | csv?} replace every dataset with the rows of a CSV, comma- or tab-separated, in UTF-16 with a byte-order mark, UTF-8 or else Windows-1252 (header: one variable name per column, whatever its case; one row per dataset, named Data Set 1, 2… in file order). The document's variables and their bindings stay. A header that names no variable defines one, unbound: `#Name` defines a visibility variable, `@Name` defines an image variable, and a header without a prefix defines a text variable. A prefixed header whose name is a variable of another kind, and a `%` header that isn't a variable's exact name (graph data files), are refused, and so are two headers that name one variable and a header that would define a variable while the variables are locked. An import that leaves the variables and data sets as they were adds no undo step → {imported, names, defined, deletedDataSets}", has_doc, dataset_import ), @@ -847,7 +847,7 @@ pub fn specs() -> Vec { "Export Data Sets…", ["Window", "Variables"], None, - "{path?} write every dataset as CSV (header: name, then one column per variable: `#Name` for a visibility variable, `@Name` for an image variable, and the name alone for a text or graph variable, as dataset.import reads them). Refused when the document has more than 1000 data sets or 256 variables, the most dataset.import reads, or when a variable's header is another variable's name (a visibility variable `Show` next to a text variable `#Show`) → {path, datasets, warning?}; without path → {csv, datasets, warning?}. `warning` names the variables whose header or values make a document without them refuse the file (a % header, a # header whose values aren't true or false); otherwise it names the variables such a document doesn't read back with their kind (text variables whose names start with # or @, graph variables)", + "{path?} write every dataset as CSV (header: one column per variable and no dataset names: `#Name` for a visibility variable, `@Name` for an image variable, and the name alone for a text or graph variable, as dataset.import reads them). Refused when the document has more than 1000 data sets or 256 variables, the most dataset.import reads, or when a variable's header is another variable's name (a visibility variable `Show` next to a text variable `#Show`) → {path, datasets, warning?}; without path → {csv, datasets, warning?}. `warning` names the variables whose header or values make a document without them refuse the file (a % header, a # header whose values aren't true or false); otherwise it names the variables such a document doesn't read back with their kind (text variables whose names start with # or @, graph variables)", has_doc, dataset_export ), diff --git a/crates/engine/src/cmd/variablecmds/csv.rs b/crates/engine/src/cmd/variablecmds/csv.rs index 5a4ba5100..31e46787f 100644 --- a/crates/engine/src/cmd/variablecmds/csv.rs +++ b/crates/engine/src/cmd/variablecmds/csv.rs @@ -102,7 +102,8 @@ fn tab_separated(text: &str) -> bool { /// by commas, or by tabs when [`tab_separated`] says so. `"` quotes a cell (doubled inside), /// which may then hold separators and line breaks, as [`csv_escape`] writes them. An unquoted /// cell is trimmed. A quoted cell keeps what its quotes hold as written, spaces included, and -/// drops white space after the closing quote at the end of the cell. Blank lines are skipped. +/// drops white space after the closing quote at the end of the cell. A line whose cells are all +/// blank and unquoted is skipped; a quoted empty cell keeps its line as a row. fn parse_datasets_csv(text: &str, cmd: &str) -> Result> { let text = text.strip_prefix('\u{feff}').unwrap_or(text); let sep = if tab_separated(text) { '\t' } else { ',' }; @@ -132,7 +133,10 @@ fn parse_datasets_csv(text: &str, cmd: &str) -> Result> { Some('\n') | None if !in_quotes => { quoted.push(closed.is_some()); row.push(end_csv_cell(&mut cur, &mut closed)); - if row.iter().any(|c| !c.trim().is_empty()) { + if row.len() > MAX_CSV_COLS { + return Err(bad(cmd, format!("line {line}: more than {MAX_CSV_COLS} columns"))); + } + if row.iter().any(|c| !c.trim().is_empty()) || quoted.contains(&true) { rows.push((start, std::mem::take(&mut row), std::mem::take(&mut quoted))); // +1 for the header. if rows.len() > MAX_CSV_ROWS + 1 { @@ -174,8 +178,10 @@ fn end_csv_cell(cur: &mut String, closed: &mut Option) -> String { } fn csv_escape(s: &str) -> String { - // An empty string is quoted so that it reads back as an empty value, not as a missing one. - if s.is_empty() || s.contains([',', '"', '\n', '\r']) || s != s.trim() { format!("\"{}\"", s.replace('"', "\"\"")) } else { s.to_string() } + // An empty string is quoted so that it reads back as an empty value, not as a missing one. A + // tab is quoted so that a header of one variable whose name holds a tab isn't read as + // tab-separated. + if s.is_empty() || s.contains([',', '"', '\n', '\r', '\t']) || s != s.trim() { format!("\"{}\"", s.replace('"', "\"\"")) } else { s.to_string() } } /// A visibility cell: true/false, 1/0, yes/no (ASCII, case-insensitive). An error names the @@ -240,8 +246,7 @@ fn kinds_a_new_document_loses(vars: &Variables) -> (Vec<&str>, Vec<&str>) { for (i, v) in vars.variables.iter().enumerate() { let header = export_header(v); let cell = header.trim(); - // Column 1 holds the dataset names. - let read = if cell.is_empty() { None } else { header_variable(cell, i + 2, &none, &mut defined, C).ok() }; + let read = if cell.is_empty() { None } else { header_variable(cell, i + 1, &none, &mut defined, C).ok() }; // The import refuses a header that names a variable an earlier column named. let Some((name, kind)) = read.filter(|(name, _)| seen.insert(name.clone())) else { refused.push(v.name.as_str()); @@ -260,30 +265,36 @@ fn kinds_a_new_document_loses(vars: &Variables) -> (Vec<&str>, Vec<&str>) { (refused, other) } -/// Build the CSV of every dataset (header `name` then each variable's [`export_header`] in -/// definition order). Refused when a variable's header is another variable's name, which the -/// import reads as that variable (a visibility variable `Show` next to a text variable `#Show`). +/// Build the CSV of every dataset: a header of each variable's [`export_header`] in definition +/// order, then one row per dataset. The dataset names are not written. Refused when a variable's +/// header is another variable's name, which the import reads as that variable (a visibility +/// variable `Show` next to a text variable `#Show`). A dataset with no value for any variable would +/// be a blank line, which the import skips, so its row gets a quoted empty cell: in the first column +/// whose variable isn't a text variable, where it reads back as no value, or else in the first +/// column, where it reads back as an empty text. fn datasets_to_csv(vars: &Variables, cmd: &str) -> Result { let names: BTreeSet<&str> = vars.variables.iter().map(|v| v.name.as_str()).collect(); - let mut out = String::from("name"); + let mut header = Vec::with_capacity(vars.variables.len()); for v in &vars.variables { - let header = export_header(v); - if header != v.name && names.contains(header.as_str()) { - let (name, other) = (&v.name, &header); - return Err(bad(cmd, format!("the header `{header}` for `{name}` would read back as the variable `{other}`: rename one of them"))); + let cell = export_header(v); + if cell != v.name && names.contains(cell.as_str()) { + let (name, other) = (&v.name, &cell); + return Err(bad(cmd, format!("the header `{cell}` for `{name}` would read back as the variable `{other}`: rename one of them"))); } - out.push(','); - out.push_str(&csv_escape(&header)); + header.push(csv_escape(&cell)); } + let mut out = header.join(","); out.push('\n'); + let keep = vars.variables.iter().position(|v| v.kind != VariableKind::Text).unwrap_or(0); for ds in &vars.datasets { - out.push_str(&csv_escape(&ds.name)); - for v in &vars.variables { - out.push(','); - if let Some(val) = ds.values.get(&v.name) { - out.push_str(&csv_escape(&value_to_cell(val))); - } + let mut row: Vec = + vars.variables.iter().map(|v| ds.values.get(&v.name).map(|val| csv_escape(&value_to_cell(val))).unwrap_or_default()).collect(); + if row.iter().all(String::is_empty) + && let Some(cell) = row.get_mut(keep) + { + *cell = csv_escape(""); } + out.push_str(&row.join(",")); out.push('\n'); } Ok(out) @@ -329,12 +340,11 @@ fn header_variable( } } -/// Import datasets from CSV. A first header cell `name` makes the first column the dataset -/// names; the other headers name variables, as [`header_variable`] reads them, and the -/// variables the document doesn't define yet are defined, unbound. Without the `name` column, -/// every header names a variable and the rows take the first free names "Data Set 1", "Data -/// Set 2" and so on. Two headers that name one variable are refused. Existing rows of the same -/// name are replaced; new names are appended. One undo step. +/// Import datasets from CSV. Every header cell names a variable, as [`header_variable`] reads it, +/// and the variables the document doesn't define yet are defined, unbound. Two headers that name +/// one variable are refused. The rows replace every dataset of the document and are named "Data +/// Set 1", "Data Set 2" and so on, in file order. The document's variables and their bindings +/// stay. One undo step, and none when the variables and datasets stay as they were. pub(super) fn dataset_import(s: &mut Session, p: &Value) -> Result { const C: &str = "dataset.import"; let (bytes, given_as_text) = datasets_csv_bytes(p, C)?; @@ -344,16 +354,10 @@ pub(super) fn dataset_import(s: &mut Session, p: &Value) -> Result { let Some((_, header, _)) = rows.first() else { return Err(bad(C, "CSV is empty")); }; - let name_column = header.first().is_some_and(|h| h.trim().eq_ignore_ascii_case("name")); - let (kinds, existing_names) = { - let st = s.doc()?; - let kinds: BTreeMap = st.doc.variables.variables.iter().map(|v| (v.name.clone(), v.kind)).collect(); - let existing: BTreeSet = st.doc.variables.datasets.iter().map(|d| d.name.clone()).collect(); - (kinds, existing) - }; + let kinds: BTreeMap = s.doc()?.doc.variables.variables.iter().map(|v| (v.name.clone(), v.kind)).collect(); let mut columns: Vec<(usize, String, VariableKind)> = Vec::new(); let mut defined: Vec = Vec::new(); - for (i, h) in header.iter().enumerate().skip(usize::from(name_column)) { + for (i, h) in header.iter().enumerate() { let cell = h.trim(); if cell.is_empty() { return Err(bad(C, format!("empty header in column {}", i + 1))); @@ -373,27 +377,8 @@ pub(super) fn dataset_import(s: &mut Session, p: &Value) -> Result { return Err(bad(C, "CSV has a header but no data rows")); } let mut parsed: Vec = Vec::new(); - // The names of the rows read so far, and the last number given to a row without a name. - let (mut seen, mut number) = (BTreeSet::new(), 0usize); - for &(line, ref row, ref quoted) in rows.iter().skip(1) { - let raw_name = if name_column { row.first().map_or("", |c| c.trim()) } else { "" }; - let name = if raw_name.is_empty() { - // Same numbering as Capture Data Set when a row leaves the name blank or the file has - // no name column. The search starts after the last number given. - let (n, name) = (number + 1..) - .map(|n| (n, format!("Data Set {n}"))) - .find(|(_, n)| !seen.contains(n) && !existing_names.contains(n)) - .unwrap_or_else(|| (line, format!("Data Set {line}"))); - number = n; - name - } else if raw_name.chars().count() > MAX_NAME { - return Err(bad(C, format!("line {line}: `name` is longer than {MAX_NAME} characters"))); - } else { - raw_name.to_string() - }; - if !seen.insert(name.clone()) { - return Err(bad(C, format!("line {line}: dataset `{name}` appears twice"))); - } + for (n, &(line, ref row, ref quoted)) in rows.iter().skip(1).enumerate() { + let name = format!("Data Set {}", n + 1); let mut values = BTreeMap::new(); for &(col, ref var, kind) in &columns { let cell = row.get(col).map(String::as_str).unwrap_or(""); @@ -406,26 +391,23 @@ pub(super) fn dataset_import(s: &mut Session, p: &Value) -> Result { } let names: Vec = parsed.iter().map(|d| d.name.clone()).collect(); let new_variables: Vec = defined.iter().map(|v| v.name.clone()).collect(); + let deleted: Vec = s.doc()?.doc.variables.datasets.iter().map(|d| d.name.clone()).collect(); // The import is one undo step, and an import that leaves the variables and data sets as they - // were adds none. - let (imported, updated) = super::edit_variables(s, "Import Data Sets", |vars| { - vars.variables.extend(defined); - let (mut imported, mut updated) = (0usize, 0usize); - for row in parsed { - if let Some(existing) = vars.datasets.iter_mut().find(|x| x.name == row.name) { - *existing = row; - updated += 1; - } else { - vars.datasets.push(row); - imported += 1; - } + // were adds none, so the active data set stays when nothing changes. + super::edit_variables(s, "Import Data Sets", |vars| { + if defined.is_empty() && vars.datasets == parsed { + return Ok(()); } - Ok((imported, updated)) + vars.variables.extend(defined); + vars.datasets = parsed; + vars.active_dataset = None; + Ok(()) })?; - Ok(json!({"imported": imported, "updated": updated, "names": names, "defined": new_variables})) + Ok(json!({"imported": names.len(), "names": names, "defined": new_variables, "deletedDataSets": deleted})) } -/// Export every dataset as CSV ([`datasets_to_csv`]). With `path`, write the file; else return +/// Export every dataset as CSV ([`datasets_to_csv`]: one column per variable, no dataset names). +/// With `path`, write the file; else return /// `{csv}`. The reply's `warning` names the variables whose column makes a document without them /// refuse the file, or else the variables it doesn't read back with their kind /// ([`kinds_a_new_document_loses`]); the app shows it in its status bar. @@ -439,7 +421,6 @@ pub(super) fn dataset_export(s: &mut Session, p: &Value) -> Result { if st.doc.variables.datasets.len() > MAX_CSV_ROWS { return Err(bad(C, format!("more than {MAX_CSV_ROWS} data sets: Import Data Sets reads at most {MAX_CSV_ROWS}"))); } - // The name column comes first, so the import's column cap leaves room for this many variables. if st.doc.variables.variables.len() > MAX_CSV_COLS { return Err(bad(C, format!("more than {MAX_CSV_COLS} variables: Import Data Sets reads at most {MAX_CSV_COLS}"))); } diff --git a/crates/engine/src/tests/variables.rs b/crates/engine/src/tests/variables.rs index 493871de2..e6ef5e69d 100644 --- a/crates/engine/src/tests/variables.rs +++ b/crates/engine/src/tests/variables.rs @@ -245,25 +245,26 @@ fn mismatched_and_missing_bindings_are_skipped_and_counted() { fn a_row_with_no_value_for_a_bound_variable_warns_that_nothing_was_applied() { let mut s = session(); let t = text(&mut s, "x"); - s.execute("dataset.import", &json!({"csv": "name,Name,Title\nA,Ann,Boss\nB,Bob,Chief\n"})).unwrap(); + s.execute("dataset.import", &json!({"csv": "Name,Title\nAnn,Boss\nBob,Chief\n"})).unwrap(); + let (a, b) = ("Data Set 1", "Data Set 2"); let warning = |row: &str| format!("nothing in data set {row} was applied: no object is bound to a variable it has a value for"); - let v = s.execute("dataset.select", &json!({"name": "A"})).unwrap(); - assert_eq!((v["applied"].as_u64(), v["skipped"].as_u64(), &v["warning"]), (Some(0), Some(0), &json!(warning("A"))), "{v}"); + let v = s.execute("dataset.select", &json!({"name": a})).unwrap(); + assert_eq!((v["applied"].as_u64(), v["skipped"].as_u64(), &v["warning"]), (Some(0), Some(0), &json!(warning(a))), "{v}"); let v = s.execute("dataset.next", &json!({})).unwrap(); - assert_eq!(v["warning"], json!(warning("B")), "{v}"); + assert_eq!(v["warning"], json!(warning(b)), "{v}"); let v = s.execute("dataset.prev", &json!({})).unwrap(); - assert_eq!(v["warning"], json!(warning("A")), "{v}"); + assert_eq!(v["warning"], json!(warning(a)), "{v}"); assert_eq!(plain(&s, t), "x"); // A bound variable that the row has no value for still applies nothing. s.execute("variable.define", &json!({"name": "Other", "kind": "text"})).unwrap(); s.execute("variable.bind", &json!({"variable": "Other", "ids": [t.0]})).unwrap(); - let v = s.execute("dataset.select", &json!({"name": "B"})).unwrap(); - assert_eq!((v["applied"].as_u64(), v["skipped"].as_u64(), &v["warning"]), (Some(0), Some(1), &json!(warning("B"))), "{v}"); + let v = s.execute("dataset.select", &json!({"name": b})).unwrap(); + assert_eq!((v["applied"].as_u64(), v["skipped"].as_u64(), &v["warning"]), (Some(0), Some(1), &json!(warning(b))), "{v}"); // Bound to a variable with a value, the row applies it and gives no warning. s.execute("variable.bind", &json!({"variable": "Title", "ids": [t.0]})).unwrap(); - let v = s.execute("dataset.select", &json!({"name": "A"})).unwrap(); + let v = s.execute("dataset.select", &json!({"name": a})).unwrap(); assert_eq!(v["applied"].as_u64(), Some(1), "{v}"); assert!(v.get("warning").is_none(), "{v}"); assert_eq!(plain(&s, t), "Boss"); @@ -271,7 +272,7 @@ fn a_row_with_no_value_for_a_bound_variable_warns_that_nothing_was_applied() { // Locked art is skipped, and its binding still counts as bound. s.execute("select.set", &json!({"ids": [t.0]})).unwrap(); s.execute("object.lock", &json!({})).unwrap(); - let v = s.execute("dataset.select", &json!({"name": "B"})).unwrap(); + let v = s.execute("dataset.select", &json!({"name": b})).unwrap(); assert_eq!(v["applied"].as_u64(), Some(0), "{v}"); assert!(v.get("warning").is_none(), "{v}"); } @@ -613,8 +614,8 @@ fn locked_variables_refuse_new_deleted_and_renamed_variables() { s.execute("variable.unbind", &json!({"ids": [r.0], "variable": "Show"})).unwrap(); s.execute("dataset.capture", &json!({})).unwrap(); s.execute("dataset.new", &json!({"name": "Bob", "values": {"Name": "Bob"}})).unwrap(); - s.execute("dataset.import", &json!({"csv": "name,Name\nCarol,Carol\n"})).unwrap(); - s.execute("dataset.select", &json!({"name": "Carol"})).unwrap(); + s.execute("dataset.import", &json!({"csv": "Name\nCarol\n"})).unwrap(); + s.execute("dataset.select", &json!({"name": "Data Set 1"})).unwrap(); assert_eq!(plain(&s, t), "Carol"); // Unlocking lets a variable be defined again; `locked` sets the state rather than toggling. @@ -1395,12 +1396,11 @@ fn datasets_csv_keeps_line_breaks_in_quoted_cells() { let value = "Line one, \"quoted\"\nLine two"; s.execute("dataset.new", &json!({"name": "A", "values": {"Name": value}})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); - s.execute("dataset.delete", &json!({"names": ["A"]})).unwrap(); s.execute("dataset.import", &json!({"csv": csv})).unwrap(); let sets = &s.doc().unwrap().doc.variables.datasets; assert_eq!(sets.len(), 1); assert_eq!(sets[0].values.get("Name"), Some(&vectorcraft_doc::DataValue::Text(value.into()))); - assert!(s.execute("dataset.import", &json!({"csv": "name,Name\nX,\"open"})).is_err(), "unclosed quote"); + assert!(s.execute("dataset.import", &json!({"csv": "Name\n\"open"})).is_err(), "unclosed quote"); } /// An empty text value is not a missing value: it comes back from the exported CSV as an empty @@ -1413,23 +1413,45 @@ fn datasets_csv_keeps_an_empty_text_value() { s.execute("variable.bind", &json!({"variable": "Name", "ids": [t.0]})).unwrap(); s.execute("dataset.new", &json!({"name": "A", "values": {"Name": ""}})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); - s.execute("dataset.delete", &json!({"names": ["A"]})).unwrap(); s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - let value = s.doc().unwrap().doc.variables.dataset("A").and_then(|d| d.values.get("Name")).cloned(); - assert_eq!(value, Some(vectorcraft_doc::DataValue::Text(String::new()))); - s.execute("dataset.import", &json!({"csv": "name,Name\nB,\nC,\"\"\n"})).unwrap(); - let value = |name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("Name")).cloned(); - assert_eq!(value("C"), Some(vectorcraft_doc::DataValue::Text(String::new()))); + let value = |s: &Session, name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("Name")).cloned(); + assert_eq!(value(&s, "Data Set 1"), Some(vectorcraft_doc::DataValue::Text(String::new()))); + s.execute("variable.define", &json!({"name": "Other", "kind": "text"})).unwrap(); + s.execute("dataset.import", &json!({"csv": "Name,Other\n,x\n\"\",y\n"})).unwrap(); + assert_eq!(value(&s, "Data Set 2"), Some(vectorcraft_doc::DataValue::Text(String::new()))); // Compatibility rule for files written before empty text was quoted: a bare empty cell stays "no value". - assert_eq!(value("B"), None); + assert_eq!(value(&s, "Data Set 1"), None); // A data set with no value for the variable exports a bare cell and stays without a value. - s.execute("dataset.new", &json!({"name": "D", "values": {}})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); - assert!(csv.contains("\nD,\n"), "{csv:?}"); - s.execute("dataset.delete", &json!({"names": ["D"]})).unwrap(); + assert_eq!(csv, "Name,Other\n,x\n\"\",y\n"); s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - let value = |name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("Name")).cloned(); - assert_eq!(value("D"), None); + assert_eq!(value(&s, "Data Set 1"), None); +} + +/// A data set with no value for any variable exports as a row that imports back: a quoted empty +/// cell in the first column that isn't a text variable reads back as no value, and with only text +/// variables the first column reads back as an empty text. +#[test] +fn datasets_csv_keeps_a_data_set_without_values() { + let mut s = session(); + s.execute("variable.define", &json!({"name": "Title", "kind": "text"})).unwrap(); + s.execute("dataset.new", &json!({"name": "A", "values": {}})).unwrap(); + s.execute("dataset.new", &json!({"name": "B", "values": {"Title": "x"}})).unwrap(); + let export = |s: &mut Session| s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); + let csv = export(&mut s); + assert_eq!(csv, "Title\n\"\"\nx\n"); + let imp = s.execute("dataset.import", &json!({"csv": csv})).unwrap(); + assert_eq!(imp["names"], json!(["Data Set 1", "Data Set 2"])); + let values = |s: &Session, name: &str| s.doc().unwrap().doc.variables.dataset(name).map(|d| d.values.clone()).unwrap(); + assert_eq!(values(&s, "Data Set 1").get("Title"), Some(&vectorcraft_doc::DataValue::Text(String::new()))); + + s.execute("variable.define", &json!({"name": "Show", "kind": "visibility"})).unwrap(); + s.execute("dataset.new", &json!({"name": "C", "values": {}})).unwrap(); + let csv = export(&mut s); + assert!(csv.ends_with("\n,\"\"\n"), "{csv:?}"); + s.execute("dataset.import", &json!({"csv": csv})).unwrap(); + assert_eq!(s.doc().unwrap().doc.variables.datasets.len(), 3); + assert!(values(&s, "Data Set 3").is_empty()); } /// A quoted text cell keeps its leading and trailing spaces, so a padded value comes back from @@ -1442,25 +1464,28 @@ fn datasets_csv_keeps_spaces_in_quoted_cells() { s.execute("variable.bind", &json!({"variable": "Name", "ids": [t.0]})).unwrap(); s.execute("dataset.new", &json!({"name": "A", "values": {"Name": " padded "}})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); - s.execute("dataset.delete", &json!({"names": ["A"]})).unwrap(); s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - s.execute("dataset.import", &json!({"csv": "name,Name\nB, typed \nC,\" quoted \" \n"})).unwrap(); - let value = |name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("Name")).cloned(); + let value = |s: &Session, name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("Name")).cloned(); let text_value = |v: &str| Some(vectorcraft_doc::DataValue::Text(v.into())); - assert_eq!(value("A"), text_value(" padded ")); - assert_eq!(value("B"), text_value("typed")); - assert_eq!(value("C"), text_value(" quoted ")); + assert_eq!(value(&s, "Data Set 1"), text_value(" padded ")); + s.execute("dataset.import", &json!({"csv": "Name\n typed \n\" quoted \" \n"})).unwrap(); + assert_eq!(value(&s, "Data Set 1"), text_value("typed")); + assert_eq!(value(&s, "Data Set 2"), text_value(" quoted ")); } /// Export refuses a document with more data sets or variables than `dataset.import` reads, so /// every file it writes imports back: 1,000 data sets and 256 variables export and import, and -/// one more of either is refused. +/// one more of either is refused. The import refuses a line of 257 columns, the last one included. #[test] fn datasets_export_refuses_more_than_import_reads() { let mut s = session(); let header: Vec = (1..=256).map(|i| format!("V{i}")).collect(); + for wide in [format!("{},V257\nx\n", header.join(",")), format!("{}\nx{}\n", header.join(","), ",".repeat(256))] { + let err = s.execute("dataset.import", &json!({"csv": wide})).unwrap_err().to_string(); + assert!(err.contains("more than 256 columns"), "{err}"); + } let rows: String = (1..=1000).map(|i| format!("Set {i},x\n")).collect(); - s.execute("dataset.import", &json!({"csv": format!("name,{}\n{rows}", header.join(","))})).unwrap(); + s.execute("dataset.import", &json!({"csv": format!("{}\n{rows}", header.join(","))})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); s.execute("dataset.import", &json!({"csv": csv})).unwrap(); @@ -1488,59 +1513,102 @@ fn datasets_csv_reads_tab_separated_files() { s.execute("variable.define", &json!({"name": "Title", "kind": "text"})).unwrap(); s.execute("variable.define", &json!({"name": "Show", "kind": "visibility"})).unwrap(); s.execute("variable.define", &json!({"name": "Last, First", "kind": "text"})).unwrap(); - let tsv = "name\tTitle\tShow\r\nA\tTwo words\ttrue\r\nB\t\"a, b\tc\"\tfalse\r\n"; + let tsv = "Title\tShow\r\nTwo words\ttrue\r\n\"a, b\tc\"\tfalse\r\n"; let imp = s.execute("dataset.import", &json!({"csv": tsv})).unwrap(); - assert_eq!(imp["names"], json!(["A", "B"])); - s.execute("dataset.import", &json!({"csv": "name,Title\nC,one\ttwo"})).unwrap(); - s.execute("dataset.import", &json!({"csv": "name\t\"Last, First\"\nD\tSmith, Jo"})).unwrap(); - let value = |name: &str, var: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get(var)).cloned(); + assert_eq!(imp["names"], json!(["Data Set 1", "Data Set 2"])); + let value = |s: &Session, name: &str, var: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get(var)).cloned(); let text_value = |v: &str| Some(vectorcraft_doc::DataValue::Text(v.into())); - assert_eq!(value("A", "Title"), text_value("Two words")); - assert_eq!(value("A", "Show"), Some(vectorcraft_doc::DataValue::Visible(true))); - assert_eq!(value("B", "Title"), text_value("a, b\tc")); - assert_eq!(value("B", "Show"), Some(vectorcraft_doc::DataValue::Visible(false))); - assert_eq!(value("C", "Title"), text_value("one\ttwo")); - assert_eq!(value("D", "Last, First"), text_value("Smith, Jo")); - let unclosed = s.execute("dataset.import", &json!({"csv": "name\tTitle\t\"Show\nA\tx\ttrue"})).unwrap_err().to_string(); + assert_eq!(value(&s, "Data Set 1", "Title"), text_value("Two words")); + assert_eq!(value(&s, "Data Set 1", "Show"), Some(vectorcraft_doc::DataValue::Visible(true))); + assert_eq!(value(&s, "Data Set 2", "Title"), text_value("a, b\tc")); + assert_eq!(value(&s, "Data Set 2", "Show"), Some(vectorcraft_doc::DataValue::Visible(false))); + s.execute("dataset.import", &json!({"csv": "Title,Show\none\ttwo,true"})).unwrap(); + assert_eq!(value(&s, "Data Set 1", "Title"), text_value("one\ttwo")); + s.execute("dataset.import", &json!({"csv": "\"Last, First\"\tShow\nSmith, Jo\ttrue"})).unwrap(); + assert_eq!(value(&s, "Data Set 1", "Last, First"), text_value("Smith, Jo")); + // A file of one variable whose name holds a tab is exported with the name quoted, and reads back as one column. + let mut one = session(); + one.execute("variable.define", &json!({"name": "Tab\tName", "kind": "text"})).unwrap(); + one.execute("dataset.new", &json!({"name": "A", "values": {"Tab\tName": "v"}})).unwrap(); + let csv = one.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); + assert_eq!(csv, "\"Tab\tName\"\nv\n"); + let imp = one.execute("dataset.import", &json!({"csv": csv})).unwrap(); + assert_eq!(imp["defined"], json!([]), "{imp}"); + assert_eq!(value(&one, "Data Set 1", "Tab\tName"), text_value("v")); + let unclosed = s.execute("dataset.import", &json!({"csv": "Title\t\"Show\nx\ttrue"})).unwrap_err().to_string(); assert!(unclosed.contains("unclosed quote"), "{unclosed}"); } -/// Without a `name` column every header cell is a variable, and the rows take the first free -/// names Data Set 1, 2 and so on. A first header cell `name`, in any case, still holds the names. +/// Every header cell is a variable, whatever its case: a file laid out as the set-up page shows +/// (`Name,Age`) defines and fills `Name`, and `name` and `NAME` are variables too. The rows are +/// named Data Set 1, 2 and so on, in file order. #[test] -fn datasets_csv_without_a_name_column_numbers_the_rows() { +fn datasets_csv_reads_every_header_cell_as_a_variable() { let mut s = session(); - s.execute("variable.define", &json!({"name": "Title", "kind": "text"})).unwrap(); - s.execute("dataset.new", &json!({"name": "Data Set 2", "values": {"Title": "kept"}})).unwrap(); - let imp = s.execute("dataset.import", &json!({"csv": "Title\nx\ny\n"})).unwrap(); - assert_eq!((imp["imported"].as_u64(), imp["updated"].as_u64()), (Some(2), Some(0))); - assert_eq!(imp["names"], json!(["Data Set 1", "Data Set 3"])); - let value = |s: &Session, name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("Title")).cloned(); - let text_value = |v: &str| Some(vectorcraft_doc::DataValue::Text(v.into())); - assert_eq!(value(&s, "Data Set 1"), text_value("x")); - assert_eq!(value(&s, "Data Set 2"), text_value("kept")); - assert_eq!(value(&s, "Data Set 3"), text_value("y")); - let named = s.execute("dataset.import", &json!({"csv": "NAME,Title\nData Set 1,z"})).unwrap(); - assert_eq!((named["imported"].as_u64(), named["updated"].as_u64()), (Some(0), Some(1))); - assert_eq!(value(&s, "Data Set 1"), text_value("z")); + let t = text(&mut s, "x"); + let imp = s.execute("dataset.import", &json!({"csv": "Name,Age\nBill Tucker,36\nJo Lee,41\n"})).unwrap(); + assert_eq!((imp["imported"].as_u64(), &imp["defined"]), (Some(2), &json!(["Name", "Age"])), "{imp}"); + assert_eq!(imp["names"], json!(["Data Set 1", "Data Set 2"])); + s.execute("variable.bind", &json!({"variable": "Name", "ids": [t.0]})).unwrap(); + s.execute("dataset.select", &json!({"name": "Data Set 2"})).unwrap(); + assert_eq!(plain(&s, t), "Jo Lee"); + for header in ["name", "NAME"] { + let imp = s.execute("dataset.import", &json!({"csv": format!("{header},Age\nx,1\n")})).unwrap(); + assert_eq!((&imp["defined"], &imp["names"]), (&json!([header]), &json!(["Data Set 1"])), "{header}: {imp}"); + } + let value = |s: &Session, var: &str| s.doc().unwrap().doc.variables.dataset("Data Set 1").and_then(|d| d.values.get(var)).cloned(); + assert_eq!(value(&s, "NAME"), Some(vectorcraft_doc::DataValue::Text("x".into()))); } -/// An error about a row's values or name gives the file line the row starts on, counting blank -/// lines and line breaks inside quoted cells. +/// An import replaces every data set of the document, as Load Variable Library does, and clears +/// the active data set. The document's variables stay with their bindings, including those the +/// file has no column for, which then have no value in any data set. Undo brings the data sets back. +#[test] +fn datasets_csv_import_replaces_the_data_sets() { + let mut s = session(); + let t = text(&mut s, "x"); + let r = rect(&mut s); + s.execute("variable.define", &json!({"name": "Title", "kind": "text"})).unwrap(); + s.execute("variable.define", &json!({"name": "Show", "kind": "visibility"})).unwrap(); + s.execute("variable.bind", &json!({"variable": "Title", "ids": [t.0]})).unwrap(); + s.execute("variable.bind", &json!({"variable": "Show", "ids": [r.0]})).unwrap(); + for (name, title) in [("First", "a"), ("Data Set 2", "b"), ("Data Set 3", "c")] { + s.execute("dataset.new", &json!({"name": name, "values": {"Title": title, "Show": true}})).unwrap(); + } + s.execute("dataset.select", &json!({"name": "First"})).unwrap(); + let imp = s.execute("dataset.import", &json!({"csv": "Title\ny\nz\n"})).unwrap(); + assert_eq!(imp["names"], json!(["Data Set 1", "Data Set 2"])); + assert_eq!(imp["deletedDataSets"], json!(["First", "Data Set 2", "Data Set 3"])); + let vars = &s.doc().unwrap().doc.variables; + let sets: Vec<_> = vars.datasets.iter().map(|d| (d.name.as_str(), d.values.get("Title").cloned(), d.values.get("Show").cloned())).collect(); + let text_value = |v: &str| Some(vectorcraft_doc::DataValue::Text(v.into())); + assert_eq!(sets, [("Data Set 1", text_value("y"), None), ("Data Set 2", text_value("z"), None)]); + assert_eq!(vars.active_dataset, None); + let listed = s.execute("variable.list", &json!({})).unwrap(); + assert_eq!(listed["variables"], json!([{"name": "Title", "kind": "text", "bindings": 1}, {"name": "Show", "kind": "visibility", "bindings": 1}])); + s.execute("dataset.select", &json!({"name": "Data Set 2"})).unwrap(); + assert_eq!(plain(&s, t), "z"); + + // The first undo takes back the selection, the second the import. + s.execute("edit.undo", &json!({})).unwrap(); + s.execute("edit.undo", &json!({})).unwrap(); + let vars = &s.doc().unwrap().doc.variables; + let names: Vec<&str> = vars.datasets.iter().map(|d| d.name.as_str()).collect(); + assert_eq!((names, vars.active_dataset.as_deref()), (vec!["First", "Data Set 2", "Data Set 3"], Some("First"))); +} + +/// An error about a row's values gives the file line the row starts on, counting blank lines and +/// line breaks inside quoted cells. #[test] fn datasets_csv_row_errors_name_the_file_line() { let mut s = session(); s.execute("variable.define", &json!({"name": "Title", "kind": "text"})).unwrap(); s.execute("variable.define", &json!({"name": "Show", "kind": "visibility"})).unwrap(); let error = |s: &mut Session, csv: &str| s.execute("dataset.import", &json!({"csv": csv})).unwrap_err().to_string(); - let blank_line = error(&mut s, "name,Show\n\nX,maybe"); + let blank_line = error(&mut s, "Show\n\nmaybe"); assert!(blank_line.contains("line 3:"), "{blank_line}"); - let line_break = error(&mut s, "name,Title,Show\nA,\"one\ntwo\",true\nB,x,maybe"); + let line_break = error(&mut s, "Title,Show\n\"one\ntwo\",true\nx,maybe"); assert!(line_break.contains("line 4:"), "{line_break}"); - let twice = error(&mut s, "name,Title\nA,x\n\nA,y"); - assert!(twice.contains("line 4:"), "{twice}"); - let long = error(&mut s, &format!("name,Title\n\n{},x", "n".repeat(256))); - assert!(long.contains("line 3:") && long.contains("longer than 255"), "{long}"); } /// A refused cell's error gives its line and why the value was refused: a visibility value that @@ -1551,10 +1619,10 @@ fn datasets_csv_cell_errors_say_why() { s.execute("variable.define", &json!({"name": "Show", "kind": "visibility"})).unwrap(); s.execute("variable.define", &json!({"name": "Sales", "kind": "graph"})).unwrap(); let error = |s: &mut Session, csv: &str| s.execute("dataset.import", &json!({"csv": csv})).unwrap_err(); - let maybe = error(&mut s, "name,Show\nX,maybe"); + let maybe = error(&mut s, "Show\nmaybe"); let want = "line 2: value for `Show` must be true or false"; assert!(matches!(&maybe, EngineError::BadParams { msg, .. } if msg == want), "{maybe}"); - let long = error(&mut s, &format!("name,Sales\nX,\"{}\"", "1,".repeat(140_000))); + let long = error(&mut s, &format!("Sales\n\"{}\"", "1,".repeat(140_000))); let want = "line 2: value for `Sales` is longer than 262144 bytes"; assert!(matches!(&long, EngineError::BadParams { msg, .. } if msg == want), "{long}"); assert!(s.doc().unwrap().doc.variables.datasets.is_empty()); @@ -1567,10 +1635,10 @@ fn datasets_csv_cell_errors_say_why() { #[test] fn datasets_csv_header_prefixes_define_variables() { let mut s = session(); - let csv = "name,#Shown,@Photo,Title\nA,true,a.png,x"; + let csv = "#Shown,@Photo,Title\ntrue,a.png,x"; let imp = s.execute("dataset.import", &json!({"csv": csv})).unwrap(); assert_eq!(imp["defined"], json!(["Shown", "Photo", "Title"])); - assert_eq!((imp["imported"].as_u64(), imp["names"].clone()), (Some(1), json!(["A"]))); + assert_eq!((imp["imported"].as_u64(), imp["names"].clone()), (Some(1), json!(["Data Set 1"]))); let listed = s.execute("variable.list", &json!({})).unwrap(); assert_eq!( listed["variables"], @@ -1580,23 +1648,23 @@ fn datasets_csv_header_prefixes_define_variables() { {"name": "Title", "kind": "text", "bindings": 0}, ]) ); - let value = |s: &Session, var: &str| s.doc().unwrap().doc.variables.dataset("A").and_then(|d| d.values.get(var)).cloned(); + let value = |s: &Session, var: &str| s.doc().unwrap().doc.variables.dataset("Data Set 1").and_then(|d| d.values.get(var)).cloned(); assert_eq!(value(&s, "Shown"), Some(vectorcraft_doc::DataValue::Visible(true))); assert_eq!(value(&s, "Photo"), Some(vectorcraft_doc::DataValue::Image("a.png".into()))); assert_eq!(value(&s, "Title"), Some(vectorcraft_doc::DataValue::Text("x".into()))); - let again = s.execute("dataset.import", &json!({"csv": "name\t#Shown\t@Photo\tTitle\nA\tfalse\tb.png\ty"})).unwrap(); - assert_eq!((again["updated"].as_u64(), again["defined"].clone()), (Some(1), json!([]))); + let again = s.execute("dataset.import", &json!({"csv": "#Shown\t@Photo\tTitle\nfalse\tb.png\ty"})).unwrap(); + assert_eq!((again["imported"].as_u64(), again["defined"].clone()), (Some(1), json!([]))); assert_eq!(value(&s, "Shown"), Some(vectorcraft_doc::DataValue::Visible(false))); let error = |s: &mut Session, csv: &str| s.execute("dataset.import", &json!({"csv": csv})).unwrap_err().to_string(); - let other_kind = error(&mut s, "name,New,#Title\nB,n,true"); + let other_kind = error(&mut s, "Shown,New,#Title\ntrue,n,true"); assert!(other_kind.contains("column 3:") && other_kind.contains("text variable"), "{other_kind}"); let twice = error(&mut s, "Fresh,@Fresh\nn,f.png"); assert!(twice.contains("column 2:") && twice.contains("text variable"), "{twice}"); - let bare = error(&mut s, "name,#\nB,true"); + let bare = error(&mut s, "Title,#\nx,true"); assert!(bare.contains("column 2:") && bare.contains("names no variable"), "{bare}"); - let long = error(&mut s, &format!("name,@{}\nB,b.png", "n".repeat(256))); + let long = error(&mut s, &format!("Title,@{}\nx,b.png", "n".repeat(256))); assert!(long.contains("column 2:") && long.contains("longer than 255"), "{long}"); assert_eq!(s.doc().unwrap().doc.variables.variables.len(), 3, "a failed import defines nothing"); @@ -1620,11 +1688,10 @@ fn datasets_csv_exact_names_win_over_prefixes() { let values = json!({"#Shown": "hello", "@Photo": true, "%Chart": ",Q\nX,3", "Shown": "s.png"}); s.execute("dataset.new", &json!({"name": "A", "values": values})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); - assert!(csv.starts_with("name,#Shown,#@Photo,%Chart,@Shown\n"), "{csv}"); - s.execute("dataset.delete", &json!({"names": ["A"]})).unwrap(); + assert!(csv.starts_with("#Shown,#@Photo,%Chart,@Shown\n"), "{csv}"); let imp = s.execute("dataset.import", &json!({"csv": csv})).unwrap(); assert_eq!((imp["imported"].as_u64(), imp["defined"].clone()), (Some(1), json!([]))); - let value = |var: &str| s.doc().unwrap().doc.variables.dataset("A").and_then(|d| d.values.get(var)).cloned(); + let value = |var: &str| s.doc().unwrap().doc.variables.dataset("Data Set 1").and_then(|d| d.values.get(var)).cloned(); assert_eq!(value("#Shown"), Some(vectorcraft_doc::DataValue::Text("hello".into()))); assert_eq!(value("@Photo"), Some(vectorcraft_doc::DataValue::Visible(true))); assert_eq!(value("%Chart"), Some(vectorcraft_doc::DataValue::Graph(",Q\nX,3".into()))); @@ -1644,7 +1711,7 @@ fn datasets_csv_export_marks_visibility_and_image_columns() { } s.execute("dataset.new", &json!({"name": "A", "values": {"Name": "Ann", "Show": false, "Photo": "a.png"}})).unwrap(); let out = s.execute("dataset.export", &json!({})).unwrap(); - assert_eq!(out["csv"], "name,Name,#Show,@Photo\nA,Ann,false,a.png\n"); + assert_eq!(out["csv"], "Name,#Show,@Photo\nAnn,false,a.png\n"); assert!(out.get("warning").is_none(), "{out}"); let mut fresh = session(); @@ -1653,7 +1720,8 @@ fn datasets_csv_export_marks_visibility_and_image_columns() { let listed = fresh.execute("variable.list", &json!({})).unwrap(); let kinds: Vec<_> = listed["variables"].as_array().unwrap().iter().map(|v| v["kind"].clone()).collect(); assert_eq!(kinds, [json!("text"), json!("visibility"), json!("image")]); - let value = |var: &str| fresh.doc().unwrap().doc.variables.dataset("A").and_then(|d| d.values.get(var)).cloned(); + let value = |var: &str| fresh.doc().unwrap().doc.variables.dataset("Data Set 1").and_then(|d| d.values.get(var)).cloned(); + assert_eq!(value("Name"), Some(vectorcraft_doc::DataValue::Text("Ann".into()))); assert_eq!(value("Show"), Some(vectorcraft_doc::DataValue::Visible(false))); assert_eq!(value("Photo"), Some(vectorcraft_doc::DataValue::Image("a.png".into()))); @@ -1662,7 +1730,48 @@ fn datasets_csv_export_marks_visibility_and_image_columns() { } let out = s.execute("dataset.export", &json!({})).unwrap(); assert_eq!(out["warning"], "a new document won't read these variables back with their kind: `#Tag`, `Chart`, `@Note`"); - assert!(out["csv"].as_str().unwrap().starts_with("name,Name,#Show,@Photo,#Tag,Chart,@Note\n"), "{out}"); + assert!(out["csv"].as_str().unwrap().starts_with("Name,#Show,@Photo,#Tag,Chart,@Note\n"), "{out}"); +} + +/// A file `dataset.export` writes has no column of data set names, and `dataset.import` reads +/// every column as a variable, so the file imports back to the same data sets: the same variables +/// with their kinds, and each data set's values in order, a data set without values included, named +/// Data Set 1, 2 and so on. In a document whose data sets already have those names and values, the +/// import changes nothing. +#[test] +fn datasets_csv_export_imports_back_to_the_same_data_sets() { + let mut s = session(); + for (name, kind) in [("Name", "text"), ("Show", "visibility"), ("Photo", "image")] { + s.execute("variable.define", &json!({"name": name, "kind": kind})).unwrap(); + } + s.execute("dataset.new", &json!({"name": "A", "values": {"Name": "Ann, Jr.", "Show": true, "Photo": "a.png"}})).unwrap(); + s.execute("dataset.new", &json!({"name": "B", "values": {"Show": false}})).unwrap(); + s.execute("dataset.new", &json!({"name": "C", "values": {}})).unwrap(); + s.execute("dataset.new", &json!({"name": "D", "values": {"Name": "", "Photo": "d.png"}})).unwrap(); + let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); + assert!(csv.starts_with("Name,#Show,@Photo\n"), "{csv}"); + let before = s.doc().unwrap().doc.variables.clone(); + + let mut fresh = session(); + fresh.execute("dataset.import", &json!({"csv": csv})).unwrap(); + let vars = fresh.doc().unwrap().doc.variables.clone(); + assert_eq!(vars.variables, before.variables, "the same variables, with their kinds"); + let values = |v: &vectorcraft_doc::Variables| v.datasets.iter().map(|d| d.values.clone()).collect::>(); + assert_eq!(values(&vars), values(&before), "the same values in every data set"); + let names: Vec<&str> = vars.datasets.iter().map(|d| d.name.as_str()).collect(); + assert_eq!(names, ["Data Set 1", "Data Set 2", "Data Set 3", "Data Set 4"]); + + // Exporting the imported data sets writes the same file, and importing it changes nothing: no + // undo step, and the active data set stays. + let again = fresh.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); + assert_eq!(again, csv); + fresh.execute("dataset.select", &json!({"name": "Data Set 1"})).unwrap(); + let vars = fresh.doc().unwrap().doc.variables.clone(); + assert!(vars.active_dataset.is_some()); + let undo = fresh.doc().unwrap().history.undo.len(); + fresh.execute("dataset.import", &json!({"csv": again})).unwrap(); + assert_eq!(fresh.doc().unwrap().history.undo.len(), undo, "no undo step"); + assert_eq!(fresh.doc().unwrap().doc.variables, vars); } /// The export's `warning` names the variables whose column a document without the variables @@ -1719,7 +1828,7 @@ fn datasets_csv_export_refuses_a_header_that_names_another_variable() { fn datasets_csv_refuses_graph_data_file_columns() { let mut s = session(); let error = |s: &mut Session, csv: &str| s.execute("dataset.import", &json!({"csv": csv})).unwrap_err().to_string(); - let named = error(&mut s, "name,Title,%Sales\nA,x,sales.txt"); + let named = error(&mut s, "Name,Title,%Sales\nA,x,sales.txt"); assert!(named.contains("column 3:") && named.contains("`%Sales`"), "{named}"); let unnamed = error(&mut s, "%Sales\tTitle\nsales.txt\tx"); assert!(unnamed.contains("column 1:") && unnamed.contains("`%Sales`"), "{unnamed}"); @@ -1735,10 +1844,10 @@ fn datasets_csv_refuses_two_columns_for_one_variable() { s.execute("variable.define", &json!({"name": "Shown", "kind": "visibility"})).unwrap(); let error = |s: &mut Session, csv: &str| s.execute("dataset.import", &json!({"csv": csv})).unwrap_err().to_string(); for (csv, first, second, var) in [ - ("name,City,City\nR,Paris,Rome", 2, 3, "City"), - ("name,City,City\nR,Paris,", 2, 3, "City"), + ("Title,City,City\nR,Paris,Rome", 2, 3, "City"), + ("Title,City,City\nR,Paris,", 2, 3, "City"), ("City,Title,City\nParis,x,Rome", 1, 3, "City"), - ("name,Shown,#Shown\nR,true,false", 2, 3, "Shown"), + ("Title,Shown,#Shown\nR,true,false", 2, 3, "Shown"), ("Title\t@Photo\t@ Photo\nx\ta.png\tb.png", 2, 3, "Photo"), ] { let err = error(&mut s, csv); @@ -1757,21 +1866,22 @@ fn locked_variables_refuse_header_cells_that_define_variables() { s.execute("variable.define", &json!({"name": "Shown", "kind": "visibility"})).unwrap(); s.execute("variable.define", &json!({"name": "Title", "kind": "text"})).unwrap(); s.execute("variable.lock", &json!({"locked": true})).unwrap(); - for csv in ["name,#Shown,Title,New\nA,true,x,n", "@Photo\nb.png", "name,#Fresh\nA,true"] { + for csv in ["#Shown,Title,New\ntrue,x,n", "@Photo\nb.png", "Title,#Fresh\nx,true"] { let e = s.execute("dataset.import", &json!({"csv": csv})).unwrap_err(); assert!(matches!(&e, EngineError::Disabled(cmd, why) if cmd == "dataset.import" && why == "variables are locked"), "{csv}: {e}"); } let vars = &s.doc().unwrap().doc.variables; assert!(vars.variables.len() == 2 && vars.datasets.is_empty(), "a refused import defines and imports nothing"); - let imp = s.execute("dataset.import", &json!({"csv": "name,#Shown,Title\nA,true,x"})).unwrap(); + let imp = s.execute("dataset.import", &json!({"csv": "#Shown,Title\ntrue,x"})).unwrap(); assert_eq!((imp["imported"].as_u64(), imp["defined"].clone()), (Some(1), json!([]))); s.execute("variable.lock", &json!({"locked": false})).unwrap(); - let imp = s.execute("dataset.import", &json!({"csv": "name,New\nB,n"})).unwrap(); + let imp = s.execute("dataset.import", &json!({"csv": "New\nn"})).unwrap(); assert_eq!(imp["defined"], json!(["New"])); } -/// Import / export datasets as CSV: header `name` then variables; round-trip keeps values. +/// Import / export datasets as CSV: one column per variable and no dataset names; round-trip keeps +/// the values in order, and the rows come back as Data Set 1, 2. #[test] fn datasets_csv_round_trips_and_rejects_bad_input() { let dir = Folder::new("vars-csv"); @@ -1795,28 +1905,27 @@ fn datasets_csv_round_trips_and_rejects_bad_input() { let exported = s.execute("dataset.export", &json!({})).unwrap(); let csv = exported["csv"].as_str().unwrap().to_string(); - assert!(csv.starts_with("name,Name,#Show,@Photo\n"), "{csv}"); + assert!(csv.starts_with("Name,#Show,@Photo\n"), "{csv}"); assert!(csv.contains("Alice") && csv.contains("Bob") && csv.contains("false"), "{csv}"); - // Clear rows, import again, apply B. - s.execute("dataset.delete", &json!({"names": ["A", "B"]})).unwrap(); + // Import again, which replaces A and B, and apply the second row. let imp = s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - assert_eq!((imp["imported"].as_u64(), imp["updated"].as_u64()), (Some(2), Some(0))); - s.execute("dataset.select", &json!({"name": "B"})).unwrap(); + assert_eq!((imp["imported"].as_u64(), &imp["deletedDataSets"]), (Some(2), &json!(["A", "B"]))); + s.execute("dataset.select", &json!({"name": "Data Set 2"})).unwrap(); assert_eq!(plain(&s, t), "Bob"); assert!(!s.doc().unwrap().doc.node(r).unwrap().visible); assert_eq!(image(&s, img).link.as_ref().map(|l| l.path.as_str()), Some(blue_path.as_str())); - // Re-import updates in place and defines nothing; a bad visibility value is an error. + // Re-import replaces the rows and defines nothing; a bad visibility value is an error. let again = s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - assert_eq!((again["imported"].as_u64(), again["updated"].as_u64()), (Some(0), Some(2))); + assert_eq!((again["imported"].as_u64(), &again["deletedDataSets"]), (Some(2), &json!(["Data Set 1", "Data Set 2"]))); assert_eq!(again["defined"], json!([])); - assert!(s.execute("dataset.import", &json!({"csv": "name,Show\nX,maybe"})).is_err()); + assert!(s.execute("dataset.import", &json!({"csv": "Show\nmaybe"})).is_err()); let out = dir.file("sets.csv"); s.execute("dataset.export", &json!({"path": out})).unwrap(); let from_file = s.execute("dataset.import", &json!({"path": out})).unwrap(); - assert_eq!(from_file["updated"].as_u64(), Some(2)); + assert_eq!(from_file["names"], json!(["Data Set 1", "Data Set 2"])); // One undo undoes the last import's replacements. s.execute("edit.undo", &json!({})).unwrap(); @@ -1831,17 +1940,17 @@ fn datasets_csv_reads_windows_1252_files() { let import = |s: &mut Session, bytes: &[u8]| s.execute("dataset.import", &json!({"dataBase64": vectorcraft_format::base64_encode(bytes)})); let value = |s: &Session, name: &str| s.doc().unwrap().doc.variables.dataset(name).and_then(|d| d.values.get("City")).cloned(); let text_value = |v: &str| Some(vectorcraft_doc::DataValue::Text(v.into())); - let imp = import(&mut s, b"name,City\r\nA,Jos\xe9\r\nB,\x80 5 \x96 \x93Caf\xe9\x94\r\n").unwrap(); - assert_eq!(imp["names"], json!(["A", "B"])); - assert_eq!(value(&s, "A"), text_value("José")); - assert_eq!(value(&s, "B"), text_value("€ 5 – “Café”")); - import(&mut s, "name,City\nC,José\n".as_bytes()).unwrap(); - assert_eq!(value(&s, "C"), text_value("José")); - import(&mut s, "\u{feff}name,City\nD,Zoë\n".as_bytes()).unwrap(); - assert_eq!(value(&s, "D"), text_value("Zoë")); - let e = import(&mut s, b"\xef\xbb\xbfname,City\nE,Jos\xe9\n").unwrap_err().to_string(); + let imp = import(&mut s, b"City\r\nJos\xe9\r\n\x80 5 \x96 \x93Caf\xe9\x94\r\n").unwrap(); + assert_eq!(imp["names"], json!(["Data Set 1", "Data Set 2"])); + assert_eq!(value(&s, "Data Set 1"), text_value("José")); + assert_eq!(value(&s, "Data Set 2"), text_value("€ 5 – “Café”")); + import(&mut s, "City\nJosé\n".as_bytes()).unwrap(); + assert_eq!(value(&s, "Data Set 1"), text_value("José")); + import(&mut s, "\u{feff}City\nZoë\n".as_bytes()).unwrap(); + assert_eq!(value(&s, "Data Set 1"), text_value("Zoë")); + let e = import(&mut s, b"\xef\xbb\xbfCity\nJos\xe9\n").unwrap_err().to_string(); assert!(e.contains("CSV is not UTF-8"), "{e}"); - assert_eq!(value(&s, "E"), None); + assert_eq!(value(&s, "Data Set 1"), text_value("Zoë")); } /// A CSV that starts with a UTF-16 byte-order mark is read as UTF-16, and @@ -1858,24 +1967,24 @@ fn datasets_csv_reads_utf16_files() { let units = text.encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }); bom.iter().copied().chain(units).collect() }; - import(&mut s, &utf16(b"\xff\xfe", "name\tCity\r\nA\tZoë 😀\r\n", true)).unwrap(); - assert_eq!(value(&s, "A"), text_value("Zoë 😀")); - import(&mut s, &utf16(b"\xfe\xff", "name,City\nB,José\n", false)).unwrap(); - assert_eq!(value(&s, "B"), text_value("José")); - let mut odd = utf16(b"\xff\xfe", "name,City\nC,x\n", true); + import(&mut s, &utf16(b"\xff\xfe", "City\r\nZoë 😀\r\n", true)).unwrap(); + assert_eq!(value(&s, "Data Set 1"), text_value("Zoë 😀")); + import(&mut s, &utf16(b"\xfe\xff", "City\nJosé\n", false)).unwrap(); + assert_eq!(value(&s, "Data Set 1"), text_value("José")); + let mut odd = utf16(b"\xff\xfe", "City\nx\n", true); odd.push(b'y'); let e = import(&mut s, &odd).unwrap_err().to_string(); assert!(e.contains("CSV is not UTF-16"), "{e}"); - let e = import(&mut s, &[b"\xff\xfe".as_slice(), "name,City\nD,".as_bytes(), b"\x00\xd8"].concat()).unwrap_err().to_string(); + let e = import(&mut s, &[b"\xff\xfe".as_slice(), "City\nAB,".as_bytes(), b"\x00\xd8"].concat()).unwrap_err().to_string(); assert!(e.contains("CSV is not UTF-16"), "{e}"); - let e = import(&mut s, &utf16(b"", "name,City\nE,x\n", true)).unwrap_err().to_string(); + let e = import(&mut s, &utf16(b"", "City\nx\n", true)).unwrap_err().to_string(); assert!(e.contains("CSV holds NUL bytes"), "{e}"); - assert_eq!((value(&s, "C"), value(&s, "D"), value(&s, "E")), (None, None, None)); - s.execute("dataset.new", &json!({"name": "F", "values": {"City": "a\u{0}b"}})).unwrap(); + assert_eq!(value(&s, "Data Set 1"), text_value("José"), "a refused file changes nothing"); + s.execute("dataset.set", &json!({"name": "Data Set 1", "values": {"City": "a\u{0}b"}})).unwrap(); let csv = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); - s.execute("dataset.delete", &json!({"names": ["F"]})).unwrap(); + s.execute("dataset.delete", &json!({"names": ["Data Set 1"]})).unwrap(); s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - assert_eq!(value(&s, "F"), text_value("a\u{0}b")); + assert_eq!(value(&s, "Data Set 1"), text_value("a\u{0}b")); } /// A data set of [`library`]: its name, and each variable's name with its paragraphs. @@ -1993,31 +2102,30 @@ fn a_variable_library_replaces_every_variable_and_data_set() { } /// An import or a library load that leaves the variables and data sets as they were (the same CSV -/// imported again, a row rewritten with its own values, the same library loaded again) records no -/// undo step: the step to redo stays, and the document stays unmodified. An import or load that -/// changes something is still one undo step. +/// imported again, the same rows tab-separated, the file Export Data Sets writes, the same library +/// loaded again) records no undo step: the step to redo stays, and the document stays unmodified. +/// An import or load that changes something is still one undo step. #[test] fn an_import_or_library_load_that_changes_nothing_records_no_undo_step() { let mut s = session(); let t = text(&mut s, "Alice"); - let csv = "name,Name,#Shown\nA,Ann,true\nB,Bob,false\n"; + let csv = "Name,#Shown\nAnn,true\nBob,false\n"; s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - for csv in [csv, "name\tName\t#Shown\nB\tBob\tfalse\n", "Name,#Shown\nAnn,true\n"] { + let exported = s.execute("dataset.export", &json!({})).unwrap()["csv"].as_str().unwrap().to_string(); + for csv in [csv, "Name\t#Shown\nAnn\ttrue\nBob\tfalse\n", exported.as_str()] { super::select_with_a_step_to_redo(&mut s, &[t]); let before = super::history(&s); assert_eq!((before.1, before.2), (1, false), "a step to redo, and nothing unsaved"); let r = s.execute("dataset.import", &json!({"csv": csv})).unwrap(); - if csv.starts_with("name") { - assert_eq!((super::history(&s), r["imported"].clone()), (before, json!(0)), "{csv}"); - } else { - // Without a name column the row is a new data set, which is a change. - assert_eq!((super::history(&s), r["imported"].clone()), ((before.0 + 1, 0, true), json!(1)), "{csv}"); - } + assert_eq!((super::history(&s), r["imported"].clone()), (before, json!(2)), "{csv}"); + } + // A file with one of the two rows replaces both data sets with one, which is a change. + for csv in ["Name,#Shown\nAnn,true\n", "Name\nCy\n"] { + super::select_with_a_step_to_redo(&mut s, &[t]); + let undo = super::history(&s).0; + s.execute("dataset.import", &json!({"csv": csv})).unwrap(); + assert_eq!(super::history(&s), (undo + 1, 0, true), "{csv}"); } - super::select_with_a_step_to_redo(&mut s, &[t]); - let undo = super::history(&s).0; - s.execute("dataset.import", &json!({"csv": "name,Name\nB,Cy\n"})).unwrap(); - assert_eq!(super::history(&s), (undo + 1, 0, true)); let xml = library(&[("Name", "textcontent")], &[("A", &[("Name", &["Ann"])]), ("B", &[("Name", &["Bob"])])]); s.execute("variable.loadLibrary", &json!({"xml": xml})).unwrap(); diff --git a/crates/ui-egui/src/i18n/tests.rs b/crates/ui-egui/src/i18n/tests.rs index ae1a4f708..bb2de38d6 100644 --- a/crates/ui-egui/src/i18n/tests.rs +++ b/crates/ui-egui/src/i18n/tests.rs @@ -1792,8 +1792,8 @@ fn dataset_import_repeated_column_error_is_translated() { use vectorcraft_engine::{EngineError, Session}; let mut s = Session::new(); s.execute("file.new", &json!({})).unwrap(); - let e = s.execute("dataset.import", &json!({"csv": "name,City,City\nR,Paris,Rome"})).unwrap_err(); - let want = "columns 2 and 3 both name the variable `City`"; + let e = s.execute("dataset.import", &json!({"csv": "City,City\nParis,Rome"})).unwrap_err(); + let want = "columns 1 and 2 both name the variable `City`"; assert!(matches!(&e, EngineError::BadParams { msg, .. } if msg == want), "{e}"); let e = e.to_string(); for code in COMPLETE_MESSAGES { @@ -1842,8 +1842,8 @@ fn dataset_export_limit_errors_are_translated() { use vectorcraft_engine::{EngineError, Session}; let mut s = Session::new(); s.execute("file.new", &json!({})).unwrap(); - let rows: String = (1..=1000).map(|i| format!("Set {i},x\n")).collect(); - s.execute("dataset.import", &json!({"csv": format!("name,V1\n{rows}")})).unwrap(); + let rows = "x\n".repeat(1000); + s.execute("dataset.import", &json!({"csv": format!("V1\n{rows}")})).unwrap(); s.execute("dataset.new", &json!({"name": "Set 1001"})).unwrap(); let mut errors = vec![]; let e = s.execute("dataset.export", &json!({})).unwrap_err(); @@ -1906,8 +1906,8 @@ fn dataset_import_cell_errors_are_translated() { s.execute("variable.define", &json!({"name": "Sales", "kind": "graph"})).unwrap(); let mut errors = vec![]; for (csv, want) in [ - ("name,Show\nX,maybe".to_string(), "line 2: value for `Show` must be true or false"), - (format!("name,Sales\nX,\"{}\"", "1,".repeat(140_000)), "line 2: value for `Sales` is longer than 262144 bytes"), + ("Show\nmaybe".to_string(), "line 2: value for `Show` must be true or false"), + (format!("Sales\n\"{}\"", "1,".repeat(140_000)), "line 2: value for `Sales` is longer than 262144 bytes"), ] { let e = s.execute("dataset.import", &json!({"csv": csv})).unwrap_err(); assert!(matches!(&e, EngineError::BadParams { msg, .. } if msg == want), "{e}"); diff --git a/crates/ui-egui/src/panels/variables.rs b/crates/ui-egui/src/panels/variables.rs index ab89d2faf..87aa5cd13 100644 --- a/crates/ui-egui/src/panels/variables.rs +++ b/crates/ui-egui/src/panels/variables.rs @@ -429,23 +429,24 @@ mod tests { let mut app = VectorcraftApp::new(vectorcraft_engine::Session::new(), Default::default()); app.session.execute("file.new", &json!({"width": 300, "height": 200})).unwrap(); let t = app.session.execute("text.create", &json!({"x": 10, "y": 40, "text": "x"})).unwrap()["id"].as_u64().unwrap(); - app.run("dataset.import", json!({"csv": "name,Name\nA,Ann\nB,Bob\n"})).unwrap(); + app.run("dataset.import", json!({"csv": "Name\nAnn\nBob\n"})).unwrap(); + let (a, b) = ("Data Set 1", "Data Set 2"); let message = |row: &str| format!("nothing in data set {row} was applied: no object is bound to a variable it has a value for"); - app.run("dataset.select", json!({"name": "A"})).unwrap(); - assert_eq!(app.ui.status, message("A")); + app.run("dataset.select", json!({"name": a})).unwrap(); + assert_eq!(app.ui.status, message(a)); app.run("dataset.next", json!({})).unwrap(); - assert_eq!(app.ui.status, message("B")); + assert_eq!(app.ui.status, message(b)); app.run("dataset.prev", json!({})).unwrap(); - assert_eq!(app.ui.status, message("A")); + assert_eq!(app.ui.status, message(a)); app.run("variable.bind", json!({"variable": "Name", "ids": [t]})).unwrap(); - app.run("dataset.select", json!({"name": "B"})).unwrap(); + app.run("dataset.select", json!({"name": b})).unwrap(); assert_eq!(app.ui.status, ""); // Stepping to a data set that applies clears the message as well. for step in ["dataset.next", "dataset.prev"] { app.run("variable.unbind", json!({"variable": "Name", "ids": [t]})).unwrap(); - app.run("dataset.select", json!({"name": "A"})).unwrap(); - assert_eq!(app.ui.status, message("A")); + app.run("dataset.select", json!({"name": a})).unwrap(); + assert_eq!(app.ui.status, message(a)); app.run("variable.bind", json!({"variable": "Name", "ids": [t]})).unwrap(); app.run(step, json!({})).unwrap(); assert_eq!(app.ui.status, "", "{step}"); diff --git a/crates/ui-egui/src/tests/picks.rs b/crates/ui-egui/src/tests/picks.rs index 8608b7813..574bb07b5 100644 --- a/crates/ui-egui/src/tests/picks.rs +++ b/crates/ui-egui/src/tests/picks.rs @@ -238,8 +238,8 @@ fn load_profile_asks_off_the_ui_thread() { } /// The Variables panel menu's Import Data Sets… and Export Data Sets… ask off the UI thread: the -/// data sets in the file picked are loaded, and the header and data set `B` are written to the -/// path the dialog answers. +/// data sets in the file picked are loaded, and the header and the data set's row are written to +/// the path the dialog answers. #[test] fn variables_panel_data_sets_ask_off_the_ui_thread() { let (mut app, shown, _) = app(); @@ -250,20 +250,20 @@ fn variables_panel_data_sets_ask_off_the_ui_thread() { } std::fs::create_dir_all(&dir).unwrap(); let (csv, out) = (dir.join("sets.csv"), dir.join("out.csv")); - std::fs::write(&csv, "name,Name\nB,y\n").unwrap(); + std::fs::write(&csv, "Name\ny\n").unwrap(); let sets = |app: &VectorcraftApp| app.session.doc().unwrap().doc.variables.datasets.iter().map(|d| d.name.clone()).collect::>(); click(&mut app, "Import Data Sets…", crate::panels::variables::menu); assert!(matches!(shown.requests().as_slice(), [PickRequest::Open(_)])); shown.answer(&[&csv.to_string_lossy()]); poll(&mut app); - assert_eq!(sets(&app), ["B"], "{}", app.ui.status); + assert_eq!(sets(&app), ["Data Set 1"], "{}", app.ui.status); click(&mut app, "Export Data Sets…", crate::panels::variables::menu); assert!(matches!(shown.requests().as_slice(), [PickRequest::Save(_)])); shown.answer(&[&out.to_string_lossy()]); poll(&mut app); - assert_eq!(std::fs::read_to_string(&out).ok().as_deref(), Some("name,Name\nB,y\n"), "{}", app.ui.status); + assert_eq!(std::fs::read_to_string(&out).ok().as_deref(), Some("Name\ny\n"), "{}", app.ui.status); let _ = std::fs::remove_dir_all(dir); } diff --git a/docs/mcp.md b/docs/mcp.md index 55a58ec79..6878e137b 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -1684,10 +1684,13 @@ already walks Image Trace. Short agent entry points: renames the row, in the same undo step; Edit Data Set's Name field sends it), `dataset.rename`, `dataset.delete`, `dataset.capture`, `dataset.update`, `dataset.select {name}`, `dataset.next` / `dataset.prev` (each `dataset.select` / next / prev is one undo step over every bound object), `dataset.import {path|data|dataBase64|csv}` / - `dataset.export {path?}` (CSV: `name`, then one column per variable). `dataset.import` also reads tab-separated - files, and files whose first header cell isn't `name`: every header cell is then a variable, and the rows take the - first free names `Data Set 1`, `Data Set 2` and so on. An import error about a row's values or name gives the file - line the row starts on. A header cell that names no variable defines one, unbound, in the same undo step as the + `dataset.export {path?}` (CSV: one column per variable, and no column of data set names). `dataset.import` also + reads tab-separated files. Every header cell is a variable, whatever its case (`name` and `Name` included), as the + [set-up page](https://helpx.adobe.com/illustrator/desktop/automate-visualize-data/automate-actions/set-up-data-source-files.html) + describes the first row: "Specify your data field name in the first row of your spreadsheet." The rows replace every + data set of the document and are named `Data Set 1`, `Data Set 2` and so on, in file order; the document's variables + and their bindings stay, and the reply's `deletedDataSets` lists the data sets that were replaced. An import error + about a row's values gives the file line the row starts on. A header cell that names no variable defines one, unbound, in the same undo step as the rows. `#Shown` defines a visibility variable `Shown`, `@Photo` defines an image variable `Photo`, and a cell without a prefix defines a text variable. `dataset.export` writes a visibility variable's header as `#Shown` and an image variable's as `@Photo`, and a text or graph variable's as its name alone. Its reply's `warning` names the variables diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index 755c57da5..7d526a04a 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -572,7 +572,7 @@ Moved from the ROADMAP's "Shipped so far" on 2026-10-10. Update it when a featur - Export for Screens, Save for Web (slice images, and files named after the document's title) and the Libraries panel name files with the characters Windows does not allow written as `-` and with a `-` after a Windows device name such as `CON`, on every platform (`document.exportForScreens`, `document.exportForWeb`, `library.create`). - Export for Screens numbers distinct row files whose names collide, compared without case, as `-2`, `-3`… instead of leaving them out; identical rows still write one file. It puts row folders that differ only in case into one folder (`document.exportForScreens`). - Variables (data merge): text, visibility and image variables bound to objects (an object may carry several), dataset rows applied in one undo step (`variable.*`, `dataset.*`) from a Variables panel and Window › Variables, with dialogs to define, bind and edit, Make Text/Visibility/Image Dynamic, Capture/Update Data Set, CSV import/export (`dataset.import` / `dataset.export`), and a row in italics when an edit leaves the art not matching it; an image dataset value is a file path and apply keeps the object's bounds; bindings let go of deleted art and locked art is left alone; every command, like the rest, is drivable over MCP and the CLI. - - Variables: Import Data Sets… (`dataset.import`) reads tab-separated files as well as comma-separated ones, and files without a `name` column, whose rows take the first free names Data Set 1, Data Set 2 and so on. An import error about a row's values or name gives the file line the row starts on. + - Variables: Import Data Sets… (`dataset.import`) reads tab-separated files as well as comma-separated ones. Every header cell is a variable, and the rows replace the document's data sets as Data Set 1, Data Set 2 and so on. An import error about a row's values gives the file line the row starts on. - Variables: Import Data Sets… (`dataset.import`) defines the variables a file's header names that the document lacks, unbound. `#Shown` defines a visibility variable, `@Photo` defines an image variable, and a header without a prefix defines a text variable. A header that is a variable's exact name keeps naming it, and any other `%` header (graph data files) is refused. - Variables: Load Variable Library… in the Variables panel menu (`variable.loadLibrary`) replaces the document's variables and data sets with a variable library's, read from its XML file, in one undo step: every variable is deleted with its bindings and values, and every data set is deleted; then the library's text variables are defined, unbound, and its data sets take the text of each value's `p` elements, one paragraph each. Variables of other kinds are left out with a warning that names their kinds. - Variables: Save Variable Library… in the Variables panel menu (`variable.saveLibrary`) writes the document's text variables and data sets as a variable library's XML file, which Load Variable Library reads back with the same text variables, data set names and text values. Variables of other kinds are left out with a warning that names their kinds.