Commit Graph
3048 Commits
Author SHA1 Message Date
Dylan M. Taylor 1b7e4f00dc Merge remote-tracking branch 'origin/switch-craft-libs-xls' into codex/reviewfix-3841 2026-11-12 05:40:59 +00:00
Dylan M. Taylor 4deedbb549 warn about dropped XLS rules 2026-11-12 05:40:53 +00:00
Dylan M. Taylor 26e7112e0f Merge remote-tracking branch 'origin/switch-craft-libs-xls' into codex/propagate-3840 2026-11-12 05:37:12 +00:00
Dylan M. Taylor 3d31729de3 test(xls): expect malformed encryption error 2026-11-12 05:36:34 +00:00
Dylan M. Taylor aeb3fb19ce Merge latest CSV branch updates 2026-11-12 05:13:54 +00:00
Dylan M. Taylor e4ec0d7ed1 GridCraft: update CSV switch to current shared crates 2026-11-12 05:13:10 +00:00
Dylan M. Taylor 23487b0cb5 Merge current XLS switch into CSV switch 2026-11-12 05:12:51 +00:00
Dylan M. Taylor 0407b63663 GridCraft: update XLS import to current shared crate 2026-11-12 05:11:53 +00:00
Dylan M. Taylor 1bd4ca8946 Test xls import with a valid compound file header 2026-11-12 05:05:01 +00:00
Dylan M. Taylor 0431741359 tests: give the xls fixture its compound-file byte order mark
craft-cfb refuses a header without the 0xFFFE mark at offset 28
([MS-CFB] 2.2), so craft-xls's sniff called the fixture LegacyBinary.
The old hand-written parser never read that field, which is why the test
only broke with the switch. The fixture is a real workbook otherwise:
sniff now says Xls and read_xls parses it.
2026-11-12 04:37:07 +00:00
Dylan M. Taylor c1adf9b239 xlsx, engine: CSV decoding, parsing and quoting, and SYLK, through craft-csv
Text decoding (byte order marks, UTF-8, Windows-1256/1252), delimiter sniffing,
the record parser and field quoting moved to storytold/craft-libs as craft-csv
(#889), with the SYLK reader as craft_csv::sylk. GridCraft keeps typed values,
date-order inference, the used-range limit and the sheet writer in csv.rs, and in
io/sylk.rs the workbook a SYLK file becomes: formulas parsed (an unparsable one
is skipped and counted, as before), error values, number-format styles and column
widths.
2026-11-12 01:17:21 +00:00
Dylan M. Taylor 21df49c636 xlsx: read .xls workbooks through craft-xls
The BIFF8 reader moved to storytold/craft-libs as craft-xls (#878). GridCraft
keeps the mapping onto its model in xls.rs: formulas parsed, Excel's implicit @
for legacy formulas, pixel widths and heights, styles interned per XF, sheet-name
checks, print areas and titles, calculation settings and GridCraft's warning text.

The workbook now opens on the sheet it was saved on when that sheet is visible,
else on the first visible sheet (from #3551 by @Morzilla, whose test is kept).
RString cells are read as text.

Every craft-libs crate moves to one revision, 119415f4.
2026-11-12 00:58:43 +00:00
FredApps 7e0d96d74b CI: nightly release dry run (#3820)
release.yml also runs nightly on main with craft-common-actions' dry-run: every release build
except macOS, with nothing signed, pushed or drafted. Breakage that only the release build hits
(riscv64, Flatpak, FreeBSD, packaging) then shows up the next morning instead of on release
day, as it did on 11 Nov 2026.
2026-11-11 23:04:17 +01:00
Dylan M. Taylor 572bec289b engine: work limits for Advanced Filter, Remove Duplicates, filters, AutoFit, macro footprints; range sizes saturate (#3295)
* engine: bound Advanced Filter, Remove Duplicates, AutoFilter, Circle Invalid Data; one-pass AutoFit; saturating range dimensions

* address review: AutoFit scans each run of adjacent selected columns separately

* address review: report a truncated circle-invalid scan, no spill scan for plain cells in Replace, one pass for many AutoFit runs

* address review: warn in the app when Circle Invalid Data stopped early; Replace test requires success

* address review: the scan note joins an open-time notice; say cell visits
2026-11-11 16:10:10 -05:00
Dylan M. Taylor 55eb394044 engine: index merges and bound the page in HTML export and clipboard HTML (#3266)
* engine: index merges and bound the page in HTML export and clipboard HTML

Save as Web Page and the clipboard's HTML scanned every merge of the
sheet for every cell, counted a merge's span over all the rows the file
gave it, placed each picture by summing every custom row before it, and
grew the page without a byte limit. Merges are now indexed once over
the exported rectangle (overlapping merges past a few visits per cell
are refused), spans use prefix counts clipped to the table, anchors use
LineIndex, and the page stops at 256 MiB with an error.

* address review: check the web page size after each chart and before returning it

* address review: bound the merge index, escape cell text within the page's room, size images before encoding

* address review: escape image attributes within the page's room
2026-11-11 16:09:43 -05:00
Dylan M. Taylor 9e9f13114f check, xtask stats: visited directories keyed by (dev, inode); stats reads only regular files (#3287)
* Directory walkers key visited directories by (dev, inode); stats skips non-regular or oversized .rs targets

* address review: record a walked directory only within the depth limit; stats skips symlinked sources and bounds the read

* address review: rescan a directory reached from a shallower path, skipping files already collected

* xtask stats: test the directory walk (links, identity, size limit)

* address review: create the test directories atomically and privately

* xtask stats tests: no unreachable! in the directory helper

* gridcraft check: unit-test that each workbook is listed once whatever path reaches it
2026-11-11 16:09:17 -05:00
Dylan M. Taylor 35e9815603 Security: one work budget across every call of a lifted function (#3197)
* functions: one work budget across every call of a lifted function

A function lifted over a computed array runs once per element, and only the
six lookups counted their repeated work against LIFTED_SCAN_BUDGET. Criteria
aggregates rescanned their ranges per criterion, CUMIPMT/CUMPRINC stepped
through up to 1e7 periods per element, and array-producing functions such as
MUNIT built a whole result per element before keeping its first cell.

The budget now covers all of it: COUNTIF/SUMIF/AVERAGEIF(S)/MAXIFS/MINIFS
charge their range cells per call, the period loops of every element are
added up before any runs, and array results charge their cells as they are
built. Past the budget the call is #NUM!, as lifted lookups already are.

* address review: charge numeric-text periods of lifted CUMIPMT/CUMPRINC to the work budget

* address review: charge only calls that reach the cumulative period loop

* address review: charge the numeric text a lifted period call parses

* address review: SUMIF/AVERAGEIF charge only the sum-range cells they read; weight cumulative period work
2026-11-11 16:08:50 -05:00
Dylan M. Taylor 6ea8d11522 print: bound the print area to the grid; render charts once and budget picture and chart output (#3314)
* print: hold the print area to the grid; render each chart once and budget picture and chart output across pages

* address review: budget chart path points and drop charts that do not fit the budget
2026-11-11 16:08:24 -05:00
Dylan M. Taylor 7c3af8a421 fix(formula): bound the wrappers stacked on one path of a parsed formula (#3813) 2026-11-11 16:07:58 -05:00
Dylan M. Taylor ca7ea0131e fix(mcp): send the control token only to a loopback address (#3814) 2026-11-11 16:07:32 -05:00
Dylan M. Taylor aa0347cc2e fix(engine): calculated pivot fields may not build arrays from scalars (#3812) 2026-11-11 16:07:06 -05:00
Dylan M. Taylor 5cf52c5fbb chore: bump version to 0.19.1 for the Flatpak update fix (#3809) 2026-11-11 14:55:53 -05:00
Dylan M. Taylor 0c5b1e85e6 sonar: stop counting the locale table as duplicated code (#3805) 2026-11-11 14:33:16 -05:00
Dylan M. Taylor 7f6a79fd98 Fix Flatpak permissions for update checks and self-updates (#3803)
* fix(flatpak): allow release checks and self-update helpers

* test: keep Flatpak permission parsing within strict Clippy rules
2026-11-11 14:24:13 -05:00
Dylan M. Taylor 6dbfbc4058 Merge pull request #3795 from storytold/chore/craft-update-6e03f65
Updates: take craft-update from craft-libs main at 6e03f65bf
2026-11-11 12:44:31 -05:00
Dylan M. Taylor e258db5f19 Updates: take craft-update from craft-libs main at 6e03f65bf 2026-11-11 12:44:25 -05:00
Dylan M. Taylor 763816d27c ci: run Harden-Runner on every job (#3786)
Adds step-security/harden-runner (audit, pinned to v2.22.4 like
craft-common-actions) as the first step of each job that did not have it,
so every workflow in the org's public repos reports its egress to
StepSecurity. Audit, not block: block-mode allow lists come from each
job's observed traffic.
2026-11-11 12:11:59 -05:00
FredApps 90180f4d87 Sandbox: build on riscv64, where libc names no kexec_file_load (#3777)
The seccomp list named libc::SYS_kexec_file_load, which libc (0.2.190) defines for x86_64 and
aarch64 but not riscv64, so the release's Linux riscv64 build stopped with E0425. riscv64 takes
the number from the kernel's generic table, 294, as aarch64 does; the test checks libc's aarch64
value against it.
2026-11-11 17:35:44 +01:00
FredApps d69a285efb Release: GridCraft v0.19.0 (#3770) 2026-11-11 16:38:52 +01:00
Dylan M. Taylor 94fc52efae xlsx: share inline dxf number formats within a styles part (#3260)
* xlsx: share each custom number format across cell formats, cap font names at 31 characters

Every cell format record copied its font (name included) and built its
own copy of its number-format code, so many short records naming one
long font or code multiplied it in memory. Custom codes are now built
once and shared, and font names are cut to Excel's 31 characters.

* xlsx: drop stray worktree entries added by mistake

* address review: compare truncated theme font for the scheme link; intern inline dxf number formats
2026-11-11 08:50:20 -05:00
Dylan M. Taylor 7b825d00be Save as Web Page: a byte budget for the charts' SVG (an error past it) (#3209)
* Charts: one point budget per chart and a byte budget for web-page SVG

A chart resolved every series reference into its own vector of up to
100,000 points, so 255 short whole-column series retained and drew
25.5 million points, and Save as Web Page wrote one SVG element per
mark without any output limit.

- resolve() shares MAX_CHART_POINTS (1,000,000) across the values,
  X values and bubble sizes of all series; a series past the budget
  gets what is left.
- chart_svg() stops writing once a chart passes the bytes left of a
  64 MiB per-page budget and leaves that chart out.

* address review: charge X values to the chart point budget only when they are kept

* address review: charge scatter Y and X values to the chart point budget as a pair
2026-11-11 08:39:25 -05:00
Dylan M. Taylor 7726e9c6ef xls: charge expanded shared-formula text to the import's text allowance (#3256)
* xls: charge expanded shared-formula text to the import's text allowance

Shared-formula followers each kept their own printed formula text, but
none of it counted against the 256 MB text allowance, so compact PtgExp
records could expand into gigabytes of retained formulas. Each attach
now charges its text; past the allowance the formula is dropped and the
cell keeps its cached value.

* address review: keep resolving array formulas after the shared-formula text allowance is spent

* address review: keep the shared-formula text cut-off across worksheets

* address review: expand shared formulas only after every sheet's mandatory text is charged

* address review: refuse duplicate sheet names at import; adapt tests to read_xls_budgeted
2026-11-11 08:28:28 -05:00
Dylan M. Taylor 9a025205bd pivot: Rank and Running Total computed once per sibling set; accumulator, caption and chart-series bounds (#3306)
* pivot: cached Rank and Running Total, accumulator-width limit, capped subtotal captions and PivotChart series

* address review: store running-total entries only for items with a number

* address review: running totals visit only the top-level items that hold the path
2026-11-11 08:16:57 -05:00
Dylan M. Taylor 01abd95420 xlsx: resolve cell pictures through a per-structure layout and a relationship index (#3255)
* xlsx: resolve cell pictures through a per-structure layout and a relationship index

Every rich value rebuilt its structure's whole key list before checking
its value count, and scanned the image relationship table for a
duplicate id. Structures are now laid out once and relationships are
indexed by id (a repeated id stays invalid), so reading cell-picture
metadata is linear in its records.

* address review: wide-structure fixture uses the right count and loads a valid picture
2026-11-11 08:02:00 -05:00
Dylan M. Taylor e5ac9474df engine: Scenario Summary size limit, bounded solver report names, undo history counts comment and link text (#3308)
* engine: bound Scenario Summary size and solver report name searches; charge comment and link text to undo history

* address review: count result rows in the scenario summary limit and reply/date vector storage in the history estimate
2026-11-11 07:40:55 -05:00
Dylan M. Taylor b21a8541e7 Row auto-fit: sweep merges once per sheet instead of per row (#3195)
* Row auto-fit: sweep merges once per sheet instead of per row

auto_row_height() picked the merges touching a row by scanning every
merge on the sheet, and opening a workbook (fit_imported_rows) or a
commit fitting many rows called it per row, so rows x merges
comparisons ran synchronously on open.

Rows are fitted in ascending order, so a MergeSweep sorts the sheet's
merges by first row once and keeps only those active on the current
row. All comparisons count against MERGE_FIT_WORK (50,000,000) per
pass; past it the remaining rows keep the heights they have.
auto_row_height keeps its signature for single rows.

* address review: share the merge-comparison budget across the whole auto-fit pass

* address review: build the merge sweep lazily, after the row fast paths

* address review: charge the merge sort to the fit budget, keep row heights once it is spent

* address review: a pass that fits one row scans the merges instead of sorting them
2026-11-11 07:26:18 -05:00
Dylan M. Taylor a6c331b63f calc: one cumulative work budget per formula, and INDEX decides multi-cell picks from their shape (#3188)
* calc: INDEX decides a multi-cell position from its shape before reading it

* calc: charge each formula's evaluation against one cumulative work budget

* address review: a database function whose computed criterion ran out of work is #CALC!

* address review: give the test's criteria cell its stored value

* address review: stop matching database records once a computed criterion exhausts the work budget

* address review: skip the criteria-row scan once work is exhausted

* address review: stop between conditions once the work budget is spent
2026-11-11 07:17:04 -05:00
Dylan M. Taylor 12d26d53cc xlsx: analyse each kept chart sheet's graph once per save (#3261)
* xlsx: analyse each kept chart sheet's graph once per save

chart_sheet_usable walked a chart sheet's relationship graph and parsed
its charts for every workbook entry, so many entries naming one shared
graph multiplied the work. The referenced sheet names are now cached by
relationship id.

* address review: cache the chart-sheet usability verdict per relationship id instead of every reference

* address review: cache chart-sheet analysis by the graph root, not the relationship id
2026-11-11 07:06:18 -05:00
Dylan M. Taylor 9dc7e3826a numfmt: TEXT returns #VALUE! when its result passes a cell's 32,767 characters (#3264)
* numfmt: TEXT returns #VALUE! when its result passes a cell's 32,767 characters

A text section repeats its input once per @ placeholder, and TEXT handed
back whatever that built, so nested TEXT calls could multiply a long
text until memory ran out. Results past the cell text limit are now
#VALUE!.

* address review: stop TEXT rendering past the cell limit and count UTF-16 units
2026-11-11 06:44:34 -05:00
Dylan M. Taylor f79a425ca9 xlsx: find a reused picture through a content hash instead of comparing every media part (#3214)
* xlsx: find a reused picture through a content hash instead of comparing every media part

Out::image_part compared each new picture byte by byte with every
media part written so far, so a workbook of many distinct pictures
sharing a long prefix was exported in quadratic time. Media parts are
now indexed by extension and SHA-256.

* address review: index preserved media parts when they are added

* address review: do not hash an image twice when it is first added
2026-11-11 06:32:44 -05:00
Dylan M. Taylor 76e53085e3 xlsx: keep the cached value of a follower whose shared-formula copy is refused (#3212)
* xlsx: one hyperlink allowance per sheet, and a copy limit for unparsable shared masters

Every <hyperlinks> element of a worksheet started a fresh cell and byte
allowance although all of them fill the same map. The sheet now keeps
one allowance across its elements.

Followers of a shared master the parser refuses each copied its full
text; a sheet full of followers of a long master multiplied that text
without bound. The copies are now charged against a 64 MiB per-sheet
limit, past which the followers keep their cached values.

* address review: keep the cached value of a follower dropped for the shared-formula copy limit inside array ranges
2026-11-11 06:15:08 -05:00
Dylan M. Taylor bc2dabc7ea FORMULATEXT and the in-cell editor keep Hidden formulas hidden (#3298)
* FORMULATEXT and the cell editor keep a hidden formula's source hidden

* address review: recalculate formulas when sheet protection changes

* address review: skip the full recalculation for a no-op unprotect

* address review: recalculate on protect/unprotect without marking the edit structural
2026-11-11 05:48:31 -05:00
Dylan M. Taylor e14f783503 Analysis ToolPak: stop building a report once it outgrows the output limit (#3207)
* Analysis ToolPak: stop building a report once it outgrows the output limit

Report builders filled their whole output map before write_report
checked its size, so Descriptive Statistics grouped by rows over a
tall, nearly empty column allocated about 16 entries for each of up
to a million variables before refusing the report.

Grid now tracks its extent as values are stored and keeps nothing
once the report's rows x columns pass MAX_OUTPUT_CELLS; write_report
refuses such a report. extent() no longer rescans every entry.

* address review: stop the descriptive builder once the grid is too large

* address review: descriptive statistics reads variables one at a time
2026-11-11 05:40:31 -05:00
Dylan M. Taylor 542aa176ce Wildcards: match * and ? patterns in one pass over the text (#3205)
* Wildcards: match * and ? patterns in one pass over the text

After a mismatch the matcher restarted the pattern after the last
star at the next text position, so a long repetitive cell against a
near-matching pattern cost text x pattern comparisons. AutoFilter
custom criteria, Find and Replace all call it once per cell.

wildcard() now runs the pattern as a bit-parallel automaton: one pass
over the text, one step per 64 pattern positions for each character.
Results are unchanged; a test compares it with the old matcher on
every short text and pattern over a small alphabet.

* address review: build wildcard literal masks lazily with a bounded cache

* address review: keep masks of the most frequent wildcard literals past the cache cap
2026-11-11 05:19:24 -05:00
Dylan M. Taylor 67a0b7f325 Security: chart rasterising, spline points and category labels stay bounded (#3204)
* chart: bound scan conversion, spline points and category label bytes

- The rasteriser tested every edge of a path at every sub-sample of every
  covered row. It now keeps an active-edge list (edges sorted by top, joining
  and leaving as the scan passes them), and one rasterize call has a total
  scan-work allowance after which remaining fills are skipped.
- Smoothed lines budgeted interpolated points per run, so gap-separated short
  runs escaped both decimation and the cap. The budget is now shared by all
  runs of a line; once spent, runs are drawn through their data points.
- Category labels copied every text value in full. Labels are clipped to 255
  characters before copying, within an 8 MiB budget per series.

* address review: chart raster and label budget fixes

* address review: bound polyline point count before stroke_paths
2026-11-11 05:07:04 -05:00
Dylan M. Taylor bbe0825404 Security: TEXTJOIN builds its result piece by piece within the text cap (#3202)
* functions: TEXTJOIN builds its result piece by piece within the text cap

TEXTJOIN converted every input and delimiter into an owned String before
joining, so many references to one long shared text copied gigabytes before
the joined length was ever checked. Errors are now found first without
building text (so they still win in argument order), then each piece and
delimiter is converted only as it is appended, stopping at the cap.

* address review: check the cap before copying each text piece
2026-11-11 04:55:21 -05:00
Dylan M. Taylor aa4951fdf4 Security: WORKDAY, PIVOTBY, SORT and the D-functions stop rescanning their input (#3200)
* functions: stop WORKDAY, PIVOTBY, SORT and the D-functions rescanning their input

- WORKDAY/WORKDAY.INTL counted the holidays each step passed by scanning every
  holiday, once per step; a run of consecutive holidays made that quadratic.
  The working-day holidays are kept sorted and counted by binary search.
- PIVOTBY totals rescanned the whole source (rebuilding keys) once per total
  row and column. The grouping pass now also indexes rows by row key and by
  column key, and totals read from those indexes in the same order.
- SORT copied one key per sort_index entry for every row, so a long
  sort_index with repeated columns multiplied the source. Repeats can't break
  a tie their first use left, so only first uses are kept.
- DSUM and the other database functions tested every record against every
  criteria row. A selection whose records x conditions exceed 5e8 is #NUM!,
  and each criteria label's database column is found once, not per cell.

* address review: bounds-check SORT indices before narrowing; empty criteria rows are not charged
2026-11-11 04:36:48 -05:00
Dylan M. Taylor 79fe7b8eb6 Forecast Sheet: count hindcast rows against the work limit (#3199)
* Forecast Sheet: count hindcast rows against the work limit

MAX_WORK bounded history rows x forecast rows, but a Forecast Start
inside the history adds a hindcast row for every history point after
it, each holding the same ETS formulas that refit the model over the
history before the start. Those rows were not counted, so a start in
the middle of 10,000 rows of history built millions of uncharged
model fits per recalculation.

The forecast row cap is now computed from the rows the model is fit
to, after subtracting the hindcast rows; a start so early that the
hindcast alone passes the limit is refused, and the default number of
periods stays within the cap.

* address review: base the in-history forecast cap on the slots the fit range spans

* address review: document the hindcast-aware forecast work limit

* address review: document the fitted-row minimum in the forecast work limit

* address review: say the forecast start row counts toward the hindcast work limit
2026-11-11 04:22:54 -05:00
Dylan M. Taylor ff9f4b8ace Find and Replace: bound the text a replacement builds (#3194)
* Find and Replace: bound the text a replacement builds

Each occurrence copied the whole replacement string, so a cell full of
short matches and a long replacement expanded tens of kilobytes of
input into about a gigabyte, in both the case-sensitive and the
case-insensitive paths.

Replacement now builds the new text only while it stays within
Excel's 32,767 characters per cell (counted as UTF-8 bytes, 4 each at
most) and within 256 MiB for the whole command, checking before each
copy. Past either limit the command fails and changes nothing.

* address review: enforce the 32,767-character limit on replacements and borrow the replacement text
2026-11-11 04:02:22 -05:00
Dylan M. Taylor 5ce63e5fc2 Get Data: refuse an import whose dense rows would be too large (#3193)
* Get Data: refuse an import whose dense rows would be too large

data.getData and data.fromTextCsv built every row densely before
writing it: JSON records were padded to the union of all keys (found
with a quadratic Vec::contains scan), and CSV cells were expanded over
the whole used rectangle. A few sparse records or one far-away CSV
cell therefore allocated billions of slots.

The importer now collects JSON keys with a HashSet, refuses more keys
than a sheet has columns, and checks the projected rows x columns
against MAX_IMPORT_CELLS (10,000,000) before building any row.

* address review: stop collecting JSON keys at the column limit and reject oversized row counts before building rows

* address review: enforce the CSV import cell limit while parsing
2026-11-11 03:48:32 -05:00