* xtask: delegate Unix group restoration in report test filesystem * craft-xlsx: xlsb tests unwrap the zip fixture's Result #1386 made the test zip writer return Result<Vec<u8>, Error>. The xlsb tests from #1263, #1323, #1330 and #1337 merged alongside it and still used the return value as bytes, so `cargo check --workspace --tests` failed on main with seven E0308 errors in crates/xlsx/tests/xlsb.rs. These call sites now unwrap the result, as #1386 did for the others. --------- Co-authored-by: dotCooCoo <RobertLeeLW@gmail.com>
craft-xtask
A library for the Craft apps' cargo xtask binaries. Each app keeps its own xtask binary, command
dispatch, step list, test targets and app-only commands, and calls this crate with its own policy.
The crate is native only and is a dependency of an app's xtask package only.
| Module | Feature | Contents |
|---|---|---|
runner |
always | Step, run_steps (steps in order, stop at the first failure, the ok, FAIL and skip summary), root, cargo, run, run_cancelable, nextest_or_test, doctest, Cancel |
assets |
always | Policy, RowMode, check, run, list_files, read_document, is_asset, ASSET_EXT: the asset attribution check against the document, folders and extensions the app supplies |
lock |
lock |
LockConfig, try_acquire, acquire, holders: 1 to 64 slots held across processes through file locks in a directory the app chooses |
report |
report |
write and to_json: the per-step JSON report (schema 1), replaced atomically through craft-atomic |
No feature is on by default. report adds serde_json and craft-atomic.
Use
use std::path::Path;
use craft_xtask::runner::{self, RunOptions, Step, TestMode};
fn ci() -> Result<(), String> {
let root = runner::root(Path::new(env!("CARGO_MANIFEST_DIR")))?;
let mut steps = vec![
Step::new("fmt", |log| {
let mut c = runner::cargo(&root);
c.args(["fmt", "--all", "--", "--check"]);
runner::run(c, log.command("cargo fmt --all -- --check"))
}),
Step::new("test", |log| {
let (c, label) = runner::nextest_or_test(&root, "xtask", TestMode::CargoTest, &["--lib", "--bins", "--tests"])?;
runner::run(c, log.command(&label))
}),
];
runner::run_steps(&mut steps, &RunOptions::default())?;
Ok(())
}
roottakes the app's ownCARGO_MANIFEST_DIRand returns its parent, which must hold aCargo.toml. It returns an error instead of panicking.nextest_or_testanddoctestbuildcargo test --workspace --exclude <xtask package>(orcargo nextest runwith the same arguments). The exclusion keeps the test build from rebuilding the runningxtaskexecutable, which Windows does not allow while it runs (storytold/craft-common-actions#185).TestMode::CargoTestis the default and probes nothing.TestMode::PreferNextestruns nextest whencargo nextest --versionsucceeds and otherwise runscargo testwith a note; the label names the command that runs.- Child processes inherit stdin, stdout and stderr. Their output is not decoded or stored.
- With
RunOptions::with_cancel, a setCancelflag stops the run before the next step, andrun_cancelableends a running child with the processes it started and waits for it. On Unix the child runs in its own process group, whichkill -s KILLends, so a terminal's Ctrl-C reaches only the xtask and the app's Ctrl-C handler sets the flag. On Windowstaskkill /T /Fends the tree.
Errors
runner::Error is Config (a root without Cargo.toml, an invalid package name, a slot count
outside 1 to 64), Step (the step and the error it returned), Canceled, Lock (the lock directory
or file could not be opened or locked) or Report (the run passed but the report could not be
written). A report that cannot be written after a failed run is printed to stderr, and the run keeps
its Step error.
Assets
The app owns its cargo xtask assets command, the printing and the gate step. The library lists the
repository's files, reads the attribution document and reports each asset without a row:
use craft_xtask::assets::{self, FolderExtensions, Policy};
const POLICY: Policy<'static> = Policy {
all_files_under: &["assets", "docs/images"],
ext_in_folders: &[FolderExtensions { folder: "examples", nested: true, extensions: &["csv", "json"] }],
..Policy::new("ATTRIBUTION.md")
};
fn assets(root: &std::path::Path) -> Result<(), String> {
let report = assets::run(root, &POLICY).map_err(|e| e.to_string())?;
if report.is_ok() {
println!("assets: all {} asset files attributed in ATTRIBUTION.md", report.assets);
return Ok(());
}
let lines: Vec<String> = report.violations.iter().map(ToString::to_string).collect();
Err(format!("{} asset rule violations:\n {}", lines.len(), lines.join("\n ")))
}
- A file is an asset when it is below a folder in
all_files_under, when its extension is inASSET_EXTorextra_ext, or when its extension is in anext_in_foldersentry and it is below that folder (at the repository root, or at any depth withnested: true).ASSET_EXTholds 69 extensions: the lists of the apps in the Origin table, andttcandotc. Extensions compare without regard to ASCII case; folders compare by whole path components and with case. attribution_fileis the document's repository path (ASSETS.md,ATTRIBUTION.md,assets/ATTRIBUTION.md). Only backticked spans in the first cell of a Markdown table row count. A table row is a line that starts with|, or a line of a table whose second line is a delimiter row (---|---), up to the next blank line. Lines in fenced code blocks and lines indented 4 columns or more are not read. A span opened by a run of backticks ends at the next run of the same length, so`` a`b.png ``namesa`b.png. A|inside a span or after\does not end a cell.RowMode::Exact(the default) needs a span that spells the path byte for byte.RowMode::Patternsreads*as any run of characters except/and a trailing/as every file below that folder. A file pattern costs its length times the path's length to match, however many*it holds. A folder pattern is matched against each folder prefix of the path, so its cost is its length times the sum of those prefixes' lengths.- An entry with
\, a leading/or drive letter, or an empty,.or..component is reported as aBadEntry. An unattributed asset whose path a row spells with different letter case is reported with that row. forbidden_extfiles are reported whatever their folder, and are not checked further.forbidden_name_fragmentsare matched against an asset's file name without regard to ASCII case. Withlicense_column: Some(n), every backticked span containing/in columnnmust name a regular file; neither the file nor a folder on the way to it may be a symbolic link.skip_foldersfiles are checked only againstforbidden_ext;Report::skipped_by_foldercounts them.list_filesrunsgit ls-files -z --cached --others --exclude-standard: tracked files, ignored or not, and untracked files that.gitignore,.git/info/excludeand the global exclude file do not ignore. It reads at mostmax_git_output_bytes(32 MiB by default) and ends the child when it writes more. A failure exit status, a name that is not UTF-8 and a name that a row cannot name (one that is not relative, or has a\, a drive prefix such asC:, or an empty,.or..component, which Git can list on Unix) are errors. A file listed more than once (an unmerged file) is checked once. Names that no longer exist are left out. Symbolic links, submodule folders, other entries that are not regular files, and files reached through a linked folder are not followed or checked;Report::skipped_entriescounts them.read_documentfails before reading a document larger thanmax_document_bytes(8 MiB by default), and fails on a document that is not UTF-8.checktakes the file list and the document text, and a function that answers whether a license file exists, so an app's tests run without Git or a file system.ErrorisPolicy,Git,GitName,DocumentorMetadata.
The source copies differ from this module in a few rules. An app that switches lists the results that change, and the files that the 69 extensions newly select:
- Every source copy but LightCraft accepts a backticked path anywhere in the document, in prose or in any column. DeckCraft also takes every backticked span in the document as a pattern. No source copy skips code blocks or reads multi-backtick spans.
- No source copy reports bad entries or case-only near rows, or checks an unmerged file once. Each one follows symbolic links to the file and counts it as an asset, and none has a size limit on the document.
- VectorCraft (
repo_filesinxtask/src/main.rs), DesignCraft, CADCraft, GridCraft and LightCraft list files without-z, so Git quotes a name with a non-ASCII letter, a", a\or a control character such as a newline, and the check passes over it. DesignCraft, CADCraft and DeckCraft do not check Git's exit status. - DeckCraft and WordCraft replace bytes of a name that is not UTF-8; this module reports an error.
- DeckCraft's patterns have no trailing-
/folder form. LightCraft's checker skips table rows with fewer than five cells and first-cell spans without a/.
Lock
RunOptions::with_lock(LockConfig::new(dir, slots, owner_label)) takes one of slots lock files in
dir before the first step and releases it when run_steps returns. The OS releases a lock when its
process ends. A slot count of 0 or more than 64, or a poll interval (with_poll, 2 s by default) of zero, is an
error before any file is opened. Next to each
held lock file, an owner file holds the owner label, which waiting runs print; holders reads at
most 4096 bytes of each and leaves out an owner file whose slot is free, such as one a killed run left
behind. Where the lock files live, the environment variables that set them, and
whether a run uses a lock at all are the app's choice.
Report
RunOptions::with_report(path) writes the report after the run, passed or not, and before the run
releases its lock slot. A run with a lock that does not take it (a configuration or lock error, or a
cancel while waiting) runs no step and writes no report. The report looks like this:
{
"schema": 1,
"steps": [
{ "step": "fmt", "command": "cargo fmt --all -- --check", "status": "ok", "duration_ms": 812 },
{ "step": "test", "command": null, "status": "fail", "duration_ms": 4031 },
{ "step": "wasm", "command": null, "status": "skip", "duration_ms": null }
]
}
status is ok, fail, canceled or skip. The file is staged beside path and renamed over it,
so a reader never sees a partial report, and a failed write leaves the previous report in place.
Machine, GPU, corpus and skip details stay in the app's own report.
Origin
| Part | Source |
|---|---|
runner: the step loop and summary, root, cargo, run, the nextest probe and fallback, doctests |
DesignCraft xtask/src/main.rs:68-82, :160-201, :204-249 at 8b169f8 |
lock and its tests one_slot_is_exclusive_until_dropped, more_slots_admit_more_runs |
LightCraft xtask/src/ci_lock.rs at 25ffe3e |
report: the per-step fields |
LightCraft xtask/src/ci.rs:347-414 at 25ffe3e |
The root and slot-count errors, the --exclude of the xtask package, cancellation, the bounded owner read and the atomic report |
written here |
assets: is_asset, the exact-row check, run, the three asset folders and the test detects_unattributed_assets |
VectorCraft xtask/src/assets.rs at bada8e1 and DesignCraft xtask/src/assets.rs at 8b169f8 |
assets: the CADCraft and WordCraft extension lists, the -z listing, the exit-status check and the tests in assets::tests::wordcraft |
CADCraft xtask/src/assets.rs at 8e69890, WordCraft xtask/src/assets.rs at 42a5df5 |
assets: the DeckCraft extension list, the pattern table and the tests in assets::tests::deckcraft |
DeckCraft xtask/src/assets.rs at b0c77eb |
assets: folder-only extensions and the tests in assets::tests::gridcraft |
GridCraft xtask/src/assets.rs at f204433 |
assets: first-cell rows, folder patterns, forbidden extensions and names, the license column and the tests in assets::tests::lightcraft |
LightCraft xtask/src/assets.rs at 25ffe3e |
assets: the policy checks, the cell parser, bad-entry and near-row reports, the size limits, UTF-8 names, skipped folders and the regular-file test |
written here |
Migration
No app uses this crate yet.
| App | Status |
|---|---|
| DesignCraft | Planned: cmd_ci, root, cargo, run, cmd_test and cmd_doctest move to runner, with TestMode::PreferNextest. The step list, TEST_TARGETS and the i18n report stay in the app. |
| LightCraft | Planned: ci_lock.rs moves to lock, with LIGHTCRAFT_CI_LOCK and LIGHTCRAFT_CI_SLOTS read by the app. The machine, GPU, corpus and skip report stays in the app. |
| VectorCraft, DesignCraft, CADCraft, DeckCraft, WordCraft, GridCraft, LightCraft | Planned: xtask/src/assets.rs moves to assets, each with its own document, folders and row mode; LightCraft with RowMode::Patterns, its forbidden extensions and names, and license_column: Some(4). The cargo xtask assets output stays in the app. |
Maintainers: the craft-libs maintainers, until a responsible maintainer or team is agreed in the review of this crate's first pull request.