craft-ai — AI providers for the Craft suite
Shared AI abstraction for every Craft app: translation, rewriting, summaries, vision captions — one crate, one settings store, one model cache.
Status: spec — this file is the contract; implementation follows.
The feature this powers
A per-app AI tab offering document-wide operations — the flagship being "translate this whole document" — backed by either:
- BYOT — the user's own API token against an OpenAI-compatible endpoint (OpenAI, Azure, Ollama, LM Studio, any compatible proxy), or
- Local model — a one-click download of a curated open model, chosen for the machine it's running on.
The user experience is deliberately informed: every step reports what is happening, why this model was picked, what it will cost in disk/RAM, and that nothing leaves the machine in local mode.
Modules
probe — capability detection
Runs once, caches the result:
| Signal | Source |
|---|---|
| Total/free RAM | sysinfo or std::alloc-safe OS calls |
| CPU threads + features | std::thread, is_x86_feature_detected! (AVX2 gates Q4 speed) |
| GPU + VRAM | wgpu adapter info (apps already link wgpu — reuse, don't re-probe) |
| Free disk at the model cache dir | fs2/std::fs |
| Network reachability | optional HEAD to the model host, skipped offline |
Output: MachineCaps { ram_gb, vram_gb, threads, avx2, disk_free_gb }.
registry — the curated model list
A small hand-maintained table (Rust consts, not runtime-fetched — supply
chain stays auditable). Each entry: id, display name, HF repo + file,
sha256, size, min RAM, recommended RAM, license id, quality tier,
languages covered.
Curation rule — permissive licenses only (the suite is MIT OR Apache-2.0 and commercial users run it):
- ✅ Qwen2.5-0.5B / 1.5B / 3B Instruct (Apache-2.0)
- ✅ SmolLM2-1.7B (Apache-2.0)
- ✅ Gemma-3 (Gemma Terms — allowed, flag in UI)
- ✅ Helsinki-NLP opus-mt-* per-language pairs (CC-BY/MIT)
- ❌ NLLB-200, M2M-100 — CC-BY-NC, excluded
- ❌ Llama-* — custom license acceptable? default: exclude until reviewed
registry::suggest(&caps) picks the best fitting model: prefers 3B when
VRAM ≥ 6 GB or RAM ≥ 16 GB, falls to 1.5B at 8 GB, 0.5B below. Always shows
alternatives — the suggestion is a default, not a cage.
provider — inference backends
trait Provider {
fn name(&self) -> &str;
fn complete(&self, prompt: &str, opts: &GenOpts) -> Result<String, AiError>;
fn test(&self) -> Result<ProviderInfo, AiError>;
}
OpenAiCompat— POST/v1/chat/completions, configurable endpoint+token+ model. Covers OpenAI, Ollama (http://localhost:11434/v1), LM Studio, Groq, OpenRouter… Token stored via the OS keyring (keyringcrate) or a 0600-permissioned config file — never in project files, never logged.LocalGguf— GGUF inference. First implementation target: candle (pure Rust, Apache/MIT — keeps the no-C++ toolchain rule).llama-cpp-2stays a possible perf upgrade later, gated behind a feature flag.
fetch — one-click download
- Resumable HTTP GET (range requests), streamed to
*.partthen renamed — a half-downloaded GGUF can never be mistaken for a valid one. - SHA-256 checked against the registry entry; mismatch = hard fail, delete.
- Progress events (
Download { bytes_done, bytes_total, bytes_per_sec }) feed the UI's live status line. - Cache dir is shared across the suite:
%LOCALAPPDATA%\artcraft\models(Windows),~/Library/Application Support/artcraft/models(macOS),~/.cache/artcraft/models(Linux).
translate — document segmentation
The model is the easy part; the hard part is what not to touch. The translator walks the document as segments, not raw text:
- Protected: LaTeX commands/envs/math (
\cite{…},$…$), code fences, placeholders ({count}), URLs, citation keys, markup tags. - Translated: prose runs, captions, headings, table cells.
- Per-app adapters declare the grammar: TexCraft protects
\command{…}and math mode; WordCraft protects field codes; MatCraft protects code. - Batch-friendly: segments → prompt with numbered items → parse numbered output; verify count matches, retry split halves on mismatch.
- Deterministic options: temperature 0-ish, glossary hook (
HashMap<term, translation>) applied post-hoc.
The AI tab (per app, ui-egui/src/ai.rs)
First-open flow — the "smooth informed" sequence:
- Two cards: Use an API provider / Run on this PC.
- BYOT card: endpoint, token, model fields + Test connection → green check with model list echo.
- Local card: Detect this PC → probe runs →
Suggested: Qwen2.5-1.5B-Instruct (Q4_K_M, 1.1 GB) — fits your 16 GB RAM; est. ~8 tok/swith an "Other models ▸" disclosure showing the registry. - Download → progress bar with MB/%/speed →
Verifying checksum…→Loading…→Ready — local, offline, nothing leaves this PC. - Then the operations: "Translate document to [language ▾]", "Summarize selection", custom prompt box. Status line narrates each step; expandable activity log for the curious.
- Language detected vs document language → gentle offer chip:
This document is French — translate to English?
Non-negotiables
- Honesty: the UI always says which provider is active, what model, and "cloud — your document text is sent to the endpoint" vs "local — nothing leaves this PC". No silent cloud calls; AI is strictly opt-in.
- No crash: model load failures, corrupt GGUFs, OOM — all
Result, all surfaced as messages. - Rust only: candle over llama.cpp bindings preserves the wasm story
(local inference on wasm is future-possible with
candle+ WGPU backend). - Licensing: registry carries SPDX ids; the UI shows them; CC-BY-NC and unreviewed custom licenses are excluded by policy.
- Everything is a command:
ai.translate_doc,ai.status,ai.models.list,ai.model.download,ai.provider.test— reachable from the app's control channel and MCP, same as every Craft convention.
Milestones
| # | Deliverable | Done when |
|---|---|---|
| A1 | probe + registry + fetch |
CLI-free test: probe reports real caps; suggest picks sanely per fixture; a tiny model downloads + sha256 verifies |
| A2 | OpenAiCompat provider |
translate a string through Ollama/OpenAI with mocked HTTP tests |
| A3 | LocalGguf on candle |
Qwen2.5-0.5B loads and answers on CPU in a test (model fixture fetched in-test, skipped without network) |
| A4 | translate segmentation |
LaTeX/code/placeholder round-trip tests — protected spans byte-identical |
| A5 | AI tab in one pilot app (suggest TexCraft or WordCraft) | download → translate document end-to-end, live status UX |
| A6 | Commands + MCP surface | ai.* commands reachable via control channel |