Architecture: a Python and Rust kernel with one seam per concern

The stack, top to bottom

Two backends behind one protocol

Every model call goes through a single Backend protocol, mirrored by a Rust trait in sultaan-core that acts as the versioned FFI contract. Two backends exist from day one: llama.cpp for quantised GGUF files and Candle for safetensors. The hello-world base model is Qwen2.5-0.5B-Instruct, and both backends generate text from it.

The Candle backend implements Qwen2 generation on CPU. CUDA and Metal feature flags exist in the crate but are not the default. A third backend would need its own decision record.

uv run sultaan generate --backend llamacpp "Hello, who are you?"
uv run sultaan generate --backend candle   "Hello, who are you?"

An agent loop written in-house

The agent is a small loop owned by the project: a system message, the user turn, tool calls and tool results, with tools registered as plain data classes. It deliberately uses no agent framework. Tools today are a Python REPL, retrieval over a document folder and a knowledge-graph lookup. A Docker sandbox with CPU, memory and time quotas is in the codebase for running untrusted code, but no command uses it by default yet; making it the default is planned session S2. A session-scoped circuit breaker stops runaway loops.

uv run sultaan agent --rag ./docs --verify "What does ADR 0017 decide?"
uv run sultaan kg add facts.db Paris capital_of France
uv run sultaan agent --kg facts.db "What is Paris the capital of?"

Retrieval, graph and memory

• Retrieval: hybrid search that combines BM25 keyword scores with dense embeddings, merges them with reciprocal rank fusion and packs context fairly across sources.
• Knowledge graph: a SQLite triple store in which every fact carries a trust tier.
• Memory: a wrapper around mem0 that runs on local storage and refuses known cloud hosts, plus typed facts and a parser for a SOUL.md identity file that shapes the system message.

Verification without a judge model

Answers can be checked by a model-free grounding verifier, which binds each claim to its best-supporting source, and by span and figure checks written with the standard library only. The evaluation page covers how the verifier is tested.

What is deliberately not here

• No server. The server package is a placeholder. If a model is ever served, the portfolio plan puts it behind the shared AI gateway as an upstream, with no API, sign-in or metering of its own.
• No home-grown deep-learning framework, tokenizer library, vector database or database. PyTorch, Candle, Hugging Face tokenizers, Qdrant and SQLite cover those.
• No hand-written CUDA or Metal kernels. Attention goes through PyTorch’s built-in fused kernels instead.
• No hosted model provider anywhere in the code path.

The decision log: 26 records on main

Every decision that outlives one change gets an architecture decision record (context, decision, consequences, alternatives). Accepted records are never rewritten; changes arrive as addenda or new records.