SULTAANresearch notebook · .io

Vol. 1Phase 3: pretraining infrastructureRepository private

Roadmap: where Sultaan is, and what comes next

The repository’s roadmap, not its long-form vision document, decides what is in scope. Work moves in phases with written exit criteria, and later-phase work is refused until the phase gate is formally lifted.

§ 4.1phases

Five phases

  1. Done Phase 0: Foundation

    Workspace, the Rust backend trait, both inference backends, hello-world generation, a perplexity baseline, pre-commit hooks, CI workflows and DVC. Closed with ADR 0008.

  2. Ratified Phase 1: Fine-tune, evaluate, minimal agent

    The agent loop with its REPL tool, the four-metric evaluation suite and experiment tracking are done. Carried over: a QLoRA adapter that beats its base on at least one benchmark. Ratified with that carry-over in ADR 0015.

  3. Code-complete Phase 2: Memory, retrieval, sandbox

    mem0 memory, hybrid retrieval, the SOUL.md parser, the Docker sandbox, the SQLite knowledge graph and the grounding verifier are built and unit-tested. Two items still need a running host to demonstrate end to end.

  4. Current Phase 3: Pretraining infrastructure

    Model, data pipeline, trainer, configuration, command line and cloud scaffolding are built, and the trainer is proven on CPU. A multi-GPU launcher exists but is not yet safe: data is not sharded by rank, so multi-process runs are to be refused until that is fixed. Still open: a T1-or-larger run that completes reasonable English, measured as perplexity below 30.

  5. Not started Phase 4: Production hardening and interface

    A desktop app, voice, distillation into a smaller model, signed multi-architecture images, the custom licence, and edge deployment guides. Gated until Phase 3 exits.

§ 4.2planned 1 Oct 2026

The next ten engineering sessions

A portfolio review on 1 October 2026 planned the following sessions in order. None has started yet; each will be logged on the follow page when it lands.

  1. Baseline and branch review. Record test counts and smoke-test losses, and review the unmerged branch commit by commit.
  2. Agent tool safety. Run model-written code in the sandbox by default, with an allow-listed environment, and tighten the memory guard to local and private hosts only.
  3. CI that runs everywhere. Builds on Linux, Windows and macOS without machine-specific paths, plus a secret scan and a dependency licence scan.
  4. Pretraining correctness. Make validation impossible to skip silently, promote checkpoints against the current validation loss, label local corpora with proper provenance, and refuse multi-process runs until data is sharded by rank.
  5. Training on Apple silicon. A preflight check and configuration for the MPS device, then a measured 500-step run.
  6. Run records and a baseline gate. The frozen evaluation set, a run record for every run, and a gate that fails anything worse than the baseline.
  7. Chat fine-tuning with LoRA. Chat templates, assistant-only loss masking, a learned end-of-sequence token and a merge command.
  8. First real fine-tune on owned hardware. Train, evaluate base against adapter, and keep or reject with a record.
  9. Rented-GPU readiness for T1. A launcher that fails fast without a GPU and stops at a wall-clock cap. Runs only after an owner budget decision.
  10. Serving through the shared gateway. Gated: only if a run record shows the model beating the hosted low-cost tier on a named product task.

sequenceSessions 5 to 8 are the shortest path to a tuned open-weight model on owned hardware. That is not a from-scratch model, and the notebook will say so when it happens.

§ 4.3out of scope

Never in scope

From the roadmap’s never-in-scope list and the repository’s anti-patterns. Changing any of these needs a written roadmap change first.

  • A home-grown deep-learning framework, tokenizer library, vector database or database.
  • Re-implementing FlashAttention from scratch.
  • Implementing every inference backend imaginable; Candle and llama.cpp are enough through Phase 3.
  • Any module that strips a model’s safety behaviour. The roadmap lists it as out of scope, and the portfolio plan rules it out for any product use.
  • Promising a 30B model trained on a single machine.
§ 4.4open questions

Questions still open

  • Which open base model, and under which licence, for the first fine-tune.
  • The release terms for any future code or weights. The custom licence is unwritten, and nothing is released before a legal review.
  • Whether to rent a GPU for a T1 run, or stay on owned hardware only. The default is owned hardware, with no spending.
  • Whether a “sovereign” model is needed by any product in the next six months. The working assumption is no: research only.