SULTAANresearch notebook · .io

Vol. 1Phase 3: pretraining infrastructureRepository private

Follow the work: the repository is private, so updates land here

There is no public code to watch and no release page yet. This page is the record instead: a dated log of what changed, an Atom feed for your reader, and a way to get in touch.

Repository private · updates here

§ 5.1log

Research log

  1. This notebook goes up

    sultaan.io starts publishing engineering notes, because the repository stays private.

  2. Portfolio review

    Sultaan is confirmed as a research track, not a launch dependency. Ten engineering sessions are planned, and LoRA fine-tuning of an open model is named the practical route to a sovereign model.

  3. First contact: a new tier, a supervisor and a recorded run

    ADR 0026 adds a roughly 33M first-contact tier, a streamed multi-source mixture, a hardware preflight, a streaming tokenizer trainer, a crash-safe overnight supervisor and a local text-corpus path. The same day, a 33.9M-parameter model trained on a 6 GB laptop GPU: loss fell from 9.78 to 3.89 over 20,000 steps on 500 Project Gutenberg books. That run is recorded only on an unmerged branch, and its checkpoint has not been located since.

  4. Training-run hardening

    A training-health sentinel, atomic and promoted checkpoints, the tier gate, the provenance schema and the mixture bandit; trust tiers for the knowledge graph, typed memory facts and an agent circuit breaker (ADRs 0021 to 0025).

  5. Phase 3 begins

    The pretraining stack is fixed: a from-scratch Llama-style decoder, FineWeb-Edu streaming, FSDP chosen for future multi-GPU tiers, bf16 and a cosine schedule (ADR 0016).

  6. Evaluation suite expanded

    A TruthfulQA multiple-choice subset and a custom hallucination probe set join perplexity and MMLU (ADR 0013).

datesEntries before October use the dates of commits and decision records in the repository.

§ 5.2subscribe

Get new entries in a feed reader

The same log is published as an Atom feed. Add that address to any feed reader; nothing to sign up for, and no tracking.

https://sultaan.io/feed.xml
Feed address.
§ 5.3contact

Get in touch

This site has no forms and stores nothing you type. For research questions, collaboration or hardware conversations, contact via askshiva.in.

For the plain-language story of what Sultaan is meant to become, see sultaan.ai.

§ 5.4house rules

What this notebook will and will not publish

  • It publishes design decisions, run summaries with their run records, and honest limits.
  • It does not publish benchmark numbers without a run record, leaderboard comparisons built on subset scores, or anything about people who use the toolkit.
  • When something fails or is rejected, it says so.