Varun Singh

Varun Singh is Arcee AI’s pre-training lead and the first author of the technical report behind Trinity, its family of open-weight mixture-of-experts language models. He builds models for reasoning, coding, and software agents by designing pre-training around the capabilities that reinforcement learning will develop later.

Singh contributed to AFM-4.5B, Arcee’s first foundation model, introduced in June 2025. Its development combined curated training data, instruction tuning, reinforcement learning, and extended-context techniques in a relatively compact model intended for organizations to operate on their own infrastructure.

He subsequently led the author list of the Trinity technical report, covering Trinity Nano, Mini, and Large: models with 6 billion, 26 billion, and 400 billion parameters, respectively. Trinity Large activates 13 billion parameters per token through sparse expert routing and incorporates interleaved local and global attention, the Muon optimizer, and Soft-clamped Momentum Expert Bias Updates to distribute work more effectively among its experts.

How Singh approaches model development

• Atomic skills for reinforcement learning. Singh argues that foundation models should acquire the basic behaviors subsequent reinforcement learning can assemble into mathematical reasoning, coding, and tool use. His AI Engineer conference talk challenges the older assumption that a model’s chief purpose is to reproduce the distribution of scraped web text.

• Synthetic data rephrasing. Trinity’s training incorporated generated rephrasings that expose models to the same underlying information in multiple forms. Singh favors targeted synthetic data that improves coverage and gives models earlier experience with instruction-like and agent-oriented tasks, while acknowledging competing approaches that avoid model-generated examples.

• Mixture-of-experts load balancing. Expert utilization can deteriorate when post-training tasks differ sharply from pre-training data. Singh connects routing stability to training-mixture design: introducing relevant task formats earlier can establish more durable representations and reduce disruptive distribution shifts.

His work also extends into agent infrastructure. In August 2026, he co-authored the introduction of NAC, Arcee’s open-source agent harness, which coordinates complex engineering tasks through persistent state, an orchestrator, and bounded worker threads. His question about hidden conditioning tokens raises a related model-security concern: whether specialized behaviors remain detectable after the associated tokenizer and output-layer entries have been removed.