Diogo Almeida

Diogo Almeida is co-founder and chief executive of TypeSafe AI, building AI systems that can make reliable decisions and automate consequential work without constant human supervision. Previously a researcher at Google Brain and OpenAI, he helped develop the human-feedback techniques behind modern conversational AI—and now argues that those techniques also constrain what AI can accomplish independently.

Almeida coauthored research on learned optimizers that adapt optimization hyperparameters across unfamiliar machine-learning tasks. At OpenAI, he contributed to InstructGPT, whose foundational 2022 paper combined human demonstrations, preference rankings, and reinforcement learning to improve instruction-following. Human evaluators preferred its 1.3-billion-parameter model over a 175-billion-parameter GPT-3 baseline, demonstrating that targeted post-training could outweigh sheer model size.

He was also acknowledged among the contributors to ChatGPT and coauthored the GPT-4 technical report. His criticism of reinforcement learning from human feedback, or RLHF, emerges from firsthand involvement in establishing its effectiveness.

What comes after human-feedback training

Almeida identifies a mismatch between exceptional benchmark results and AI’s limited ability to handle expensive operational decisions. His AI Engineer World’s Fair talk develops four related arguments:

• Assistance versus automation: RLHF rewards responses that satisfy an attentive person, producing useful collaborators without necessarily preparing models to execute unsupervised work reliably.
• Reward-model asymmetry: Preference optimization can penalize visible uncertainty more readily than confident mistakes, encouraging hallucinations, overconfidence, and the suppression of plausible alternatives.
• Calibrated decision-making: Neither human preference nor verifiable correctness alone defines good autonomous behavior; models must align confidence and action with the actual stakes of each decision.
• Automation-native software: AI should expand what software can accomplish directly, instead of remaining a chatbot attached to existing applications or merely reducing the cost of writing conventional code.

Almeida also argues that excessive on-policy reinforcement learning can weaken tool use outside familiar settings, undermining practical engineering techniques such as decomposition, abstraction, and state management, as he described in a public post about tool-use reliability.

At TypeSafe, he is pursuing an alternative post-training and software stack organized around reliable automation, affordability, and intelligence per dollar. His premise is that pretrained models already contain substantial capability; the challenge is extracting it through objectives and interfaces designed for dependable action.