Personally owned retrieval-augmented question-answering tool for saved bookmarks.
github.com
Bio, Work & Ideas
Barry Zhang
Conference affiliation: Anthropic · 2025
On this page
Barry Zhang is a research engineer at Anthropic and a co-creator of Agent Skills, a format that packages procedural knowledge into instructions, scripts, and supporting files. His agent-design work explains when models should choose their own actions, how tool interfaces can prevent mistakes, and how successful procedures can become reusable expertise.
Practical experiments with retrieval and fine-tuning
Barry was building AI products at Meta in 2023 when he adopted the title of AI engineer after reading swyx’s description of the emerging discipline. Its emphasis on making AI practically useful appealed to him, as he later recalled in How We Build Effective Agents.
He and a friend built BKMK because both had overflowing bookmark collections they wanted to question conversationally. Their locally hosted prototype indexed web pages with embeddings and retrieved relevant material to answer questions. Segmentation and recursive summarization helped fit long articles into the model’s context. They used ChatGPT to learn, code, debug, and iterate, describing themselves at the time as non-engineers by trade. The prototype had limits, particularly in what it could index.
Auto-distill-GPT explored whether examples from a stronger model could teach a cheaper model a particular task. The library generated answers with GPT-4 and used them to fine-tune GPT-3.5. Barry was still experimenting with task scope and data quantity; comparable performance at lower cost and latency was a possibility to investigate, rather than an established result.
Publishing as Yijing Barry Zhang, he also coauthored a study of single-task and multitask text classification, first submitted in December 2024. The authors compared encoder models with language models and tested consolidated fine-tuning for tasks including intent detection and slot filling. On the evaluated datasets, they reported that consolidated multitask models matched separate-model setups. The contribution was collaborative: the study examined when one adapted model could perform jobs otherwise assigned to multiple models.
Choosing autonomy and designing its interfaces
At Anthropic, Barry coauthored Building effective agents with Erik Schluntz in December 2024. They distinguished workflows, whose paths are defined in code, from agents, which choose how to proceed using environmental feedback. Barry developed that distinction further in his agent-design talk:
Use exploration where it earns its cost. When a task’s decision tree is understood, Barry favors implementing it explicitly. An agent is a stronger fit when the task requires exploration, its value justifies the expense, and errors can be discovered. Coding provides a concrete example: tests and continuous integration supply feedback for correction, while human review checks broader requirements. His personal coding workflow starts with a design document and Claude-generated tests that he verifies before delegating implementation.
Debug what the model can actually see. Barry describes an agent as a model using tools in a loop, shaped by its environment and instructions. A computer-use agent may receive a screenshot without knowing the screen dimensions needed to click accurately. He recommends examining the context available at each decision and reviewing complete execution trajectories. The coauthored essay describes a related failure in the team’s coding agent: relative file paths caused mistakes after it changed directories. Requiring absolute paths eliminated that problem in their testing. Changing the tool interface addressed the failure directly.
Treat production complexity as a separate problem. Barry contributed to Claude Research’s multi-agent search system and was one of six coauthors of its engineering account. A lead agent delegates searches to subagents working in parallel, then synthesizes their findings. The team found that vague assignments led subagents to duplicate searches; specifying objectives, expected outputs, sources, and task boundaries improved delegation. His June 2025 reflection on multi-agent search contrasts a first prototype he built in a day with the much greater complexity of bringing it to production. His agent-design talk identified reasons to pursue multiple agents—parallel work, separation of concerns, and protection of the main agent’s context—but also unresolved questions about their communication. He separately called for ways to define and enforce time, money, and token budgets. These were open engineering questions, rather than capabilities he claimed were already solved.
Agent Skills: retaining procedures for future work
Barry coauthored the Agent Skills introduction with Keith Lazuka and Mahesh Murag, alongside contributions from colleagues across Anthropic. Skills address the procedural knowledge and organizational context a general-purpose agent needs for specialized work. Ordinary folders let people inspect, edit, version, and share that knowledge.
Progressive disclosure makes the format practical for large libraries. An agent initially receives each skill’s name and description, reads its main instructions when relevant, and opens additional resources as needed. Expertise can remain available in the filesystem without occupying the context window all at once.
Scripts preserve operations that are better executed than regenerated. The article’s PDF example bundles Python code for extracting form fields; the agent can run it without loading the entire script or document into context. Model-directed work can therefore incorporate an established, repeatable operation. The format became an open standard in December 2025 for portability across agent products.
Model Context Protocol supplies connections to external tools and data; skills can supply procedures for working with them. This distinction gives domain knowledge a reusable form alongside the agent’s access and runtime. In his public introduction to Skills, Barry presented files as an interim route toward continuous learning. Capturing successful approaches and recurring mistakes changes the instructions and code available to future sessions, allowing expertise to accumulate without requiring a change to the model’s weights.
Barry Zhang and Mahesh Murag explain how reusable folders of instructions, scripts, and assets can give general-purpose agents specialized expertise without rebuilding their underlying architecture.
A capable general-purpose agent still needs domain expertise; reusable skills package that expertise without requiring a different agent architecture for every field. 1:20 ↗2:18 ↗
Skills are organized folders containing procedural instructions, reusable scripts, and supporting assets, with skill.md acting as the main instruction and navigation file. 2:18 ↗3:33 ↗4:39 ↗
Progressive disclosure protects the context window by exposing skill metadata first and loading detailed instructions or files only when a task requires them. 3:33 ↗4:39 ↗9:07 ↗
In the emerging agent stack, MCP supplies connectivity to external tools and data, while skills supply the procedural expertise needed to orchestrate useful workflows. 8:08 ↗9:07 ↗
As skills become more sophisticated, teams need software-style evaluation, versioning, dependency management, and runtime predictability; the speakers present these as active development priorities. 10:09 ↗11:14 ↗
Shared, agent-created skills offer a path toward transferable procedural learning, but they are not a complete memory system and do not capture every type of information. 12:26 ↗13:20 ↗14:24 ↗
Barry Zhang explains when autonomous agents are worth their cost, how to construct them around a simple tool-use loop, and why better agent behavior starts with understanding the model’s limited perspective.
Use an agent when the task is ambiguous, valuable, and difficult to encode as a fixed decision tree; otherwise, a predefined workflow typically offers better cost control and predictability. 1:18 ↗2:30 ↗
Evaluate token economics, critical model capabilities, error severity, and error discoverability before granting autonomy; unit tests and CI make coding particularly amenable to verification. 3:29 ↗4:29 ↗
Begin with the minimal agent architecture: an environment, a set of tools, and a system prompt, connected by a model-driven tool-use loop. 5:39 ↗6:53 ↗
Optimize only after the basic behavior works, using techniques such as trajectory caching, parallel tool calls, and progress reporting when they address actual cost, latency, or trust requirements. 6:53 ↗7:46 ↗
Debug from inside the agent’s limited context window, supplying missing environmental details and examining prompts, tool descriptions, and complete trajectories to understand unexpected decisions. 7:46 ↗9:01 ↗10:05 ↗11:10 ↗
Treat budget-aware execution, self-evolving tools, and asynchronous multi-agent communication as open engineering questions rather than solved production capabilities. 11:10 ↗12:11 ↗13:13 ↗