Dan Fu is an assistant professor of computer science and engineering at UC San Diego, vice president of kernels at Together AI, and a co-creator of FlashAttention, the influential algorithm that accelerated transformer models by reorganizing how GPUs access memory. His research makes AI models faster and more practical by designing algorithms, model architectures, and GPU kernels around the actual behavior of modern hardware.
Fu studied computer science at Harvard before completing doctoral research at Stanford with Christopher Ré and Kayvon Fatahalian. Earlier work spanned automatic parallelization, video analysis, and weak supervision; he also interned at Google and Argo AI and founded the Stanford MLSys Seminar in 2020. He began collaborating with Together AI as an academic partner in 2022 and now leads UC San Diego’s SandyResearch Lab alongside Together’s production-oriented kernels team.
Fu, Tri Dao, and collaborators introduced FlashAttention in 2022. Its IO-aware attention algorithm computes exact attention while reducing transfers between GPU memory and faster on-chip storage, using tiled computation and kernel fusion to improve speed without approximation. Fu subsequently helped bring the approach to Stable Diffusion and Hugging Face Diffusers.
- Hardware-aware model architectures. Fu contributed to Hungry Hungry Hippos and Monarch Mixer and co-developed FlashFFTConv, which uses tensor cores to accelerate long convolutions. These projects investigate whether alternatives to conventional attention become practical when their mathematical operations are designed for available accelerators.
- ThunderKittens and programmable performance. He co-developed ThunderKittens, a tile-oriented framework for writing high-performance GPU kernels. Subsequent work on NVIDIA Blackwell tackled tensor-memory management, coordinated thread blocks, and dataflow bottlenecks.
- ParallelKernelBench and reusable benchmarks. Fu co-authored ParallelKernelBench, which tests whether language models can generate correct, fast multi-GPU kernels for unresolved training and inference problems. Benchmark-specific improvements can therefore produce implementations useful in production.
- Inference infrastructure for long-running agents. Fu’s work on open-model serving addresses sparse attention, quantization, and architectural differences between models. Coding agents add repeated tool calls and large shared contexts, making KV-cache management a distributed-systems challenge involving cache placement, reuse, retrieval, and movement across machines.
