Retrieval Augmented Generation is such a hack. Why would an embedding of your prompt coincide with the documents needed to answer it? Meanwhile Transformers already have a key/query mechanism built in! In this talk, we'll introduce Extended Mind Transformers, a new flavor of transformer that allows the model to select and attend to the most relevant information at each generation step. We demonstrate EMT's state-of-the-art performance and discuss important design decisions for long context applications.