Livefront SpeaksWhat's a Transformer? Understanding the Innovation That Changed Artificial Intelligence

"Transformer" is a very generic name for a specific machine learning architecture, first published in a paper titled "Attention Is All You Need". Transformers were conceived to handle text translation, but are surprisingly effective in other (non-text) domains. What was the key innovation that set transformers apart from previous methods? In this talk we'll look at embeddings, gradient decent, loss functions, and how Transformers learn to pick up on conversational context using matrix multiplication. We'll also take a look at a newer architecture called "Mamba", and how it improved upon some of weaknesses of Transformers.