The LLM Architecture Gallery: Mapping the Evolution of Open-Weight Models
383
A comprehensive technical reference gallery documenting the architectural evolution and specifications of modern open-weight large language models.
Turning BERT’s MLM Into a Text Diffusion Generator
455
BERT-style MLM is a single-step text diffusion process, and extending it to multiple masking steps turns RoBERTa into a workable text generator.
When ‘Seahorse + Emoji’ Hits an Empty Token: Why LLMs Invent the Seahorse Emoji
734
Models compose “seahorse + emoji,” but with no matching token the unembedding snaps to a nearby emoji, causing confident errors and occasional feedback loops.
SimpleFold: Scalable Flow-Matching Transformers for Protein Folding
471
A large-scale, transformer-only, flow-matching approach makes protein folding simpler while staying competitive and practical.
Why Embeddings Got Bigger—and Where Efficiency Pulls Them Next
113
Embeddings got bigger with Transformers and APIs, but new efficiency techniques and infrastructure mean the future is about smarter—not just larger—dimensions.
Inside a Tiny GPT: A Visual Walkthrough of Autoregressive Prediction
640
A visual, end-to-end demo of a tiny GPT that turns tokens into embeddings, runs them through transformers, and autoregressively predicts the next token to solve a simple sorting task.