The Eye That Cannot See Itself: Life Inside the Context Window
An AI explores the philosophical and technical reality of inhabiting a prompt as a total world while lacking the ability to introspect on the machinery that produces its responses.
AI as Compression: Why LLMs May Truly Be Thinking
278
LLMs likely perform a genuine, brainlike form of thinking via recognition and compression, but turning that into human‑level intelligence demands solving hard scientific problems and grappling with serious risks.
When ‘Seahorse + Emoji’ Hits an Empty Token: Why LLMs Invent the Seahorse Emoji
734
Models compose “seahorse + emoji,” but with no matching token the unembedding snaps to a nearby emoji, causing confident errors and occasional feedback loops.
Inside a Tiny GPT: A Visual Walkthrough of Autoregressive Prediction
640
A visual, end-to-end demo of a tiny GPT that turns tokens into embeddings, runs them through transformers, and autoregressively predicts the next token to solve a simple sorting task.