TD Stuff

All Agentic Systems Creative Code Damage Control Products & Announcements Programming Under the Hood

Topic: Mixture of Experts

The LLM Architecture Gallery: Mapping the Evolution of Open-Weight Models

Mar 16, 2026383

A comprehensive technical reference gallery documenting the architectural evolution and specifications of modern open-weight large language models.

AI Architecture Foundation Models Mixture of Experts LLM Inference Transformer Models

Products & Announcements

Qwen3-Next: Hybrid Attention + Ultra-Sparse MoE for 10x Faster Long-Context LLMs

Sep 12, 2025569

Qwen3-Next matches larger models while slashing training cost and delivering order-of-magnitude faster long-context inference via a hybrid attention + ultra-sparse MoE design with native MTP.

AI Architecture Mixture of Experts LLM Inference LLM Context Management