Blogs

A tutorial spanning the AI system stack from the bottom up: microarchitecture, kernels, compilers, architecture, systems, and algorithms.

An interactive chart of frontier-model parameter counts against per-accelerator dense FP16/BF16 compute, DRAM bandwidth and capacity, and scale-up interconnect bandwidth, drawn from a survey of industrial AI accelerators.

Transformer-based LLMs have revolutionized AI, but deploying them is expensive on two fronts: parameter counts of 10–1000B with context lengths of 128K–1M demand enormous memory capacity, and autoregressive decoding emits one token at a time while re-reading the model weights, which strains memory bandwidth. Quantization attacks both at once, by reducing the number of bits each value takes. This post covers what to quantize, when to quantize, and the methods used in practice. ...

This tutorial introduces three commonly used developer tools: Git, Tmux and Vim, as well as Hugo, the engine used to deploy this blog. Git handles version control and collaboration with branching and merging. Tmux keeps long-running terminal sessions alive and lets you split your screen into panes. Vim is a powerful modal editor for fast navigation and editing. Hugo is a static site generator for building and deploying websites quickly. Each section below briefly explains what the tool does, why it’s useful, and gives concise descriptions of the components you’ll use most. ...