AI Model Size vs Hardware Scaling

About the survey

Rapidly growing AI workloads are driving large investments in AI datacenters. This survey (arXiv:2609.26829) classifies industrial AI accelerators into four architectural categories and compares their compute and memory organizations. It examines how node-, rack-, and pod-scale scale-up interconnects support collective communication, and it traces architectural evolution across accelerator generations, showing how increases in arithmetic throughput are accompanied by changes in data delivery, execution coordination, communication, and deployment infrastructure. Together, these analyses highlight key challenges in future AI datacenter design, from workload flexibility, data movement, and communication efficiency to power delivery, cooling, and rapid model evolution, and they show how the trade-off between generality and specialization extends from individual accelerators to datacenter-scale systems. The chart below is an interactive version of the survey's scaling figure: frontier-model parameter counts against per-accelerator dense FP16/BF16 compute, DRAM bandwidth and capacity, and scale-up interconnect bandwidth, extended with accelerator generations documented in the survey's data repository (GitHub: github.com/Yufeng98/AI-datacenter).

Model scaling versus accelerator scaling

Each series is normalized to its first observation in the survey's Figure 2 and plotted on a log scale. Legend percentages are endpoint annualized growth rates over the points currently shown. Hover a point for its value and source; click it for details. Scroll or drag to zoom, and use the filters to choose metrics, vendors, product status, and data sets.

Metric

Vendor

Status

Data set

Show point labels

Hollow markers: preliminary or forthcoming products.

Click a point to see the product, its recorded metrics, notes, and sources.

Methodology

Dense FP16/BF16 compute is the vendor-stated peak dense matrix (tensor-engine) throughput per accelerator, counting a fused multiply-add as two operations. No sparse or lower-precision figure is converted into an FP16 equivalent; products that disclose only FP8 or INT8 peaks contribute no compute point.
DRAM bandwidth and capacity are per accelerator (chip or package as sold). HBM is the norm; GDDR, LPDDR, and DDR products are noted as such. SRAM-only designs contribute no DRAM point. Wafer-scale engines are left out of the compute series because a wafer is not comparable to a single package.
Scale-up interconnect bandwidth is the aggregate bidirectional chip-to-chip bandwidth per accelerator as reported by the vendor. Unidirectional or Tbps figures are converted, and each conversion is recorded in the point's note.
Model size is the total parameter count, including all mixture-of-experts parameters rather than active parameters.
Normalization divides each value by a fixed baseline: the first observation of that series in the survey's Figure 2 (GPT for model size; TPU v2 for compute, DRAM bandwidth, and capacity; NVLink 2 for interconnect). Baselines do not change with the filters, so the "Survey Figure 2" data set reproduces the published figure and its growth rates.
Growth rates in the legend follow the figure's method: the annualized rate between the earliest visible observation and the last record-setting observation of that series.

Show the visible points as a table

Show the reference list

Sources are numbered in order of first use. Entries used by the points currently shown are highlighted; the rest are dimmed.

Citation

If you find this survey or its data useful, please cite:

@article{gu2026balancing,
  title   = {Balancing Generality and Specialization: A Survey on AI Datacenter Hardware Architecture},
  author  = {Gu, Yufeng and Wang, Jiazhen and Das, Reetuparna},
  journal = {arXiv preprint arXiv:2609.26829},
  year    = {2026}
}