will-it-asic
how it works

ASICs vs GPUs, in plain terms

A GPU is a general parallel computer that happens to be great at matrix math. An AI ASIC (a TPU, Trainium, or Gaudi) is silicon built for almost nothing but matrix math. That difference decides what runs, how fast, and at what cost.

the chip

Systolic arrays vs thousands of cores

A GPU spreads work across thousands of small cores (NVIDIA calls them CUDA cores, grouped into streaming multiprocessors) plus dedicated Tensor Cores. It's flexible: any operation you can express in CUDA runs.

A TPU instead pours most of its transistors into a matrix multiply unit, a systolic array: a grid of multiply-accumulate cells that streams data through in lockstep. It does one thing enormously well, and skips the general-purpose machinery a GPU spends area on. Trainium's NeuronCores and Gaudi's MME engines follow the same idea (Google's first-TPU write-up is still the clearest explanation).

GPU · SIMT

Thousands of cores + Tensor Cores. Runs arbitrary kernels. The whole CUDA ecosystem works out of the box. You pay area and power for that generality.

ASIC · systolic array

A giant MXU/MME with little else. Unmatched throughput-per-watt on the ops it supports, but only the ops the compiler knows how to lower.

the bottleneck

It's usually memory, not math

For large-model inference, the limiter is rarely peak FLOPS. It is whether the weights and the KV cache fit in high-bandwidth memory, and how fast that memory can be read. A 70B model in bf16 is ~140 GB of weights before you serve a single token; the KV cache then grows with batch size and context length.

This is why HBM capacity and bandwidth headline every accelerator's spec sheet, and why the calculator leads with a per-chip occupancy meter rather than a TFLOPS number. Training adds gradients and optimizer state on top. Mixed-precision Adam needs roughly 18 bytes per parameter, so a 70B model wants ~1.3 TB before activations.

the gate

The compiler decides what runs

Here is the part GPU calculators miss. On a GPU, if a model fits, it almost certainly runs, because CUDA covers the ecosystem. On an ASIC, your model must be lowered by a vendor compiler: XLA for TPU, AWS Neuron for Trainium/Inferentia, and the Intel Gaudi software suite for Gaudi. A custom CUDA kernel, an unusual attention variant, or a brand-new architecture can fit perfectly in memory and still fail to compile.

That's why a verdict here is never memory alone. We combine the memory fit with a compiler-support signal, plus curated, cited notes for popular models confirming a model actually runs. When we can't confirm it, we say UNKNOWN rather than guess.

training vs inference

Two very different questions

Inference needs weights + KV cache. It shards cleanly across chips with tensor and pipeline parallelism, and inference-tuned parts (Inferentia, TPU v5e, the upcoming inference TPUs) optimize for memory bandwidth and latency.

Training also stores gradients and optimizer state, and keeps activations for the backward pass. ZeRO/FSDP shard the model state across chips; activation checkpointing trades compute to shrink the activation footprint. The calculator models all of this. Toggle the workload to watch the memory change.

the trade

When an ASIC is worth it

leans yes
  • · Standard transformer architectures
  • · Steady, large-scale training or serving
  • · Cost-per-token or perf-per-watt matters most
  • · You can adopt JAX/XLA or the vendor SDK
  • · GPU supply is the constraint
leans no
  • · Custom CUDA kernels or exotic ops
  • · Fast-moving research, frequent arch changes
  • · Small / bursty workloads
  • · A tiny team on a tight deadline
  • · You need the broadest tooling ecosystem

Not sure where you land? Swipe through the trade-offs or run your model through the calculator.