What Does TOPS Mean? Reading the AI Performance Number on New Chips

Every new laptop and AI PC now carries a TOPS rating stamped on the box, usually a number like 40 or 50 with "AI" next to it. It looks like a clock speed for intelligence, a single figure that tells you how clever the machine is. It is not. TOPS measures one narrow thing under ideal conditions, and it routinely fails to predict how fast that chip will actually run the AI workloads you care about.

Quick Answer

TOPS stands for trillions of operations per second, the peak throughput an NPU can theoretically hit. It is a ceiling, not a real-world speed, and for tasks like running a local language model the limiting factor is usually memory bandwidth, not TOPS. A 40 TOPS chip is the floor for Microsoft's Copilot+ badge.

What the Number Actually Counts

TOPS counts how many low-precision multiply-and-add operations a neural processing unit can perform every second at its theoretical maximum. The operations are usually measured at INT8 or even lower precision, because that is what most on-device AI inference uses. Multiply the number of math units by their clock speed by how many operations each does per cycle, and you get the headline figure.

The catch is in the word "theoretical." That number assumes every math unit is fed perfectly, with no waiting, no data starvation, and the model already sitting where the NPU can reach it. Real workloads almost never hit that, which is why two chips with the same TOPS can deliver very different speeds.

If you are shopping for a machine to run AI features locally rather than chasing the spec sheet, it is worth comparing actual configurations in the AI PC range at Evetech rather than going on TOPS alone.

Why Memory Bandwidth Usually Wins

Running a language model is mostly a memory problem, not a maths problem. To generate each token the chip has to read the model's weights out of memory, and a useful local model can be several gigabytes. The NPU finishes the arithmetic quickly and then sits idle, waiting for the next chunk of data to arrive over the memory bus.

That means the real ceiling is how fast the system can move weights into the processor, measured in gigabytes per second. A chip with a huge TOPS rating but modest memory bandwidth will spend most of its time waiting, and its real token-per-second output lands far below what the TOPS figure implies. This is why bandwidth, memory capacity, and the model size you intend to run matter more than the marketing number.

The 40 TOPS Threshold and What It Buys

The reason 40 TOPS keeps appearing is Microsoft's Copilot+ requirement: a PC needs an NPU rated at 40 TOPS or higher to qualify for that badge and run certain on-device features. So the number has become a marketing target chips are designed to clear, rather than a measure tuned to your workload.

Clearing 40 TOPS is genuinely useful for the things NPUs are good at, such as background blur on video calls, live captioning, image processing, and small on-device models that run efficiently. It is far less predictive of how a chip handles a large local LLM, where bandwidth and memory headroom decide the experience.

For the broad market, current AI PCs sell across a wide band, and the PCs moving fastest at Evetech give a realistic sense of where pricing and real configurations sit today.

Frequently Asked Questions

Is a higher TOPS number always better?

Higher TOPS gives a chip more peak headroom, but it does not guarantee faster real performance. If memory bandwidth is the bottleneck, extra TOPS sit unused. Compare the whole platform, not one figure.

What does 40 TOPS actually let me do?

It clears the bar for Microsoft's Copilot+ features and handles on-device tasks like live captions, video effects, and small efficient models smoothly. It is the entry point for the current generation of AI PCs.

Why do two chips with the same TOPS perform differently?

Because TOPS ignores memory bandwidth, cache, software optimisation, and the precision the workload uses. The chip that can feed its math units faster will win in practice even at identical TOPS.

Does TOPS matter for gaming?

Not directly. Gaming performance comes from the GPU and CPU, not the NPU. TOPS only describes the AI accelerator, which games today barely touch.

Should I buy based on TOPS at all?

Treat it as a minimum-spec filter, not a ranking. Make sure the machine clears the threshold you need, then judge it on memory, bandwidth, and real reviews of the AI tasks you plan to run.