Google is trying to make its open Gemma 4 models feel less like a research download and more like something developers can actually run day to day. The latest step is speed: new Multi-Token Prediction drafters that can accelerate local inference by guessing several future tokens, then letting the main model verify them.
Ars Technica reports that Google says the technique can make Gemma 4 models up to 3x faster, with no quality loss when the draft tokens are accepted by the main model. The practical benefit is simple: local AI responses may feel less sluggish, especially on machines that are constrained by memory bandwidth rather than raw compute.
The bottleneck Google is attacking
Most large language models generate answers one token at a time. That is reliable, but slow. Each new token requires another pass through the model, even when the next word is obvious.
Speculative decoding changes the flow. A smaller drafter model proposes a short run of likely tokens. The larger Gemma 4 model checks those proposed tokens in parallel. If they match what the larger model would have produced, the system accepts the whole sequence instead of crawling forward one token at a time.
That matters more for local AI than it does for a giant cloud cluster. Consumer GPUs, laptops, and edge devices often spend a lot of time moving model weights through memory. If the model can do more useful work per pass, the same hardware feels faster without needing a new chip.
Why this fits the Gemma 4 pitch
Google’s broader Gemma 4 announcement positioned the model family around open, local, and edge deployment. The company says Gemma has passed 400 million downloads and that Gemma 4 is released under Apache 2.0, a much more familiar license for commercial developers.
The license change is not a footnote. VentureBeat argued that Apache 2.0 removes a major adoption blocker for teams that avoided earlier Gemma releases because of custom license terms. The Verge made a similar point, noting that the new license gives developers clearer room to modify, redistribute, and build commercial products on top of the model.
Faster inference strengthens that same argument. If an open model is legally easier to use but painfully slow on available hardware, many teams will still default to hosted APIs. If it is permissive and fast enough, local deployment becomes a real option for privacy-sensitive apps, offline tools, on-device assistants, and developer experiments.
The developer angle
For developers, this is less about benchmark bragging and more about latency budgets. A local coding assistant, document tool, or internal agent is only useful if users do not wait forever for each response.
The MTP drafters also keep the original model in control of the final output. That is important because speed hacks are only useful if they do not silently change answers. Google’s claim is that the main Gemma model verifies the draft tokens, so accepted sequences should match what the model would have produced anyway.
There are still caveats. Ars notes that the actual speedup depends on hardware and workload. A claimed 3x improvement will not be universal. Experimental drafters also add another deployment component for developers to manage, test, and monitor.
What to watch next
The interesting question is whether local model tools quickly make this easy. Gemma 4 already has attention from the open-model ecosystem. If frameworks and desktop apps package MTP support cleanly, developers may get the speedup without becoming inference engineers.
Google is not just competing on model quality here. It is competing on deployment friction: license clarity, hardware efficiency, ecosystem support, and response speed. That is the right battlefield for open models. The winner will not be the model with the flashiest demo; it will be the one teams can actually ship.
