1596 points•608 comments•0mo ago•by jfb•vickiboykis.com ↗

Running local models is good now

A writer describes experiments running AI models locally on a 2022 M2 Mac with 64 GB RAM, testing Gemma-4, Qwen, Mistral and other variants. They report that recent local-model tooling, notably using LM Studio and Docker with Pi as the agent harness, delivers roughly 75% the speed and accuracy of frontier models for agentic coding, making local inference more viable. The piece notes ongoing downsides such as slower inference and limited context windows, and cautions that local models are not yet production-ready, though the ecosystem is improving.

Discussion (608)

No comments yet

Top
Trending
New
For You
Bookmarks
Dossier