1192 points•759 comments•2w ago•by stared•quesma.com ↗

Qwen 3.6 27B is the sweet spot for local development

The article identifies Qwen 3.6 27B as the practical sweet spot for local development, noting a larger 35B A3B variant as faster but more resource-intensive. It describes running the models locally with llama.cpp using 8-bit quantization from Hugging Face and a 64k-token context, and cites benchmark-like results from a MacBook Max M5 (128 GB) showing varying token-per-second performance that favors 35B but still demonstrates solid performance for 27B. It also outlines setup steps for running via llama-server and OpenCode configurations.

Discussion (759)

No comments yet

Top
Trending
New
For You
Bookmarks
Dossier