33 points•8 comments•2h ago•by saikatsg•github.com ↗

WebLLM: high-performance in-browser LLM inference engine

WebLLM is a high-performance in-browser language model inference engine that runs entirely in web browsers with WebGPU acceleration. It provides OpenAI API compatibility, supports streaming and JSON-mode generation, and offers multiple model families such as Llama, Phi, Gemma, Mistral, and Qwen, installable via npm, Yarn, or CDN.

Discussion (8)

No comments yet

Top
Trending
New
For You
Bookmarks
Dossier