← Lcoalhost
I put the 381 tok/s Qwen stack on one RTX 3090, real chat does 118 and the headline mode crashes
Source :
DEV · #llm
See it live in context on Lcoalhost →