← Lcoalhost
A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream
Source :
DEV · #llm
See it live in context on Lcoalhost →