← Lcoalhost

A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

Source : DEV · #llm

See it live in context on Lcoalhost →