← Lcoalhost

A 671B model reads 36GB per token and still needs 671GB of HBM

Source : DEV · #llm

See it live in context on Lcoalhost →