← Lcoalhost

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

Source : DEV · #llm

See it live in context on Lcoalhost →