hailin
/
vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
vllm/vllm_v0.10.0/docs/features/quantization/README.md

551 B
Raw Permalink Blame History

Quantization

Quantization trades off model precision for smaller memory footprint, allowing large models to be run on a wider range of devices.

Contents: