Tag: post-training quantization

Running LLMs on Edge Devices: Compression and Quantization Guide

Running LLMs on Edge Devices: Compression and Quantization Guide

Learn how to run large language models on edge devices using compression and quantization. Explore PTQ, QAT, GPTVQ, and tools like AIMET for efficient local AI deployment.

Read More
Post-Training Quantization for Large Language Models: 8-Bit and 4-Bit Methods Explained

Post-Training Quantization for Large Language Models: 8-Bit and 4-Bit Methods Explained

Post-training quantization cuts LLM memory use and speeds up inference by 2-3x without retraining. Learn how 8-bit and 4-bit methods like SmoothQuant, AWQ, and GPTQ make it possible-and what you need to know to use them.

Read More