Tag: distributed transformer inference

Distributed Transformer Inference: Tensor vs Pipeline Parallelism for LLMs

Learn how to deploy large language models using distributed transformer inference. We compare Tensor and Pipeline Parallelism, discuss KV cache bottlenecks, and explore tools like vLLM and llm-d for efficient LLM serving.

Read More