{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/11"},{"label":"Information Analysis","url":"https://skillfed.io/packages/category/scientific-engineering-information-analysis/3"}],"enrichment":{"capability":"vllm-tpu is a high-throughput LLM inference and serving engine optimized for Google TPUs, supporting 200+ model architectures with efficient memory management, continuous batching, and structured output generation.","skillfed_tags":["llm-inference","tpu-optimized","model-serving"],"use_cases":["Deploy open-source LLMs on Google TPUs with high throughput for production inference","Build OpenAI-compatible API services for LLM inference without modifying client code","Serve multi-modal models or mixture-of-experts models on TPU clusters","Generate structured outputs using xgrammar or guidance constraints during inference","Implement distributed inference across multiple TPUs using tensor, pipeline, or data parallelism"],"what_it_does":"vllm-tpu is a specialized distribution of vLLM optimized for serving large language models on Google TPUs. It provides high-throughput inference with efficient memory management through PagedAttention, continuous batching, and support for 200+ model architectures including decoder-only LLMs, mixture-of-experts models, and multi-modal models. The package includes an OpenAI-compatible API server, support for structured output generation via xgrammar and guidance, and distributed inference across multiple TPUs.\n\nThe package brings together 66 runtime dependencies\u2014including transformers, fastapi, tokenizers, and specialized inference libraries\u2014to handle the full stack of model loading, tokenization, inference scheduling, and serving. It targets developers and researchers deploying LLMs at scale, offering both programmatic access via Python and a production-ready HTTP API. Installation requires Python 3.10\u20133.14 and TPU hardware or compatible accelerators.","worth_installing":"Yes, if you are deploying LLMs on Google TPUs and need production-grade serving infrastructure. The active maintenance, permissive license, and broad model support make it a solid choice. Medium install friction is acceptable for a specialized inference engine. No security vulnerabilities reported. Not suitable if you lack TPU hardware or require GPU-only solutions."},"id":"vllm-tpu","links":{"html":"https://skillfed.io/packages/vllm-tpu","md":"https://skillfed.io/packages/vllm-tpu.md","pypi":"https://pypi.org/project/vllm-tpu/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-31","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"vllm-tpu","python_support":"supports_current","summary":"A high-throughput and memory-efficient inference and serving engine for LLMs"},"popularity":{"monthly_downloads":85080,"position":13954,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.26.0"}