---
id: smg-grpc-servicer
version: "0.8.0"
license: Apache-2.0
license_treatment: permissive
maintenance: active
---

# smg-grpc-servicer — SMG gRPC servicer implementations for LLM inference engines (vLLM, MLX, TokenSpeed, SGLang)

License: permissive · Maintenance: active · Downloads: 1.1M/mo

## What it is and what it does

smg-grpc-servicer is a bridge library that wraps four popular open-source LLM inference engines—vLLM, MLX, TokenSpeed, and SGLang—behind a unified gRPC interface. Instead of calling each engine's native Python API directly, you run one of the supported backends with gRPC enabled (or via the servicer's CLI modules), and clients connect over gRPC to send inference requests and receive responses.

The package isolates backend dependencies using optional extras ([vllm], [mlx], [sglang]) so you only install what you need; TokenSpeed integrates via an external runtime. It depends on grpcio, grpcio-reflection, and grpcio-health-checking for service discovery and health checks. The library is actively maintained, supports Python 3.10–3.13, and carries no known vulnerabilities.

Use it for:

- Serve a vLLM model over gRPC to multiple remote clients without exposing HTTP endpoints.
- Run MLX inference on macOS and expose it as a gRPC service for distributed workloads.
- Build a microservice architecture where LLM inference is decoupled from application logic via gRPC.
- Load-balance inference requests across multiple backend instances using gRPC service discovery.
- Integrate TokenSpeed or SGLang inference into a polyglot system using language-agnostic gRPC clients.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides gRPC servicer implementations that expose LLM inference engines (vLLM, MLX, TokenSpeed, SGLang) as gRPC services for remote model serving.

Yes. The package solves a real problem—exposing LLM inference engines as gRPC services—with low install friction, active maintenance, permissive licensing, and no known vulnerabilities. Install it if you need to serve LLM inference over gRPC to remote clients or integrate multiple inference backends into a distributed system.

## Install

pip install smg-grpc-servicer
uv add smg-grpc-servicer
poetry add smg-grpc-servicer

## Installing smg-grpc-servicer

Before you install: Low friction: pure Python wheel with four runtime dependencies (smg-grpc-proto, grpcio, grpcio-reflection, grpcio-health-checking). Active maintenance—released 15 days ago with 461 repository stars.

License in practice: Apache-2.0 permissive license allows commercial and private use with minimal restrictions.

Quickstart:

# For vLLM:
pip install smg-grpc-servicer[vllm]
vllm serve meta-llama/Llama-2-7b-hf --grpc

# For MLX:
pip install smg-grpc-servicer[mlx]
python -m smg_grpc_servicer.mlx --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051

Requires Python >=3.10. Each backend (vLLM, MLX, TokenSpeed, SGLang) must be installed separately; the servicer itself provides only the gRPC bridge.

Verify before relying:

- Whether gRPC service discovery and reflection work out-of-the-box or require additional configuration.
- Performance characteristics and latency overhead of the gRPC bridge relative to native backend APIs.
- Load-balancing and concurrency limits when serving multiple concurrent inference requests.

## Package facts

- License: Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 1.1M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

grpc llm inference server, vllm grpc servicer, llm model serving grpc, distributed llm inference, grpc language model endpoint, mlx tokenspeed sglang grpc, llm inference bridge, llm-inference, grpc-bridge, model-serving

[View on SkillFed](https://skillfed.io/packages/smg-grpc-servicer) · [View on PyPI](https://pypi.org/project/smg-grpc-servicer/)