smg-grpc-servicer

What it is and what it does

smg-grpc-servicer is a bridge library that wraps four popular open-source LLM inference engines—vLLM, MLX, TokenSpeed, and SGLang—behind a unified gRPC interface. Instead of calling each engine's native Python API directly, you run one of the supported backends with gRPC enabled (or via the servicer's CLI modules), and clients connect over gRPC to send inference requests and receive responses.

The package isolates backend dependencies using optional extras ([vllm], [mlx], [sglang]) so you only install what you need; TokenSpeed integrates via an external runtime. It depends on grpcio, grpcio-reflection, and grpcio-health-checking for service discovery and health checks. The library is actively maintained, supports Python 3.10–3.13, and carries no known vulnerabilities.

Use it for:

• Serve a vLLM model over gRPC to multiple remote clients without exposing HTTP endpoints.
• Run MLX inference on macOS and expose it as a gRPC service for distributed workloads.
• Build a microservice architecture where LLM inference is decoupled from application logic via gRPC.
• Load-balance inference requests across multiple backend instances using gRPC service discovery.
• Integrate TokenSpeed or SGLang inference into a polyglot system using language-agnostic gRPC clients.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides gRPC servicer implementations that expose LLM inference engines (vLLM, MLX, TokenSpeed, SGLang) as gRPC services for remote model serving.

Yes. The package solves a real problem—exposing LLM inference engines as gRPC services—with low install friction, active maintenance, permissive licensing, and no known vulnerabilities. Install it if you need to serve LLM inference over gRPC to remote clients or integrate multiple inference backends into a distributed system.

Install

smg-grpc-servicer on PyPI

pip install smg-grpc-servicer

uv add smg-grpc-servicer

poetry add smg-grpc-servicer

Installing smg-grpc-servicer

Before you install

Low friction: pure Python wheel with four runtime dependencies (smg-grpc-proto, grpcio, grpcio-reflection, grpcio-health-checking). Active maintenance—released 15 days ago with 461 repository stars.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions.

Quickstart

# For vLLM:
pip install smg-grpc-servicer[vllm]
vllm serve meta-llama/Llama-2-7b-hf --grpc

# For MLX:
pip install smg-grpc-servicer[mlx]
python -m smg_grpc_servicer.mlx --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051

Requires Python >=3.10. Each backend (vLLM, MLX, TokenSpeed, SGLang) must be installed separately; the servicer itself provides only the gRPC bridge.

Verify before relying

• Whether gRPC service discovery and reflection work out-of-the-box or require additional configuration.
• Performance characteristics and latency overhead of the gRPC bridge relative to native backend APIs.
• Load-balancing and concurrency limits when serving multiple concurrent inference requests.

Package facts

Evidence: smg_grpc_servicer-0.8.0-py3-none-any.whl

Tags

More Artificial Intelligence packages