Orchestra-Research

llama-cpp is a lightweight C/C++ inference engine for running large language models on CPUs and non-NVIDIA hardware, including Apple Silicon and AMD/Intel GPUs. It supports GGUF…

TensorRT-LLM accelerates large language model inference on NVIDIA GPUs through advanced optimization techniques including quantization, in-flight batching, and multi-GPU…

Fine-tuning with TRL provides post-training methods to align language models with human preferences through multiple approaches. Train models on instruction data with SFT,…

serving-llms-vllm accelerates LLM inference for production environments through PagedAttention-based memory optimization and continuous batching. It supports OpenAI-compatible…