mmgp

What it is and what it does

mmgp is a memory management layer for PyTorch that enables large generative models to run on consumer-grade GPUs with limited VRAM by replacing or augmenting the accelerate library's offloading. It provides five predefined profiles (HighRAM_HighVRAM through VerylowRAM_LowVRAM) that automatically configure model loading, quantization, and GPU memory budgets based on your hardware. The package handles smart model loading/unloading, on-the-fly 8-bit quantization, model slicing, pinned RAM transfers, and async VRAM loading to reduce pauses.

Typically used as a drop-in replacement after instantiating a model pipeline, mmgp intercepts safetensors calls and manages which model components stay in VRAM, which are quantized, and which are offloaded to system RAM. It targets models like Flux, Mochi, CogView, and HunyuanVideo that would otherwise exceed VRAM on mid-range GPUs. Configuration is either profile-based (recommended for most users) or fine-grained via parameters like pinnedMemory, budgets, and quantizeTransformer.

Use it for:

• Run text-to-image generation on consumer GPUs by selecting an appropriate profile and letting mmgp quantize the transformer model.
• Generate long videos with limited VRAM using VerylowRAM_LowVRAM profile for slower but functional output.
• Accelerate model loading by pinning frequently-used components to reserved RAM while keeping VRAM budget tight for inference data.
• Avoid repeated out-of-memory errors in pipelines that load/unload VAE and text encoders multiple times by using smart automated offloading.
• Trade inference speed for memory by manually tuning budgets and async transfers for long-batch or long-video generation tasks.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Optimizes GPU memory usage for running large generative models on consumer GPUs with limited VRAM by managing model loading, quantization, and offloading.

Yes, if you run generative models on consumer GPUs and hit VRAM limits with standard PyTorch/accelerate setups. The five profiles provide a low-friction starting point, and active maintenance (release 8 days old) signals ongoing support. However, verify the unclear license status before use in proprietary work, and test your specific model and hardware combination first.

Install

mmgp on PyPI

pip install mmgp

uv add mmgp

poetry add mmgp

Installing mmgp

Before you install

Low install friction; pure Python wheel with five runtime dependencies (torch, optimum-quanto, accelerate, safetensors, psutil). Active maintenance with release 8 days old.

License in practice

License status is unclear—no SPDX identifier or raw license text provided. Verify licensing terms before use in proprietary or commercial projects.

Quickstart

pip install mmgp

from mmgp import offload, profile_type
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16).to("cpu")
offload.profile(pipe, profile_type.HighRAM_LowVRAM_Fast)

Requires Python >=3.10; models must be loaded to CPU device first; minimum 6 GB VRAM and 24 GB RAM; Windows requires an extra 16 GB RAM vs. Linux.

Verify before relying

• Actual compatibility with PyTorch versions and specific model architectures beyond those listed in examples.
• Performance benchmarks comparing the five profiles under identical hardware/model conditions.
• Whether safetensors library rewrite maintains full API compatibility with standard safetensors.
• Stability and edge cases when switching profiles mid-pipeline or with custom model architectures.

Package facts

Evidence: mmgp-3.7.12-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading