garak

What it is and what it does

Garak is a red-teaming and vulnerability assessment toolkit for large language models. It works like a security scanner (similar in spirit to nmap or Metasploit, but for LLMs) by running static, dynamic, and adaptive probes against a target model to uncover ways it can fail or behave undesirably. It supports a wide range of LLM sources—Hugging Face Hub models, OpenAI, Anthropic, Cohere, Replicate, AWS Bedrock, and REST-accessible endpoints—and can test local models or remote APIs.

The tool combines multiple probe frameworks and detectors to check for specific vulnerabilities: prompt injection attacks, hallucination, data leakage, jailbreaks, toxicity generation, misinformation, and others. It generates test prompts, collects model responses, and evaluates them against detectors to produce a report showing which probes succeeded and the failure rate for each. Results are logged in detail to JSON and a summary log, making it suitable for both one-off security checks and systematic vulnerability assessment workflows.

Use it for:

• Test a commercial LLM API (OpenAI, Cohere, Anthropic) for susceptibility to prompt injection or encoding-based attacks before deploying it in production.
• Evaluate a locally-hosted Hugging Face model for jailbreak vulnerabilities or toxicity generation as part of a model selection process.
• Run a comprehensive probe suite against a custom LLM endpoint to identify which vulnerability classes it is most susceptible to.
• Automate security regression testing on LLM updates to detect newly introduced failure modes or weakened defenses.
• Generate detailed vulnerability reports for compliance, risk assessment, or responsible AI documentation.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Garak is a command-line tool that probes large language models for security vulnerabilities, testing for hallucination, data leakage, prompt injection, misinformation, toxicity, jailbreaks, and other failure modes.

Yes, with conditions. Garak is actively maintained, has low install friction, and fills a clear niche in LLM security testing. However, the unclear license status requires verification before use in commercial or restricted contexts. The large dependency footprint (45 runtime packages including torch and multiple LLM SDKs) means it is best suited to dedicated security testing environments rather than lightweight integrations. Install it if you need systematic LLM vulnerability assessment; skip it if you need a minimal, license-transparent tool or have strict dependency constraints.

Install

garak on PyPI

pip install garak

uv add garak

poetry add garak

Installing garak

Before you install

Low friction install from PyPI; pure Python wheel. Requires Python 3.10–3.12 and pulls in 45 runtime dependencies including transformers, torch, and multiple LLM provider SDKs (OpenAI, Anthropic, Cohere, etc.). Active maintenance with recent releases.

License in practice

License status is unclear—no SPDX identifier or raw license text is recorded in the package metadata. Verify the actual license terms at the GitHub repository before adopting in a commercial or restricted-use context.

Quickstart

pip install garak
export OPENAI_API_KEY="sk-..."
python3 -m garak --target_type openai --target_name gpt-5-nano --probes encoding

Requires Python 3.10–3.12; most LLM providers require API keys set as environment variables; some probes may need additional system libraries (e.g., python-magic-bin for file type detection).

Verify before relying

• Whether the unclear license status reflects a missing declaration or a genuinely ambiguous licensing situation at the source repository.
• Whether all 45 runtime dependencies are required for basic use or if many are optional for specific probe/provider combinations.
• Performance and resource requirements when running large probe suites against local models with torch.

Package facts

Evidence: garak-0.16.0-py3-none-any.whl

Tags

More Security packages

Further reading