garak
LLM vulnerability scanner
Decision gist · record as of 2026-08-14
Yes, with conditions. Garak is actively maintained, has low install friction, and fills a clear niche in LLM security testing. However, the unclear license status requires verification before use in commercial or restricted contexts. The large dependency footprint (45 runtime packages including torch and multiple LLM SDKs) means it is best suited to dedicated security testing environments rather than lightweight integrations. Install it if you need systematic LLM vulnerability assessment; skip it if you need a minimal, license-transparent tool or have strict dependency constraints.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.10–3.12; most LLM providers require API keys set as environment variables; some probes may need additional system libraries (e.g., python-magic-bin for file type detection).
- Low friction install from PyPI; pure Python wheel.
- Requires Python 3.10–3.12 and pulls in 45 runtime dependencies including transformers, torch, and multiple LLM provider SDKs (OpenAI, Anthropic, Cohere, etc.).
License · maintenance · safety
(unclear) — License status is unclear—no SPDX identifier or raw license text is recorded in the package metadata. Verify the actual license terms at the GitHub repository before adopting in a commercial or restricted-use context.
last release 2026-08-04 (10 days) · last repo commit 2026-08-14 · 8,802 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 80,181 downloads/mo, #14,311 on PyPI
Alternatives
Verify before relying
pip install garak
export OPENAI_API_KEY="sk-..."
python3 -m garak --target_type openai --target_name gpt-5-nano --probes encoding- Whether the unclear license status reflects a missing declaration or a genuinely ambiguous licensing situation at the source repository.
- Whether all 45 runtime dependencies are required for basic use or if many are optional for specific probe/provider combinations.
- Performance and resource requirements when running large probe suites against local models with torch.
What it is and what it does
Garak is a red-teaming and vulnerability assessment toolkit for large language models. It works like a security scanner (similar in spirit to nmap or Metasploit, but for LLMs) by running static, dynamic, and adaptive probes against a target model to uncover ways it can fail or behave undesirably. It supports a wide range of LLM sources—Hugging Face Hub models, OpenAI, Anthropic, Cohere, Replicate, AWS Bedrock, and REST-accessible endpoints—and can test local models or remote APIs.
The tool combines multiple probe frameworks and detectors to check for specific vulnerabilities: prompt injection attacks, hallucination, data leakage, jailbreaks, toxicity generation, misinformation, and others. It generates test prompts, collects model responses, and evaluates them against detectors to produce a report showing which probes succeeded and the failure rate for each. Results are logged in detail to JSON and a summary log, making it suitable for both one-off security checks and systematic vulnerability assessment workflows.
Use it for
- Test a commercial LLM API (OpenAI, Cohere, Anthropic) for susceptibility to prompt injection or encoding-based attacks before deploying it in production.
- Evaluate a locally-hosted Hugging Face model for jailbreak vulnerabilities or toxicity generation as part of a model selection process.
- Run a comprehensive probe suite against a custom LLM endpoint to identify which vulnerability classes it is most susceptible to.
- Automate security regression testing on LLM updates to detect newly introduced failure modes or weakened defenses.
- Generate detailed vulnerability reports for compliance, risk assessment, or responsible AI documentation.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, with conditions.
Garak is actively maintained, has low install friction, and fills a clear niche in LLM security testing. However, the unclear license status requires verification before use in commercial or restricted contexts. The large dependency footprint (45 runtime packages including torch and multiple LLM SDKs) means it is best suited to dedicated security testing environments rather than lightweight integrations. Install it if you need systematic LLM vulnerability assessment; skip it if you need a minimal, license-transparent tool or have strict dependency constraints.
Install
garak on PyPI
Before you install
Low friction install from PyPI; pure Python wheel. Requires Python 3.10–3.12 and pulls in 45 runtime dependencies including transformers, torch, and multiple LLM provider SDKs (OpenAI, Anthropic, Cohere, etc.). Active maintenance with recent releases.
Requires Python 3.10–3.12; most LLM providers require API keys set as environment variables; some probes may need additional system libraries (e.g., python-magic-bin for file type detection).
License in practice
License status is unclear—no SPDX identifier or raw license text is recorded in the package metadata. Verify the actual license terms at the GitHub repository before adopting in a commercial or restricted-use context.
Quickstart
pip install garak
export OPENAI_API_KEY="sk-..."
python3 -m garak --target_type openai --target_name gpt-5-nano --probes encodingVerify before relying
- Whether the unclear license status reflects a missing declaration or a genuinely ambiguous licensing situation at the source repository.
- Whether all 45 runtime dependencies are required for basic use or if many are optional for specific probe/provider combinations.
- Performance and resource requirements when running large probe suites against local models with torch.
Package facts
| License | Not declared unclear |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 45 packagesmikeshardmind-base2048transformersdatasetscoloramatqdmcohereanthropicopenaireplicategoogle-api-python-clientbackoffnltkaccelerateavidtoolsstdlibslangchaincmd2torchsentencepiecemarkdownnumpyzalgolibecojideepllitellmllmjsonpath-nghuggingface_hubpython-magic-binpython-magic |
| Maintenance | Actively maintained 10 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 80,181 / month, #14,311 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 3 - AlphaOperating System :: OS IndependentProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12 |
Evidence: garak-0.16.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “language model vulnerability scanner”
- garakGarak is a command-line tool that probes large language models for…
- picklescanScans Python Pickle files and related serialized model formats to…
- python-gvmPython library that abstracts the Greenbone Management Protocol (GMP)…
Give your agent the search over MCP, or paste the wish link into any chat.
More Security packages
Provides Python bindings to the FreeDesktop.org Secret Service API for securely storing and retrieving passwords and secrets through GNOME Keyring, KWallet, or KeePassXC.
MSAL for Python handles OAuth2 and OpenID Connect authentication with Microsoft identity services, managing token acquisition, caching, and refresh for applications integrating with Microsoft Entra ID, Microsoft Accounts, and Azure AD B2C.
joserfc implements JOSE standards (JWS, JWE, JWK, JWT, and related RFCs) for signing, encrypting, and managing JSON-based cryptographic tokens in Python.
Authlib provides a complete implementation of OAuth 1.0, OAuth 2.0, and OpenID Connect 1.0 for building both authentication clients and servers, with built-in support for JWS, JWK, JWA, and JWT standards.
Provides low-level CFFI bindings to the official Argon2 password hashing algorithm for use by libraries and applications that need direct access to Argon2 without higher-level abstractions.
ADAL for Python authenticates applications with Azure Active Directory to obtain tokens for accessing Azure AD-protected resources.
Install only if maintaining existing code that already depends on it, and plan a migration.
See also llm-guard · deepteam · lm-eval · cisco-ai-skill-scanner · pyrit · promptlayer · cisco-ai-mcp-scanner · aiperf · llm · inspect-evals