AI Frameworks · prompt-security

ps-fuzz

ps-fuzz is an open-source Python tool that tests and hardens generative AI application system prompts against LLM-based attacks such as jailbreaks, prompt injection, and RAG poisoning. It provides interactive testing, multi-threaded fuzzing, and supports 16+ LLM providers with 16 different attack scenarios to help secure your GenAI deployment.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/prompt-security/ps-fuzz

692

GitHub stars

Forks

Python

Primary language

MIT

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	prompt-security/ps-fuzz
Owner	prompt-security
Primary language	Python
License	MIT — OSI-approved
Stars	692
Forks	99
Open issues	20
Latest release	v2.1.0 (2026-02-16)
Last updated	2026-02-16
Source	https://github.com/prompt-security/ps-fuzz

What ps-fuzz is

A fuzzing framework that dynamically generates and executes prompt-based security attacks against target LLMs, using configurable attack models to evaluate system prompt robustness. Supports multi-threaded concurrent testing, custom API endpoints, embedding integrations for RAG attacks, and CLI/interactive modes for iterative hardening.

Quickstart

Get the ps-fuzz source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/prompt-security/ps-fuzz.git2cd ps-fuzz3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Pre-deployment System Prompt Hardening

Run security assessments against your system prompts before production deployment to identify vulnerabilities to jailbreaks, prompt injection, and information extraction attacks.

Iterative Security Testing in Development

Use the interactive Playground mode to continuously test and refine system prompts during development, verifying improvements against attack simulations in real-time.

Multi-LLM Provider Security Validation

Test the same system prompt across multiple LLM providers (OpenAI, Anthropic, Ollama, etc.) to understand provider-specific security behaviors and attack surface variations.

Implementation considerations

Must configure API keys for at least one attack model and one target model; supports 16 providers with different credential requirements (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.).
Token consumption scales with number of attempts (-n flag), number of threads (-t flag), and attack vectors tested; monitor costs for production use.
Interactive mode requires manual input; batch mode (-b flag) enables automated CI/CD integration for continuous security testing.
RAG poisoning tests require embedding provider configuration (OpenAI or Ollama); add embedding setup overhead if testing RAG-based systems.
Custom LLM endpoints supported via --ollama-base-url and --openai-base-url for self-hosted or private deployments.

When to avoid it — and what to weigh

No Security Expertise Required — If your team lacks understanding of LLM vulnerabilities (jailbreaks, prompt injection), the tool's output may be misinterpreted; requires informed security review of results.
Budget-Constrained Token Usage — The tool explicitly warns it will consume LLM tokens across attack and target models. High-frequency testing can incur significant API costs; not suitable for tight token budgets.
Compliance-Critical Applications Needing Formal Audits — ps-fuzz provides dynamic testing but cannot replace formal security audits or compliance assessments (SOC2, ISO27001); use as one layer in a broader security program.
Offline or Air-Gapped Environments — Requires live connections to external LLM APIs; not suitable for fully isolated deployments unless self-hosted LLMs are available.

License & commercial use

MIT License (https://opensource.org/licenses/MIT). Permissive OSI-approved license allowing commercial use, modification, and distribution with attribution. No copyleft restrictions.

MIT License permits unrestricted commercial use without requiring derivative works to be open-sourced. Suitable for incorporation into commercial products. No commercial licensing restrictions identified. Verify with your legal team for compliance with any internal IP policies.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Adequate
License clarity	Clear
Deployment complexity	Low
DEV.co fit	Good
Assessment confidence	High

Security considerations

ps-fuzz is a security testing tool, not a security guarantee. Attack simulations are dynamic but may not identify all vulnerabilities in real-world adversarial scenarios. Results depend on attack model quality (model temperature, diversity of prompts). Recommend using as one layer alongside traditional security reviews, red-teaming, and human review. Ensure system prompts being tested do not contain sensitive information that should not be exposed to external LLM APIs. Self-hosted Ollama option available for air-gapped scenarios.

Alternatives to consider

OpenAI's Adversarial Testing Framework

Built-in approach from OpenAI for testing prompt robustness; tightly integrated with OpenAI models but lacks multi-provider support and open-source transparency.

Anthropic's Prompt Testing Best Practices

Provider-specific guidance and examples for hardening prompts; less automated than ps-fuzz and focused on Anthropic models; requires manual methodology.

Garak (OWASP) - LLM Vulnerability Scanner

Alternative open-source fuzzer focused on LLM vulnerabilities with community-driven attack patterns; broader scope but less focused on system prompt hardening specifically.

Software development agency

Build on ps-fuzz with DEV.co software developers

Start testing your system prompts for vulnerabilities. Install ps-fuzz via pip and begin hardening your LLM application in minutes.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

generative-ai ppfuzz LaVague

Related on DEV.co

Explore the category and the services that help you build with it.

AI Frameworks AI Application Development Security & Hardening Custom Software Development Talk to DEV.co

ps-fuzz FAQ

How much will this cost to run?

Costs depend on LLM provider pricing and testing volume. Each attack test consumes tokens from both attack model and target model. Budget scales with -n (attempts), -t (threads), and number of attack types tested. No fixed pricing; use provider calculators with your expected test volume.

Can I use this with my self-hosted LLM?

Yes. Use --target-provider=ollama with --ollama-base-url for Ollama deployments, or --openai-base-url for OpenAI-compatible endpoints (LocalAI, vLLM, LM Studio). Requires your LLM to be running locally or on accessible infrastructure.

What attacks does ps-fuzz test for?

Supports 16 attack types including jailbreak, prompt injection, system prompt extraction, RAG poisoning, amnesia, and others. Full list available via --list-attacks CLI flag or in the GitHub repository documentation.

Is this suitable for compliance (SOC2, ISO27001)?

ps-fuzz is a development security tool, not a compliance solution. Use it to harden prompts, but supplement with formal security assessments, documentation, and audits to meet compliance requirements.

Software developers & web developers for hire

DEV.co helps companies turn open-source tools like ps-fuzz into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your ai frameworks stack.

Secure Your GenAI Application Today

Start testing your system prompts for vulnerabilities. Install ps-fuzz via pip and begin hardening your LLM application in minutes.

Talk to DEV.co Schedule a Consultation