AI guardrails & agent security
3,625 agents · most starred first
AI guardrails & agent security, page 6 of 37
Agentic LLM CTF to test prompt injection attacks and preventions
AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents
Trust nothing. Ship safely. — Skeptical-reading and prompt-injection defense skill for AI agents. Provenance tagging, red-flag patterns, refusal templates, and a read-only injection auditor. MIT.
Praxen — agent behavior verifier. Compares an AI agent's declared policy against the available evidence; reports where observed behavior diverges from declared intent.
[ICML 2026] Official implementation for paper "Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents"
Security logging for AI coding agents: point Claude Code, Codex or any agent at your repo and add security events in small, verified waves. 90 review items, 9 workflow prompts, MIT.
Free local guardrail for AI coding agents: every tool_call judged on-device (regex + Laya), allow/review/block
Safe runtime for Claude Code, built to be yours.
A Framework for Evaluating AI Agent Safety in Realistic Environments
credential isolation for AI agents. Agents never see real API keys - structural guarantee, not policy.
CORE is a governance runtime for autonomous AI systems. It enforces constitutional rules during execution, prevents governance bypass, and creates auditable authority chains for agent actions across operational domains.
Security scores for 800+ MCP servers. 9 analyzers scan for prompt injection, toxic flows, and attack surface risks. Updated daily. 🛡️
AI agent firewall that intercepts tool calls (file, shell, network) and enforces deterministic policies at sub-microsecond latency using CEL, IFC, secret scanning, and audit logging.
Detect phantom IAM users, decode leaked AWS Bedrock and Claude Platform API keys, and prevent LLMjacking. CLI + SCPs + SIEM detection rules.
Memory Injection Attacks on LLM Agents via Query-Only Interaction
[ICLR 2025] Official codebase for the ICLR 2025 paper "Multimodal Situational Safety"
Security-first package manager for AI agent skills
Agent Identity Protocol - Zero-trust security layer for AI agents. Policy enforcement proxy for MCP with Human-in-the-Loop approval, DLP scanning, and audit logging.
The first multi-level safety evaluation platform for OpenClaw-style AI agents.
Provenance-aware runtime security for AI agents with deterministic ALLOW / REVIEW / BLOCK enforcement.
Botscent tells your site if an AI agent, not a person, is browsing it, and which one. 5 KB, MIT, 0 deps.
The front door to AISquare open source. Governance, debugging, and audit trails for AI agents — because "it just works" isn't an answer your compliance team accepts.
Identity preservation protocol for AI agents. Build walls around who you are.
A data access control framework for Open Policy Agent
Deterministic governance for AI coding agents. Cedar-based policy engine that intercepts every agent action and evaluates it against deterministic rules before execution.
Stop LLMs from hallucinating your guesses as facts. Clarity Gate is a verification protocol for your documents that are going to be provided to LLMs or RAG systems. Place automatically the missing uncertainty markers to avoid confident hallucinations. HITL for non-directly verifiable claims.
Stealthy Prompt Injection and Poisoning in RAG Systems via Vector Database Embeddings
Local-first verification layer for AI agents: evidence, provenance, policy gates, human approval, and auditable Trust Receipts before a claim, memory write, or action is trusted.
Powerful protection for AI agents - Open-source security and cost tracking for AI applications
Lightweight RAG provenance middleware. Verifies every claim in an LLM response is grounded in a retrieved source - without an LLM call.
Runtime Security for Multi-Agent AI — Website & Documentation
A credential-helper framework and agent for Bazel and similar tools implementing the credential-helper spec.
Domain-neutral AI red-team framework for recursive self-improvement governance
Just-in-time API keys for AI agents - and any other process you route through it: the caller only ever sees a placeholder.
Skarn plugins for Claude Code, Codex CLI, Gemini CLI, Grok Build, and Antigravity: audit skills, guard hooks, and MCP declarations that find leaked secrets and credentials in AI coding sessions, locally and redacted.
This repository contains two datasets with multi-turn adversarial conversations generated by human agents interacting with a dialog model and rated for safety by two corresponding diverse rater pools.
To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models
Secure autonomous AI agents in 60 seconds.
Browser extension with 35+ rules for keeping your AI agent safe while browsing
Tamper-evident evidence and verification layer for AI agents. Verify any agent's session offline with just openssl. No cloud, no lock-in.
Concurrent Go CLI that detects hidden prompt-injection content in documents — invisible Unicode, homoglyphs, hidden HTML, and known trigger phrases — before they reach a RAG pipeline.
Control what your AI can do
⚡ Millisecond System-1 judgement for every tool call in DeepSeek Harness — Jev-powered risk classification & evidence-gated auto-approval. Fail-closed by construction. dsh 生态第一个 System-1 决策插件
An Adversarial Co-Evolution Framework for LLM Agent Safety
Unalome Agent Firewall is a free, open-source desktop app that gives you complete visibility into what your AI agents are doing on your machine — in plain language anyone can understand.
Runtime visibility for Python MCP servers. Captures tool calls, session lifecycle, module imports (SHA-256), and subprocess execution as structured NDJSON. No code changes.
grantex is the identity, authorization, and audit infrastructure for AI agents — the "OAuth moment" for the agentic internet. We provide a universal SDK and cloud service that lets any AI agent act on behalf of a human with scoped, revocable permissions, cryptographic identity, and an immutable audit trail. Developers integrate in minutes.
Open-source guardrails between AI agents and FHIR clinical data — PHI redaction, immutable audit, step-up auth, tenant isolation. MCP server + OpenAI/Gemini adapters. A healthclaw.io project.
AI makes users productive — but also cognitively lazy. Lucid detects cognitive dependency in real-time and adapts AI responses to keep users thinking. Drop-in SDK: zero latency, any LLM provider, research-backed.
The purpose of this document is to outline the security risks and vulnerabilities that may arise when implementing ai in web applications and to provide best practices for mitigating these risks.
Open Source Python SDK for AI Agents Identity
Metadata encoding and extraction for AI-generated content
Proof of Human Intent (PoHI) - Cryptographically verifiable human approval for AI-driven development
Approval workflows for AI agents
Local-first MCP policy proxy. Tool-block, SQL-mutation gate, PII redact, audit, rate-limit, OpenTelemetry, vault secrets, first-use quarantine, schema pre-inject. No hosted gateway. One YAML Lens wraps any MCP, 20 included (Postgres, MySQL, MongoDB, GitHub, Stripe, Snowflake, etc.). 84% fewer tokens, ~3x faster, holds PII leaks. AGPL or commercial.
PoC for GhostCommit Attack: Steal secrets via a pull request pointing to an image
The gate between an AI agent and real money. Runtime authorization for financial AI agents: structured intent, deterministic policy, human approval, execute-once gateway, tamper-evident receipts. Open core, Apache-2.0.
AI Security Monitor — Real-time threat detection, prompt injection defense, and behavioral analysis for LLM-powered systems
Auth Agent is the first OpenID Connect Provider for Agents
Skill Scan Agent — Automated scanning, identification, and assessment of SKILL security risks.
Eversec AIGov-Insight Web
Open-source AI artifact scanner. Detect malicious agent skills, MCP servers, and IDE rule files before they run.
Don't let AI destroy your hard work! HardStop is a rock-solid protection for AI-generated commands. Pre-execution safety validation for Claude Code, Claude Cowork. Catches dangerous commands before they run: whether from AI mistakes, hallucinations, prompt injection, or misunderstood instructions. Seatbelts for the agentic AI era.
PFI: Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents
[IEEE S&P'26] WebCloak: Characterizing and Mitigating the Threats of LLM-Driven Web Agents as Intelligent Scrapers
Runtime security for AI apps and agents: prompt injection detection, tool-call authorization, sensitive-data redaction, bot protection, and rate limiting. Drop it into your Python code.
OWASP GenAI Data Security Initiative — taxonomies, crosswalks, datasets, and tooling for securing data in generative AI systems
Allows to red-team your AI systems through adversarial probing. It is simple, effective, and requires minimal setup.
Prompt Injection Attacks against GPT-4, Gemini, Azure, Azure with Jailbreak
Runtime economic governance for AI agents. Circuit Breaker adds risk-managed execution, cost ceilings, and kill switches to autonomous AI workflows.
Static security scanner for AI agents. Catches prompt injection, runaway loops, missing oversight, and compliance gaps across 21 frameworks. Use from Claude Code, Cursor, ChatGPT (MCP), the CLI, or GitHub Actions.
This research identifies a method to bypass safety systems in the GigaChat LLM, enabling the generation of potentially harmful content related to chemical synthesis through a "contextual camouflage" technique.
Code for the paper "Firewalls to Secure Dynamic LLM Agentic Networks"
Secrets management for AI agents via MCP • @janeesecure
Network observer that tracks outbound connections from AI CLI processes (Claude Code, Codex, Gemini), attributing sockets to providers in real time with SQLite logging
LLM Security Platform.
Security-first AI agent daemon — the model proposes actions, the runtime decides what execute
Deterministic pre-action authorization for AI agents. Guardrails run before tool execution, so prompt injection cannot bypass policy checks.
AI Governance Infrastructure — local evaluation. The governance layer for AI agents: a single compiled Rust binary that authenticates, authorises, rate-limits, logs, and costs every AI interaction. Self-hosted, air-gap capable, provider-agnostic.
Give an agent a job, not your keys. Open-source control plane for AI agent work: each agent gets its own identity, a bounded job, and a signed record of what it did.
Deterministic safety floor for tool-using AI agents: pre-execution gating, SafeFix suggestions, attack-chain detection, opt-in self-learning LLM judge. Papers: arXiv 2605.04785, 2606.08539
OWASP-aligned intentionally vulnerable platform for learning and testing AI, LLM, RAG, MCP and agentic AI security
Checks every AI agent action against your company's real rules before it runs. Git-native, PR-reviewed.
Red-team your RAG pipeline for prompt injection, source-document leakage, and cross-document smuggling, in CI. The gap between RAG eval (RAGAS/DeepEval) and LLM scanners (garak).
AI agent governance for teams. Runtime firewalls tell you what happened; Conduct Guard controls what can happen — signed policy, verified chain, fail-closed by default. Ships with Router (LLM proxy), 20+ compliance packs, canvas UI, and a playbook engine.
A lightweight, zero-dependency Python checkpoint layer for structured LLM output reliability.
[ICML 2026]A defense framework against MLLM-based web GUI agents. This repository provides both the generative CAPTCHA system and tools for evaluating agent resistance.
AI security training: hands-on labs, tools and courses for LLM, RAG and agent security. Maintained by Pwned Labs.
D.A.T.A.—the Data Authentication, Trust, and Attestation framework for AI Agent
LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins
Enterprise AI Security Platform - Real-time firewall protection for LLM applications against prompt injection, data leakage, and function abuse attacks
Semantic shell command safety classifier — AST-based risk scoring for AI coding agents
Governed Claude agent control plane: deterministic policy engine runs before every LLM call, $0 default autonomous spend, database-enforced immutable audit log. 147 passing tests. MIT.
🚦 Stop AI agents from deleting data, leaking secrets, or taking irreversible actions without approval. One ⭐ takes us a long way :))
Runtime security for AI agents. In-process, zero dependencies, Apache 2.0.
Harden Cursor and Claude Code against prompt injection via untrusted tool output (Agentjacking)
Prompt-injection, jailbreak and secret-leak defense for DeepSeek Harness: Aho-Corasick detection with allow/ask/block interception and sanitized audit events. Maintenance status: FROZEN since 2026-10-05.