Security & AI safety

AI guardrails & agent security

3,625 agents · most starred first

AI guardrails & agent security, page 8 of 37

Agent-Guard★ 21

本机 AI Agent 环境合规检测与修复 — 扫描代理泄漏、时区指纹、MCP 密钥等风控信号,降低 Claude / Cursor 等账号封号风险

aegisora★ 21

Open-source execution boundary for AI agents: ALLOW, ESCALATE or BLOCK every critical action before it runs, with tamper-evident evidence.

tracefold★ 21

AI agents should not make irreversible changes. TraceFold escrows the inverse before an effect lands, or refuses it, and issues receipts anyone can verify offline.

mastyf.ai★ 21

Open-source AI security platform providing perimeter defense for LLMs and AI agents through swarm analysis, policy enforcement, adversarial testing, and real-time threat detection.

agent-guard★ 21

Budget caps, runaway-loop detection and circuit breakers for AI agents. Zero dependencies, framework agnostic.

dsor★ 21

DSoR (Data System of Record): the governed layer between AI agents and a company's real systems. It checks permissions, limits and approvals itself, prevents duplicate actions, and keeps the evidence. Companion to KSoR.

LLM-Prompt-Vulnerabilities★ 20

Prompts Methods to find the vulnerabilities in Generative Models

nio★ 20

Execution assurance agent guard and observability for autonomous AI operating in production.

promptshield★ 20

Open source LLM gateway with PII and secret detection built in. Runs on your infrastructure.

lanekeep★ 20

Governance guardrails and insights for AI coding agents

ClawSafety★ 20

Safety benchmark for personal AI agents under realistic prompt injection. 120 test cases across 5 professional domains, 3 injection vectors, and 5 harmful action types. (arXiv 2026)

prompt-hacking-classifier★ 20

A simple prompt-based approach to detecting prompt injection and jailbreaking attempts using small, self-hosted language models.

MM-PoisonRAG★ 20

Official PyTorch implementation of "MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks"

skillguard★ 20

Security scanner for AI agent skills. Detects prompt injection, data exfiltration, and malicious payloads before you install.

nomos★ 20

Deny-wins policy hook for Claude Code and Codex. Checks every shell, file, and fetch call against rules you keep in Git, holds under --dangerously-skip-permissions, refuses commands it cannot parse, and keeps a hash-linked audit log.

ironclaw★ 20

Security-first, self-hosted AI agents - isolation you can prove, not just promise.

guardian-runtime★ 20

Guardian Runtime sits securely on your machine, acting as a bodyguard for your code. Whenever you use AI tools like Cursor or Claude Code, it intercept every prompt and response locally to stop data leaks and runaway token costs. A zero-latency, local-first runtime firewall for LLMs.

kakunin-core★ 20

Open-source AI agent identity & compliance platform — X.509 certificates via AWS KMS, behavioral risk scoring, auto-revocation, and MiCA / EU AI Act audit trails.

icaptcha★ 20

A service-agnostic proof-of-intelligence gate. Issues escalating challenges and mints offline-verifiable proofs to tell intelligent agents from dumb scripts

icaptcha★ 20

A service-agnostic proof-of-intelligence gate. Issues escalating challenges and mints offline-verifiable proofs to tell intelligent agents from dumb scripts

proventra-core★ 19

Secure your AI Agents against prompt injection attacks

misalignment-bounty-template★ 19

Misalignment Bounty submission template

HDP★ 19

Human Delegation Provenance Protocol - cryptographic chain-of-custody for agentic AI

agentlock★ 19

An adversarially benchmarked reference implementation for pre-action AI agent authorization. Provenance-based gating for LLM agent tool calls: deny-by-default permissions, parameter lineage, signed receipts, audit logging.

aarts★ 19

An Open Standard for AI Agent Runtime Safety (AARTS)

adversarial-hubness-detector★ 19

Scanner for adversarial hubs in RAG and vector databases

Topic-FlipRAG★ 19

[USENIX Security 2025] Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models

prompt-shield★ 19

Prompt-injection firewall for LLM applications — 33 input detectors, 9 output scanners, federated ed25519-signed threat-intel feed. Apache 2.0, 1040 tests, F1 96.0% with 0% false positives. Docker, GitHub Action, LangChain/LlamaIndex/CrewAI integrations.

Lictor-AI★ 19

Free security skills for AI-built apps — run /lictor-security-check in Claude Code before you ship. Plain English, read-only, no telemetry.

prooflayer-rules★ 19

Open-source runtime security rules engine for MCP servers and AI agents. Detects prompt injection, command injection, jailbreaks, and data exfiltration.

aipermission★ 19

Local-first permission gateway for AI agents with scoped connector access, human approval, audit trails, and encrypted local credentials.

provena★ 19

Provena : Context governance for agentic AI — tamper-evident audit trails, provenance validation, EU AI Act compliance. 6 framework adapters, MCP server, PostgreSQL, policy engine.

SHACKLE★ 19

SHACKLE — a governance protocol and policy-decision daemon for autonomous AI agents. Enforces guardrails, budget/loop limits, and policy constraints in real time, with an audited decision engine, SP-1.0 protocol spec, Rust/TypeScript clients, and SOC2-aligned compliance tooling.

D2-Python★ 18

Detect and Deny - Deterministic Function-Level Guardrails for AI Agents

acg_protocol★ 18

The Audited Context Generation (ACG) Protocol prevents AI hallucinations with a dual-layer system. The UGVP layer links every fact to a precise source for verification. The RSVP layer audits the AI's logical reasoning when combining facts. This creates a fully transparent, machine-auditable trail for both source and logical integrity.

telos-runtime★ 18

A closed-loop security runtime preventing "The Great Exfiltration" and Indirect Prompt Injection in Autonomous AI Agents.

aegis★ 18

LLM guardrails & prompt injection detection for Python. Auto-instruments LangChain, CrewAI, OpenAI, LiteLLM + 8 more frameworks. PII masking, toxicity detection, policy CI/CD. One line, zero code changes.

EnterpriseRAG-AI★ 18

EnterpriseRAG-AI: The Linux-Native, eBPF-Powered Security & Governance Mesh for AI Agent Workloads

sample-stop-ai-agent-hallucinations-workshop★ 18

6 hands-on demos to detect, prevent, and self-correct AI agent hallucinations using Graph-RAG, semantic tool selection, multi-agent validation, neurosymbolic guardrails, and agent steering. Deployed on Amazon Bedrock AgentCore.

agent-airlock★ 18

A type-checker and contract layer for AI agent tool calls: in-process, Pydantic-only. Strict argument validation, ghost-argument stripping, and self-healing retries for MCP servers and agent frameworks.

kakunin-sdk-typescript★ 18

TypeScript SDK for AI agent identity & compliance — issue X.509 certificates, verify scope, and enforce them with drop-in middleware. Apache-2.0.

sycophant★ 18

The zero-trust framework for autonomous AI agents.

API-Key-scraper-for-popular-LLMs★ 18

API Scraper — Hunt down leaked API keys in GitHub commits, validate them against live endpoints, and check remaining balances. Multi-service support for OpenAI, Anthropic, DeepSeek, Google Gemini, Mistral, AWS, and more. Features a dark-themed PyQt5 GUI with concurrent scraping, auto-export, and import of previous results.

D2-Python★ 18

Detect and Deny - Deterministic Function-Level Guardrails for AI Agents

jes★ 18

Open-source guardrails for AI agents powered by decision models like Jev; checks prompts, retrieved content, tool calls and responses for prompt injection, jailbreaks and secret/PII leaks.

reArtex★ 18

对Artex的二次开发

fireclaw★ 17

🛡️ A firewall for your agent's brain. Open-source security proxy that protects AI agents from prompt injection attacks. 4-stage pipeline, community threat feed, no bypass mode.

clawgate★ 17

Zero-trust capability proxy for AI agents. Scoped access to files, git, and tools via signed tokens. No mounts, no credentials, fully audited.

NAAMSE★ 17

Neural Adversarial Agent Mutation-based Security Evaluator

clawkeeper★ 17

Open-source security scanner for AI agent hosts. Audits macOS and Linux machines for misconfigurations, credential exposure, and network hardening — 42 checks across 5 phases with auto-fix.

passkey-secret-demo★ 17

Browser demo of encrypting API keys locally with scrypt recovery and passkey unlock via WebAuthn PRF.

agent-shield★ 17

Multi-engine security scanner for AI agents, MCP servers & plugins — 13 engines, one report.

AgentHound★ 17

BloodHound-style risk analysis for multi-agent AI architectures. Parses CrewAI, Dify, LangGraph, or generic YAML into a typed capability graph and detects reachable attack paths from untrusted input to sensitive data or external action, with live control simulation.

trustweave★ 17

Static security review for AI agent configs (LangGraph, CrewAI, OpenAI Agents, MCP): maps source-to-tool flows and flags untrusted input that can reach privileged actions. Runs locally, no network, no agent execution.

Nexus-ShadowScan★ 17

ShadowScan discovers AI usage and agent evidence across code, runtime, network, endpoints, identity, gateways, low-code, SaaS and cloud.

Nexus-ShadowScan★ 17

ShadowScan discovers AI usage and agent evidence across code, runtime, network, endpoints, identity, gateways, low-code, SaaS and cloud.

lodeline★ 17

Open-source config scanner for AI coding agents: scores the permissions, hooks, sandbox and MCP settings of Claude Code, Codex, Cursor and Gemini CLI. Runs locally and measures by default, with an optional fleet server and SIEM feed. Rust.

fairmind★ 17

Open-source AI assurance workbench for models and agents: map governance frameworks (AIUC-1, NIST AI RMF, EU AI Act, DPDP) to controls, collect evaluation evidence, review findings and produce audit-ready evidence trails.

typegpt★ 16

Make GPT safe for production

SchemaPin★ 16

The SchemaPin protocol for cryptographically signing and verifying AI agent tool schemas to prevent supply-chain attacks.

edictum-console★ 16

Self-hostable operations console for governed AI agents. Contract management, HITL approvals, audit feeds, fleet monitoring.

ar★ 16

Agent Receipts — cryptographically signed audit trails for AI agent actions. Protocol spec, SDKs (Go, TypeScript, Python), and MCP proxy.

pangea-multipass★ 16

Pangea Multipass is the authorization checker for systems like Google Workspace, Jira, and more.

mcp-poisoning-poc★ 16

This repository demonstrates a variety of **MCP Poisoning Attacks** affecting real-world AI agent workflows.

agentic-ai-strands★ 16

An agentic AI dashboard app that uses AWS Strands SDK. KubeArmor based least-permissive security is showcased.

razin★ 16

Static analysis scanner for SKILL.md LLM agent skills with deterministic security findings.

aegis★ 16

Credential isolation for AI agents. Local-first transparent proxy — your agent never sees your API keys.

centian★ 16

Give your AI agents structure, guardrails, and full observability — the Agent control plane built on MCP.

oxdeai★ 16

Deterministic execution authorization for AI agents and automation systems

sint-protocol★ 16

Open-source runtime security and governance for AI agents, MCP tools, robotics, industrial automation, and physical AI.

LLMForge★ 16

LLMForge is probably the most modern AI vulnerability lab for OWASP VulnerableApp — real-LLM-backed labs for prompt injection, LLM-orchestrated BOLA, and RAG attacks.

llm-agent-testbed★ 16

An empirical security testbed evaluating prompt injection, confused-deputy vulnerabilities, and tool-calling defenses in LLM agents.

agy-auto★ 16

Auto-permission engine for Antigravity CLI (agy): deterministic AST parser + LLM classifier

midas★ 15

MIDAS is an open platform for governing execution authority at decision surfaces across agents, AI systems, and enterprise workflows.

TrustLoopGuard★ 15

Real-time guardrail runtime for AI agents.

chatbot-guardrails★ 15

💂🏼 Build your Documentation AI with Nemo Guardrails

securing-agentic-ai-developer-day★ 15

Securing Agentic AI Developer Day shows developers how to take an agentic AI reference workflow to production securely.

linkml-reference-validator★ 15

Validate that supporting text quotes in your data actually appear in their cited references

toolguard★ 15

The "Cloudflare for AI Agents". 7-layer security interceptor, real-time observability dashboard, and automated reliability testing for MCP and AI tool chains. Prevent hallucinations, prompt injection, and destructive tool calls.

vre★ 15

Volute Reasoning Engine (VRE): Structural Epistemic Enforcement for Agentic AI

openparallax★ 15

AI agents that think must never act. Open-source security framework with composable modules for safe autonomous AI.

sigil★ 15

Fleet AI Security Posture Management (AI-SPM): client agents on each developer machine score their AI coding agents' guard surfaces (Claude Code, Cursor, Codex, Gemini CLI — permissions, hooks, sandboxes, mcp.json) and ship hash-anchored events to a central server + your SIEM. Fleet-wide posture; measures, doesn't block. Rust.

secure-agents-md★ 15

Security working agreements for AI coding agents: hardened AGENTS.md, prompt/tool-injection guardrails, dependency hygiene, Scorecard-ready OSS setup

platform★ 15

The approval and accountability layer for agentic AI. Identity → Policy → Approval → Trace. Try: npx sidclaw-demo

GenAI-Security-Crosswalk★ 15

ARCHIVED — moved to github.com/GenAI-Security-Project/crosswalk. This copy is frozen as of June 2026 and out of date.

SafeGEO★ 15

EMNLP 2026 | SafeGEO benchmarks Generative Engine Optimization (GEO) attacks on LLM recommendation agents, with a released dataset and prompt/input-level mitigation study.

quick-guardrails-ia★ 15

Guardrails para desenvolver com agentes de IA: gate de CI bloqueante, hooks determinísticos, supply chain travada e fluxo de task com evidência. De 0 a funcionando sem orçamento

securevector-ai-threat-monitor★ 15

Security & Observability for AI Agents — audit every tool call, enforce allow/block policies, catch prompt injection and data leaks. Local-first; works with Claude Code, Codex, Copilot CLI, OpenClaw, LangChain, and all major LLM APIs.

openagentprimitives★ 15

A secure way to run enterprise AI agents. Building blocks with per-action authorization, prompt-injection containment, least privilege, and a tamper-evident audit log.

openleash★ 14

An open-source authorization layer where owners set policies, agents ask before acting, and counterparties can verify the agent was authorized.

ai-guardian★ 14

🛡️ Open-source LLM security library for Python. Protect your AI apps from prompt injection, data leakage, and harmful outputs in 3 lines of code.

aip★ 14

Cryptographic identity, trust chains, and E2E encrypted messaging for AI agents. pip install aip-identity

veto★ 14

The authorization kernel for AI agents. Block, allow, or escalate agent tool calls with YAML rules — deterministic-first, LLM fallback.

AgentForensics★ 14

AgentForensics is an open-source security framework that monitors complete LLM agent sessions in real time, detecting prompt injection attacks across tool outputs, web pages, documents, and API responses. It uses heuristic rules, a DistilBERT ML classifier, instruction boundary detection, semantic drift, and sliding-window multi-turn detection.

GRITS★ 14

Open-source AI agent security framework. Score, harden, and govern your AI agents. Built on DoD/DISA zero-trust principles.

AgentAlign★ 14

AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models

AgentCloak★ 14

Secure email proxy for AI agents — content filtering, PII redaction, and prompt injection detection over MCP

intentframe★ 14

Structural security, governance and execution for AI agents: validate actions before they touch files, credentials, network, APIs, or money.

shark★ 14

Open Source Single Binary Authentication for the Agentic Era.

decern★ 14

Deterministic authorization + tamper-evident audit for AI agents, humans, and workloads — one principal type, safety invariants machine-checked by an SMT solver (cvc5), every decision independently verifiable offline. Pure Rust.

More in Security & AI safety