Agent frameworks & infrastructure
Local LLM runtimes
1,401 agents · most starred first
Local LLM runtimes, page 2 of 15
PocketStrike is a privacy-first mobile AI workstation. It combines zero-cloud native on-device GGUF inference (ARM NEON accelerated) │ with frontier cloud LLMs, multi-step ReAct agent tools, Model Context Protocol (MCP) servers, self-evolving cross-session memory, and FLUX neural image │ generation.
Booster - open accelerator for LLM models. Better inference and debugging for AI hackers
transformers gguf unsloth qwen3 qwen text-generation
transformers gguf llama text-generation llama-factory orpo
gguf llama.cpp imatrix spark2_5 long-context coding
gguf quantized moe vlm vision agentic
Local AI desktop app: a Chromium browser with a GPU-aware llama.cpp runtime, OpenAI-compatible API, AI chat and agents, Stable Diffusion image/video, TTS and music, plus an MCP server so Claude Code, Codex and Cline can drive your browser. Windows, macOS, Linux.
transformers gguf llama llama-2 text-generation code
Openai-style, fast & lightweight local language model inference w/ documents
AI, fully local on your computer.
デジタル庁のガバメントAI「源内(GENAI)」を完全ローカル(ローカルLLM/OpenAI互換)で動かす非公式プロジェクト。SAML認証(Keycloak)・RAG(Qdrant)・文字起こし(Whisper)・画像生成(SD)・チーム単位ナレッジをローカル完結。
transformers coreml llama text-generation meta llama-2
transformers gguf llama llama-2 text-generation code
transformers gguf llama llama-2 text-generation code
gguf code codeqwen chat qwen qwen-coder
This project is a plugin for LM Studio that provides a rich set of tools to a large language model. It allows the LLM to interact with your local file system, execute code, access the internet, and more. This empowers the LLM to perform complex tasks like code generation, project scaffolding, and web research.
gguf code codeqwen chat qwen qwen-coder
A reference application for a local AI assistant with LLM and RAG
Web UI for Ollama built in Java with Vaadin, Spring Boot and Ollama4j
Tool calling using MLX Swift across iOS, macOS, and visionOS platforms
Local AI Coding Hub: Run Claude Models Offline in 2026
Local Reasoning Agent using LangChain + Ollama
Hyprdots, Hyprland Omarchy, Quickshell, Agentic Local-Ai, Custom Integrated, Minimal
transformers gguf llama facebook meta pytorch
Run OpenClaw on a local model. Zero API cost. One OpenAI-compatible endpoint over Ollama, llama.cpp, vLLM or LM Studio — plus AirLLM for models bigger than your GPU.
Modern desktop application (Rust + Tauri v2 + Svelte 5 + Candle (HF)) for communicating with AI models that runs completely locally on your computer. No subscriptions, no data sent to the internet — just you and your personal AI assistant
One app for both cloud LLMs and on-device LLMs.
LocalChat is a ChatGPT-like chat that runs on your computer
gguf chat text-generation en arxiv:2309.16609 license:other
Run OpenClaw AI agent with zero API cost, local LLM via AirLLM
InferrLM - On-device AI for iOS & Android
A personal assistant that remembers you, in one single executable: connect the API of your choice (OpenAI, Anthropic, Mistral, Ollama…). Memory, tasks, apps, browser control and tools it builds itself. Windows, macOS, Linux.
gguf chat abliterated uncensored text-generation en
A simple experiment on letting two local LLM have a conversation about anything!
PasLLM - LLM inference engine in Object Pascal (synced from my private work repository)
Practical guide to running LLMs locally on Apple Silicon, with a focus on architecture, memory scaling, and agent workflows.
transformers gguf llama llama-2 text-generation code
A private Claude-Code-style coding agent for Apple Silicon — run chat, code, and local model workflows on-device. MLX-native, Ollama/OpenAI API compatible, zero API keys.
Selfhost modern LLM stacks. Run the whole fleet from your terminal
MimiModel: Agentic LLM on a $5 chip. 100% on-device. Private by design.
gguf llama.cpp llama-cpp minicpm minicpm5 minicpm5-1b
Gradio based tool to run opensource LLM models directly from Huggingface
Eris is a private AI chat application that runs entirely on your device using Apple's MLX framework. Named after the dwarf planet that challenged our understanding of the solar system, Eris challenges the notion that AI must live in the cloud.
Premium, zero-configuration local AI inference & agentic workspace running completely offline on Windows, macOS, and Linux.
Guide on how to set up LMStudio and VSCode Continue.dev extension as a local and private coding Copilot. Targeted for 32GB RAM Apple Silicon devices.
🦙 Opilot: Ollama for GitHub Copilot VS Code Extension – Run Ollama models locally and in the cloud within GitHub Copilot Chat
🦙 Opilot: Ollama for GitHub Copilot VS Code Extension – Run Ollama models locally and in the cloud within GitHub Copilot Chat
InferrLM - On-device AI for iOS & Android
RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B:NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。
gguf rocmfp4 rocmfpx vulkan strix-halo qwen4exp
The easiest & fastest way to run LLMs in your home lab
gguf qwen3.8-flash-next qwen4exp code experimental agentionai
On-device AI SDK for Flutter — LLM inference, vision, STT, TTS, image generation, embeddings, RAG, and function calling. Metal GPU on iOS/macOS.
A wannabe Ollama equivalent for Apple MlX models
mlx llama facebook meta llama-2 text-generation
A sovereign, zero-trust local AI agent and micro-IDE.
Gemma models running 100% local on an 8 GB Jetson Orin Nano: voice assistants with RAG, multi-agent NPCs, live EN-JA translation, and a Gemma 4 voice + vision agent that decides when to look through the webcam. The demos from the Gemma 2 launch at Google Tokyo.
GPT2⚡NCNN⚡中文对话⚡x86⚡Android
Electron + Next.js desktop AI assistant that runs GGUF models locally using llama.cpp. Designed for offline use, portability, and zero-install deployment.
First LLM Studio: local-first LLM workbench for Apple Silicon + MLX. 本地优先 LLM 工作台,覆盖 Agent、对比、微调、评测与运行时遥测。
Self-hosted agentic AI platform powered by local models
Local AI for Omarchy: the model validated for your GPU, one button on the bar. Start serves it, any coding agent opens on it, one click shares it on your tailnet.
You type. It already knows. Local LLMs that read and think while you type.
Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.
gguf chat text-generation en license:apache-2.0 endpoints_compatible
An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.
Local Agents is an add-on for Godot 4.5 to run LLMs locally in games
Windows port of GreenBoost - GPU VRAM extension with system RAM & NVMe for larger LLMs. Original by Ferran Duarri (GPL v2).
gguf chat text-generation en arxiv:2309.16609 license:other
gguf llama.cpp llama-cpp minicpm minicpm5 minicpm5-1b
This is a PowerShell script that automates the process of setting up and running VICUNA on a CPU (without a graphics card) using the llama.cpp library and a pre-trained ggml-vicuna-13b-4bit.bin model. The script downloads and extracts the required files, creates a batch file to run VICUNA, and creates a desktop shortcut to launch the batch file.
GGUF Loader with its Agentic Mode, and floating button, ai Models | Open Source & Offline. Mistral, Deepseek, llama, gemma, qwen
static region:us
Local LLM agent for vintage Mac hardware. Built on a 2013 Mac Pro trashcan with dual AMD FirePro D500s. Elyan Labs.
An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harness (detailed event steams, logs, and performance data) or quickly replace closed source one (featuring out of the box plugins for web search, image gen, memory, auto research and more in dev).
transformers gguf yi text-generation base_model:01-ai/Yi-34B-Chat base_model:quantized:01-ai/Yi-34B-Chat
Browser LLM Demo (like ChatGPT). Works locally with JavaScript and WebGPU
Cross-Platform High-Level LLM Library
Ollama Client – Chat with Local LLMs Inside Your Browser A lightweight, privacy‑first extension to chat with local LLMs via Ollama, LM Studio, and llama.cpp. Supports streaming, stop/regenerate, RAG, and easy model switching — all without cloud APIs or data leaks.
PortableLM — Zero-dependency, air-gapped local AI that runs entirely from a USB drive. Chat, generate images, and synthesize speech offline on Windows, macOS, or Linux. No installs, no internet, just plug and play.
Android native AI inference library, bringing text, image, video, STT, TTS inference
Tired of paying for frontier AI models? Break free with this guide to 100% local, private LLM code generation on Apple Silicon. Optimized for 24GB+ Macs, it uses Ollama + Cline to build a disciplined, free agentic workflow. Learn to write system rules, configure custom Modelfiles, and run strict two-layer test suites.
gguf chat text-generation en arxiv:2309.16609 license:other
transformers gguf agent text-generation en base_model:janhq/Jan-code-4b
OpenClaw with Ollama is a lightweight, high-performance integration designed to run sophisticated AI agents locally on Android devices. By leveraging the power of Termux and the flexibility of Ollama, this project enables a completely private, free, and offline-capable AI workstation in your pocket.
A local model engine with a search engine and web crawler built in. Run your coding agents on your own GPUs, and get cited answers from your files, code, and the sites you crawl. One executable.
Local-first AI toolkit for the modern web. Build privacy-first AI applications that work entirely in the browser.
transformers gguf liquid lfm2 edge text-generation
gguf llama.cpp imatrix minicpm long-context coding
LLM inference in Fortran
llama.cpp gguf 27b jack jack-xml agentic
gguf chat text-generation en arxiv:2309.16609 license:other
Python bindings for llama.cpp
Ollama MCP Agent allows you to use LLM models locally on your PC for free along with MCP additional features
The Web AI Toolkit is a powerful, privacy-first JavaScript library that brings advanced AI capabilities directly to your web applications. Run OCR, speech-to-text, text summarization, image classification, and more — all locally in the browser with no data sent to external servers.
Self-contained offline environment providing local AI chat, offline Wikipedia/content archives, IRC communication, audio streaming, file server, and development tools. Designed for zero internet dependency - download once, run anywhere. Perfect for remote areas, emergency scenarios, or escaping surveillance capitalism.
Use self-hosted LLMs with an OpenAI compatible API
Make AI Free Again - Shard is a GUI for Ollama LLM's. Powered by Next.js.