Agent frameworks & infrastructure
Local LLM runtimes
1,401 agents · most starred first
Local LLM runtimes, page 4 of 15
transformers gguf code qwen qwen-coder codeqwen
gguf llama.cpp imatrix unsloth-dynamic quantized qwen35moe
[Archived] AAOSP, predecessor to JibarOS. Early experiment in OS-level on-device AI (MCP + local LLM on Android 15). See JibarOS for the active project.
Private, offline AI for Android and Windows: run local LLMs (Gemma 4, Qwen, Ternary Bonsai, any GGUF) with llama.cpp on CUDA or Vulkan, create images and videos on your own GPU, talk hands-free with on-device voice, run parallel agents, cite web research, pair phone and PC, serve an OpenAI-compatible API. No account, no cloud.
A minimal Chatbot GUI for Ollama models - Streamlit App
Introducing Muse Glimmer: open-weight 30B agentic multimodal model that runs on your device (Meta). Interactive local agent lab + guide. Apache 2.0 · on-device AI · function calling
Generative AI playground using Ollama, OpenAI API and JavaScript. Try AI models in your browser!
C# local LLM chat completions library using ONNX Runtime, compatible with Microsoft.Extensions.AI
gguf text-generation license:apache-2.0 endpoints_compatible region:us imatrix
gguf atomic-chat gemma gemma4 google llama.cpp
gguf chain-of-thought nvfp4 qwen qwen3.6 quantized
Local-model (Ollama) integration for DeepSeek Harness: discover, pull, remove, and inspect local models, route requests to them by task type or keyword with automatic fallback to the cloud, and get a one-shot status overview via /ollama.
An upgraded llama.cpp GUI (https://github.com/ggml-org) local-first cloud model llama.cpp GUI multi agent command center with RAG, MCP tools, browser automation, voice, and multi-provider orchestration. Demo Here https://llamahub.netlify.app/
Run o3-pro on your computer. 🌌
Open-source local AI platform for building, running, and extending AI systems with one unified runtime.
High-performance toolkit for running AI tools (OpenClaw, Pi Agent, n8n, Ollama, Hermes) natively on Android via Termux.
DeepSeek Harness 免安装便携版(Windows):官方 npm 包 + 内置 Node.js,双击 exe 即用,拷贝到任意 64 位 Windows 电脑独立运行
gguf llama.cpp llama-cpp ollama lm-studio minicpm
Wrapper around llama-cpp-python for chat completion with LLaMA v2 models.
A local LLM chatbot using Code Llama
A simple studio to use agent
transformers gguf llama text-generation en es
A customizable Python CLI tool for interacting with local Language Models, ensuring data privacy while providing conversation memory and extensibility through plugins and efficient Retrieval-Augmented Generation capabilities with ChromaDB integration. Also compatible with OpenAI API.
A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.
gguf glm chatglm thudm quantized GGUF
DeepSeek Harness绿色整合版,一键安装启动,不污染C盘,一个文件夹里管理,自动更新。DeepSeek Harness Green All-in-One Launcher - double-click to run, all-localized
A collection of intelligent AI agents built using Ollama, LangChain, and local LLMs — including chatbots, voice assistants, web scrapers, document readers, and custom tools. No APIs required. Fully offline.
mlx safetensors gemma4_assistant speculative-decoding mtp gemma
gguf code qwen qwen-coder codeqwen text-generation
gguf atomic-chat gemma gemma4 google llama.cpp
Run your own unrestricted 70B+ AI locally · No limits · No filters · One-time payment
A high-performance Paperclip AI adapter for running local Google Antigravity (agy) agents.
OnDevice Local AI Studio — run LLMs on iPhone offline. SwiftUI workbench with MLX, llama.cpp, whisper.cpp, RAG, and authenticated OpenAI/Anthropic/Ollama-compatible local APIs.
Local AI platform: run LLMs, Stable Diffusion, Whisper, TTS, video & 3D generation offline — open-source Ollama + ComfyUI alternative in one binary.
gguf RLHF Nexusflow Athene Chat Model text-generation
gguf imatrix conversational qwen3_next ik_llama.cpp text-generation
gguf chat text-generation en arxiv:2309.16609 license:other
Run MoE models bigger than your Mac’s RAM. PagedMoE streams experts from SSD on demand with MLX on Apple Silicon. Benchmarks inside.
Run multiple open source large language models concurrently powered by Ollama
A go wrapper around the rwkv.cpp library
transformers gguf text-generation base_model:01-ai/Yi-Coder-9B-Chat base_model:quantized:01-ai/Yi-Coder-9B-Chat license:apache-2.0
gguf mistral quantized 2-bit 3-bit 4-bit
Rust Noiron/Norion self-evolving local LLM runtime engine for non-commercial research deployment
Windows utility focused on llm performance profiles, generation speed, and low vram profile for local AI workflows.
Windows utility focused on vram optimization, llm memory usage, and gpu memory budget for local AI workflows.
Local LLM set-up
Apple's FoundationModels for the web. Chrome and Safari, macOS and iOS
mlx safetensors qwen3_moe text-generation conversational base_model:Qwen/Qwen3-Coder-480B-A35B-Instruct
gguf gpt_oss gpt-oss openai mxfp4 programming
gguf glm chatglm thudm quantized GGUF
transformers safetensors qwen2 text-generation unsloth code
gguf code-generation quantization qwen3 qwen-coder cerebras
A free, private, local AI for any app. One command, OpenAI-compatible, no API key.
A collection of ready to use ollama models
Models: Deepseek R1 models, Llama3.2, Qwen2.5. Integrations: Ollama, Gradio. Supports Local LLM. Test and deploy the latest LLM models in the fastest and most efficient way
Docker Compose to install N8N, Openweb UI, Qdrant, Ollama, EvolutionAPI, Flowise, OpenCode, Hermes-agent and many more.
transformers gguf text-generation base_model:01-ai/Yi-Coder-9B-Chat base_model:quantized:01-ai/Yi-Coder-9B-Chat license:apache-2.0
gguf code qwen qwen-coder codeqwen text-generation
Run powerful AI models entirely on your Android device. 100% offline, private, no API keys. Built with Kotlin, Jetpack Compose, Material 3, and llama.cpp. Download GGUF models from Hugging Face. On-device LLM inference for Android.
gguf llama.cpp quantized minicpm5 thinking fable5
一键本地 AI 工作站:模型市场(多源测速+断点续传)、llama.cpp+MNN 双引擎、多芯片检测(NVIDIA/AMD ROCm/华为昇腾)、LLM/TTS/图像/视频/3D/超分、Agent 助手;Electron + 原生 JS + Python sidecar,数据默认本地 / One-stop local AI workstation
En este repositorio encontraras de una forma centralizada y facil, para poder correr alguno de tus modelos favortios localmente usando docker
Cactus AI framework plugin for UE5 to run local LLMs at runtime. https://github.com/cactus-compute/cactus
Local Llama project, L³ is an electron app that runs llama 3 models locally
A local-first LLM development studio. Build, test, and customize inference workflows with your own models — no cloud, totally local.
Obrew Server: A self-hostable machine learning engine. Build agents and schedule workflows private to you.
Cactus AI framework plugin for UE5 to run local LLMs at runtime. https://github.com/cactus-compute/cactus
One-command local AI/RAG installer for macOS (Metal): Dify, Open WebUI, Ollama, Weaviate/Qdrant, Postgres, Redis. 230 tests.
transformers gguf gpt llm large language model h2o-llmstudio
transformers pytorch safetensors llama text-generation en
An institutional-intelligence operating system: a council of 14 agents that remembers every deliberation, grades its own predictions against reality, and revises what it believes. Runs on your machine — Ollama, llama.cpp, or any cloud API.
Using ChatGPT on the ESP8266 with the OpenAI API
Nuxt Ollama Chat is an open source and minimal chat UI for Ollama. UI Uses Ollama stream API.
A GUI made using iced and rust that allows you to talk to an Ollama AI.
gradio region:us
transformers gguf yi text-generation base_model:OrionStarAI/OrionStar-Yi-34B-Chat-Llama base_model:quantized:OrionStarAI/OrionStar-Yi-34B-Chat-Llama
Uncensored-LLM is a fully air-gapped, zero-dependency, plug-and-play Local AI environment designed to run seamlessly from your local hard drive or a portable USB/SSD.
A high-performance, private, on-device AI chat client built with Flutter. AstraLM runs Large Language Models (LLMs) and Vision models 100% locally on device hardware (CPU & GPU) with zero data leaving your phone.(until you choose to use cloud model which is completely OPTIONAL)
面向本地部署模型的极简 Agent 平台 · A local-first minimalist agent platform with explicit context lifecycle controls.
Run local LLMs entirely in your browser, with no server, no install and no traces. A private, single-file AI chat UI and AGENT that also works with any OpenAI-compatible API.
A self-hosted AI toolkit running locally via Docker Compose, bundling an LLM gateway, workflow automation, and a chat UI — all backed by a shared PostgreSQL database.
A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.
A frontend for large language models like 🐨 Koala or 🦙 Vicuna running on CPU with llama.cpp, using the API server library provided by llama-cpp-python. NOTE: I had to discontinue this project because its maintenance takes more time than I can and want to invest. Feel free to fork :)
Ask-Ollama is a command-line tool that allows users to interact with the Ollama models directly from the terminal. This tool provides a simple and intuitive way to ask questions and receive responses from Ollama models.
C#/.NET binding of llama.cpp
PWA chat with Gemini Nano running locally in Chrome using Prompt API, no server, no API key, works fully offline.
Uncensored Language Model using FastAPI and Wizard Vicuna 30B (PoC)
ChatGPT style interface for open-source LLMs, allowing completely free, offline, and private AI use. Connect with models downloaded from Ollama, or load models directly in the browser with WebLLM.
Gemini in Chrome 一键启用与修复工具|Cross-platform toolkit to enable/fix Gemini AI in Chrome on Windows/macOS/Linux. Fixes Gemini button missing, --disable-actor-safety-checks warning, Roxy fingerprint browser pollution, Chrome Sync contamination.
gguf baichuan chinese text-generation zh en
Run, fine-tune and serve real LLMs entirely in-process in .NET — on the CPU, on-device, offline. No Python, no GPU, no cloud.
deepseek-harness 便携式变体 | 即插即用 U 盘 AI 助手:免安装 LLM Web UI、离线可用、适配中国网络,支持 OpenAI 兼容网关(Ollama / 阿里云百炼 / OpenRouter),Windows / Linux / macOS 三端即开即用
gguf rocmfp4 rocmfpx vulkan strix-halo qwen4exp
First-class local model support for pi through Ollama — reliable tools, context management, thinking controls, vision, warm-up, and telemetry.
single-executable / library which combines llama.cpp, whisper.cpp, and stable-diffusion.cpp
mlx safetensors qwen3_next quantized text-generation conversational
transformers safetensors qwen2 text-generation code codeqwen
Local-first AI in your terminal. infai: run llama.cpp and vLLM models from one TUI with launch profiles, live telemetry, and Hugging Face downloads. infaiw: a daemon-based agent harness with sidecar loops, MCP, and human-in-the-loop approvals.
This is a simple demonstration to show how to keep an LLM loaded for prolonged time in the memory or unloading the model immediately after inferencing when using it via Ollama.