Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 4 of 15

Qwen2.5-Coder-1.5B-Q8_0-GGUF★ 28

transformers gguf code qwen qwen-coder codeqwen

Unsloth-Ornith-1.5-35B-A3B★ 28

gguf llama.cpp imatrix unsloth-dynamic quantized qwen35moe

AAOSP★ 28

[Archived] AAOSP, predecessor to JibarOS. Early experiment in OS-level on-device AI (MCP + local LLM on Android 15). See JibarOS for the active project.

on-device-llm★ 28

Private, offline AI for Android and Windows: run local LLMs (Gemma 4, Qwen, Ternary Bonsai, any GGUF) with llama.cpp on CUDA or Vulkan, create images and videos on your own GPU, talk hands-free with on-device voice, run parallel agents, cite web research, pair phone and PC, serve an OpenAI-compatible API. No account, no cloud.

ollachat★ 27

A minimal Chatbot GUI for Ollama models - Streamlit App

Muse-Glimmer★ 27

Introducing Muse Glimmer: open-weight 30B agentic multimodal model that runs on your device (Meta). Interactive local agent lab + guide. Apache 2.0 · on-device AI · function calling

langchain-gemma-ollama-chainlit★ 26
ollama-javascript-playground★ 26

Generative AI playground using Ollama, OpenAI API and JavaScript. Try AI models in your browser!

ElBruno.LocalLLMs★ 26

C# local LLM chat completions library using ONNX Runtime, compatible with Microsoft.Extensions.AI

Yi-1.5-34B-Chat-GGUF★ 26

gguf text-generation license:apache-2.0 endpoints_compatible region:us imatrix

gemma-4-E4B-it-assistant-GGUF★ 26

gguf atomic-chat gemma gemma4 google llama.cpp

Nvidia-Qwen3.6-27B-NVFP4-GGUF★ 26

gguf chain-of-thought nvfp4 qwen qwen3.6 quantized

dsh-local-ai★ 26

Local-model (Ollama) integration for DeepSeek Harness: discover, pull, remove, and inspect local models, route requests to them by task type or keyword with automatic fallback to the cloud, and get a one-shot status overview via /ollama.

LLAMA_Hub★ 26

An upgraded llama.cpp GUI (https://github.com/ggml-org) local-first cloud model llama.cpp GUI multi agent command center with RAG, MCP tools, browser automation, voice, and multi-provider orchestration. Demo Here https://llamahub.netlify.app/

Open-o3★ 25

Run o3-pro on your computer. 🌌

cluaiz★ 25

Open-source local AI platform for building, running, and extending AI systems with one unified runtime.

droid-ai-toolkit★ 25

High-performance toolkit for running AI tools (OpenClaw, Pi Agent, n8n, Ollama, Hermes) natively on Android via Termux.

dsh-portable★ 25

DeepSeek Harness 免安装便携版(Windows):官方 npm 包 + 内置 Node.js,双击 exe 即用,拷贝到任意 64 位 Windows 电脑独立运行

MiniCPM5-1B-Agentic-Tooluse-v3-GGUF★ 25

gguf llama.cpp llama-cpp ollama lm-studio minicpm

llama-cpp-chat-completion-wrapper★ 24

Wrapper around llama-cpp-python for chat completion with LLaMA v2 models.

codellama-chatbot★ 24

A local LLM chatbot using Code Llama

Local-Agent-Studio★ 24

A simple studio to use agent

llama-2-7b-langchain-chat-GGUF★ 24

transformers gguf llama text-generation en es

ollama-chat★ 23

A customizable Python CLI tool for interacting with local Language Models, ensuring data privacy while providing conversation memory and extensibility through plugins and efficient Retrieval-Augmented Generation capabilities with ChromaDB integration. Also compatible with OpenAI API.

hexllama★ 23

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

glm-4-9b-chat-GGUF★ 23

gguf glm chatglm thudm quantized GGUF

DeepSeekHarnessGreen★ 23

DeepSeek Harness绿色整合版,一键安装启动,不污染C盘,一个文件夹里管理,自动更新。DeepSeek Harness Green All-in-One Launcher - double-click to run, all-localized

ollama-langchain-agents★ 22

A collection of intelligent AI agents built using Ollama, LangChain, and local LLMs — including chatbots, voice assistants, web scrapers, document readers, and custom tools. No APIs required. Fully offline.

gemma-4-26B-A4B-it-assistant-bf16★ 22

mlx safetensors gemma4_assistant speculative-decoding mtp gemma

Qwen2.5-Coder-14B-GGUF★ 22

gguf code qwen qwen-coder codeqwen text-generation

gemma-4-31B-it-assistant-GGUF★ 22

gguf atomic-chat gemma gemma4 google llama.cpp

cloudflare-bypass-ai★ 22

Run your own unrestricted 70B+ AI locally · No limits · No filters · One-time payment

adapter-antigravity-local★ 22

A high-performance Paperclip AI adapter for running local Google Antigravity (agy) agents.

ios-local-llm★ 22

OnDevice Local AI Studio — run LLMs on iPhone offline. SwiftUI workbench with MLX, llama.cpp, whisper.cpp, RAG, and authenticated OpenAI/Anthropic/Ollama-compatible local APIs.

Vortelio★ 22

Local AI platform: run LLMs, Stable Diffusion, Whisper, TTS, video & 3D generation offline — open-source Ollama + ComfyUI alternative in one binary.

Athene-V2-Chat-GGUF★ 21

gguf RLHF Nexusflow Athene Chat Model text-generation

Qwen3-Coder-Next-GGUF★ 21

gguf imatrix conversational qwen3_next ik_llama.cpp text-generation

Qwen1.5-4B-Chat-GGUF★ 21

gguf chat text-generation en arxiv:2309.16609 license:other

paged-moe★ 21

Run MoE models bigger than your Mac’s RAM. PagedMoE streams experts from SSD on demand with MLX on Apple Silicon. Benchmarks inside.

PolyOllama★ 20

Run multiple open source large language models concurrently powered by Ollama

go-rwkv.cpp★ 20

A go wrapper around the rwkv.cpp library

Yi-Coder-9B-Chat-GGUF★ 20

transformers gguf text-generation base_model:01-ai/Yi-Coder-9B-Chat base_model:quantized:01-ai/Yi-Coder-9B-Chat license:apache-2.0

Yi-Coder-1.5B-Chat-GGUF★ 20

gguf mistral quantized 2-bit 3-bit 4-bit

rust-norion★ 20

Rust Noiron/Norion self-evolving local LLM runtime engine for non-commercial research deployment

local-llm-performance-profile-switcher★ 20

Windows utility focused on llm performance profiles, generation speed, and low vram profile for local AI workflows.

local-llm-vram-budget-optimizer★ 20

Windows utility focused on vram optimization, llm memory usage, and gpu memory budget for local AI workflows.

CrewAI-Local-Agents-1★ 19

Local LLM set-up

local-llm★ 19

Apple's FoundationModels for the web. Chrome and Safari, macOS and iOS

Qwen3-Coder-480B-A35B-Instruct-4bit★ 19

mlx safetensors qwen3_moe text-generation conversational base_model:Qwen/Qwen3-Coder-480B-A35B-Instruct

Openai_gpt-oss-20b-CODER-NEO-CODE-DI-MATRIX-GGUF★ 19

gguf gpt_oss gpt-oss openai mxfp4 programming

glm-4-9b-chat-1m-GGUF★ 19

gguf glm chatglm thudm quantized GGUF

Qwen2.5-Coder-7B-bnb-4bit★ 19

transformers safetensors qwen2 text-generation unsloth code

Qwen3-coder-REAP-25B-A3B-Rust-GGUF★ 19

gguf code-generation quantization qwen3 qwen-coder cerebras

localbrain★ 19

A free, private, local AI for any app. One command, OpenAI-compatible, no API key.

ollama-models★ 18

A collection of ready to use ollama models

SimpleChatbot★ 18

Models: Deepseek R1 models, Llama3.2, Qwen2.5. Integrations: Ollama, Gradio. Supports Local LLM. Test and deploy the latest LLM models in the fastest and most efficient way

llm-lab★ 18

Docker Compose to install N8N, Openweb UI, Qdrant, Ollama, EvolutionAPI, Flowise, OpenCode, Hermes-agent and many more.

Yi-Coder-9B-Chat-GGUF★ 18

transformers gguf text-generation base_model:01-ai/Yi-Coder-9B-Chat base_model:quantized:01-ai/Yi-Coder-9B-Chat license:apache-2.0

Qwen2.5-Coder-32B-GGUF★ 18

gguf code qwen qwen-coder codeqwen text-generation

LocalMind★ 18

Run powerful AI models entirely on your Android device. 100% offline, private, no API keys. Built with Kotlin, Jetpack Compose, Material 3, and llama.cpp. Download GGUF models from Hugging Face. On-device LLM inference for Android.

MiniCPM5-2B-Claude-Fable5-1-Thinking-Agentic-GGUF★ 18

gguf llama.cpp quantized minicpm5 thinking fable5

kevrai-omni★ 18

一键本地 AI 工作站:模型市场(多源测速+断点续传)、llama.cpp+MNN 双引擎、多芯片检测(NVIDIA/AMD ROCm/华为昇腾)、LLM/TTS/图像/视频/3D/超分、Agent 助手;Electron + 原生 JS + Python sidecar,数据默认本地 / One-stop local AI workstation

models-ai-local-docker★ 17

En este repositorio encontraras de una forma centralizada y facil, para poder correr alguno de tus modelos favortios localmente usando docker

AI_Cactus★ 17

Cactus AI framework plugin for UE5 to run local LLMs at runtime. https://github.com/cactus-compute/cactus

local-llama★ 17

Local Llama project, L³ is an electron app that runs llama 3 models locally

sigil★ 17

A local-first LLM development studio. Build, test, and customize inference workflows with your own models — no cloud, totally local.

obrew-studio-server★ 17

Obrew Server: A self-hostable machine learning engine. Build agents and schedule workflows private to you.

AI_Cactus★ 17

Cactus AI framework plugin for UE5 to run local LLMs at runtime. https://github.com/cactus-compute/cactus

AGmind-macos★ 17

One-command local AI/RAG installer for macOS (Metal): Dify, Open WebUI, Ollama, Weaviate/Qdrant, Postgres, Redis. 230 tests.

h2o-danube3-500m-chat-GGUF★ 17

transformers gguf gpt llm large language model h2o-llmstudio

TinyLlama-1.1B-Chat-v0.2★ 17

transformers pytorch safetensors llama text-generation en

skynetclaw★ 17

An institutional-intelligence operating system: a council of 14 agents that remembers every deliberation, grades its own predictions against reality, and revises what it believes. Runs on your machine — Ollama, llama.cpp, or any cloud API.

ESP8266_openai_api★ 16

Using ChatGPT on the ESP8266 with the OpenAI API

nuxt-ollama-chat★ 16

Nuxt Ollama Chat is an open source and minimal chat UI for Ollama. UI Uses Ollama stream API.

ollama-chat-iced★ 16

A GUI made using iced and rust that allows you to talk to an Ollama AI.

bertin-gpt-j-6B★ 16

gradio region:us

OrionStar-Yi-34B-Chat-Llama-GGUF★ 16

transformers gguf yi text-generation base_model:OrionStarAI/OrionStar-Yi-34B-Chat-Llama base_model:quantized:OrionStarAI/OrionStar-Yi-34B-Chat-Llama

unsensored-llm★ 16

Uncensored-LLM is a fully air-gapped, zero-dependency, plug-and-play Local AI environment designed to run seamlessly from your local hard drive or a portable USB/SSD.

AstraLM★ 16

A high-performance, private, on-device AI chat client built with Flutter. AstraLM runs Large Language Models (LLMs) and Vision models 100% locally on device hardware (CPU & GPU) with zero data leaving your phone.(until you choose to use cloud model which is completely OPTIONAL)

beichen-pi-desktop★ 16

面向本地部署模型的极简 Agent 平台 · A local-first minimalist agent platform with explicit context lifecycle controls.

HermitUI★ 16

Run local LLMs entirely in your browser, with no server, no install and no traces. A private, single-file AI chat UI and AGENT that also works with any OpenAI-compatible API.

local-ai-toolkit★ 15

A self-hosted AI toolkit running locally via Docker Compose, bundling an LLM gateway, workflow automation, and a chat UI — all backed by a shared PostgreSQL database.

llama2-chatbot-cpu★ 15

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.

Eucalyptus-Chat★ 15

A frontend for large language models like 🐨 Koala or 🦙 Vicuna running on CPU with llama.cpp, using the API server library provided by llama-cpp-python. NOTE: I had to discontinue this project because its maintenance takes more time than I can and want to invest. Feel free to fork :)

ask-ollama★ 15

Ask-Ollama is a command-line tool that allows users to interact with the Ollama models directly from the terminal. This tool provides a simple and intuitive way to ask questions and receive responses from Ollama models.

llamacpp.net★ 15

C#/.NET binding of llama.cpp

ChromeChat★ 15

PWA chat with Gemini Nano running locally in Chrome using Prompt API, no server, no API key, works fully offline.

poc-uncensored-language-with-wizard-vicuna★ 15

Uncensored Language Model using FastAPI and Wizard Vicuna 30B (PoC)

offline-chatbot★ 15

ChatGPT style interface for open-source LLMs, allowing completely free, offline, and private AI use. Connect with models downloaded from Ollama, or load models directly in the browser with WebLLM.

gemini-in-chrome-toolkit★ 15

Gemini in Chrome 一键启用与修复工具|Cross-platform toolkit to enable/fix Gemini AI in Chrome on Windows/macOS/Linux. Fixes Gemini button missing, --disable-actor-safety-checks warning, Roxy fingerprint browser pollution, Chrome Sync contamination.

baichuan2-7b-chat-gguf★ 15

gguf baichuan chinese text-generation zh en

Overfit★ 15

Run, fine-tune and serve real LLMs entirely in-process in .NET — on the CPU, on-device, offline. No Python, no GPU, no cloud.

USB-Harness★ 15

deepseek-harness 便携式变体 | 即插即用 U 盘 AI 助手:免安装 LLM Web UI、离线可用、适配中国网络,支持 OpenAI 兼容网关(Ollama / 阿里云百炼 / OpenRouter),Windows / Linux / macOS 三端即开即用

Qwen3.8-Flash-Next-ROCmFP4-FAST-GGUF★ 15

gguf rocmfp4 rocmfpx vulkan strix-halo qwen4exp

pi-ollama★ 14

First-class local model support for pi through Ollama — reliable tools, context management, thinking controls, vision, warm-up, and telemetry.

chimera★ 14

single-executable / library which combines llama.cpp, whisper.cpp, and stable-diffusion.cpp

Qwen3-Coder-Next-MLX-Q9★ 14

mlx safetensors qwen3_next quantized text-generation conversational

Qwen2.5-Coder-32B-Instruct-8bit★ 14

transformers safetensors qwen2 text-generation code codeqwen

infai★ 14

Local-first AI in your terminal. infai: run llama.cpp and vLLM models from one TUI with launch profiles, live telemetry, and Hugging Face downloads. infaiw: a daemon-based agent harness with sidecar loops, MCP, and human-in-the-loop approvals.

LLM-Load-Unload-Ollama★ 13

This is a simple demonstration to show how to keep an LLM loaded for prolonged time in the memory or unloading the model immediately after inferencing when using it via Ollama.

More in Agent frameworks & infrastructure