Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 8 of 15

ollama_desktop_client★ 4

A feature-rich desktop GUI for Ollama with Vision, RAG, and JSON support.

Hermes-Local★ 4

Configurable Windows-native, loopback-only Hermes Agent workstation with CPU/CUDA llama.cpp and Hermes Launcher

Local-LLM★ 4

🛠️ Enable offline NLP workflows with Local-LLM, a lightweight Python library for secure environments using the BERT model, ensuring reproducibility and reliability.

Qwen2.5-Coder-14B-GGUF★ 4

transformers gguf code qwen qwen-coder codeqwen

llama-tierkv★ 4

llama.cpp fork that lets a 27B coding agent run on a 16GB GPU: the KV cache lives on a three-tier desk/bookshelf/archive (VRAM window / host RAM / SSD) with attention-driven page recall, so context reaches 256K without decode slowing down.

Mr.Chatterbox-2.7B-Chat-GGUF★ 4

gguf llama.cpp historical victorian conversational vintage-model

local-llm-platform★ 4

Self-hosted private LLM chat platform with RAG: local open-source models via Ollama/vLLM, FastAPI, pgvector, Redis semantic cache, Redpanda, Prometheus

flow-llm★ 3

Local LLM gateway for Apple Silicon. Works with OpenClaw, Hermes Agent, Claude Code, and Codex (AIRun). No Ollama or LM Studio required.

the-golem-archived-model★ 3

Archived model for a constraint-native local AI runtime exploring energy, rhythm, coherence, and persistent behavior.

BigEd★ 3

A reference architecture demonstrating local-first LLM orchestration for SOC compliance workflows. Built on ~11GB VRAM edge hardware using Ollama, featuring a multi-agent coordination pattern with file-based handoffs, a two-lane integration model (human-initiated vs automated processing), and a HITL approval gate for resource-sensitive operations.

Selene★ 3

Selene é um aplicativo desktop para executar modelos de inteligência artificial localmente. O código-fonte é público e desenvolvido no Brasil 🇧🇷. Converse com modelos, envie imagens e trabalhe com um agente de programação que pode explorar arquivos e executar tarefas com sua autorização. Tudo no seu computador.

CT-2-WebUI★ 3

Local AI assistant with a SvelteKit web UI. Multi-phase pipeline (route → plan → generate → validate) on top of llama.cpp, Ollama, or LM Studio. RAG, web search, no API keys, no cloud.

QueryMate★ 3

A portable, offline-first CLI AI assistant built with C++ and llama.cpp. Features dynamic file context awareness, memory management, and runs entirely from a USB drive without dependencies.

Local-AI-Interface★ 3

Local AI Interface is an Electron desktop application designed to streamline your personal AI stack. It provides a unified, user-friendly hub for Dockerized services like Ollama (for LLMs) and n8n (for workflow automation), with built-in dark mode and easy one-command setup.

langgraph-llama-cpp-starter★ 3

A starter template for building powerful, local, tool-calling LLM agents using LangGraph and llama-cpp-python

lm-studio-filesystem-access★ 3

Allows filesystem access for LM Studio

DIZI-AI★ 3

DIZI-AI is an open-source local AI orchestration platform for developers who want full control over their AI stack. Create intelligent multi-agent workflows, automate tasks, generate code and images, and monitor every step with transparent execution and profiling. No cloud dependency. Just powerful, customizable AI running on your hardware.

modal-for-noobs★ 3

gradio region:us

qwen-agentworld-35b-zerogpu★ 3

gradio region:us

GPT4ALL_CHATBOT★ 3

gradio region:us

Qwythos-9B-Claude-Mythos-5-1M-GGUF★ 3

docker region:us

Gpt4All★ 3

gradio region:us

TheBloke-WizardLM-Uncensored-Falcon-7B-GPTQ★ 3

gradio region:us

GPT-2-Uncensored-Local★ 3

gradio region:us

GPT-2-with_gpu★ 3

gradio region:us

EleutherAI-gpt-j-6b★ 3

gradio region:us

gpt2★ 3

gradio region:us

gpt1-perversed★ 3

gradio region:us

Gpt-2★ 3

gradio region:us

TinyLlama-1.1B-Chat-v1.0-int4-ov★ 3

transformers openvino llama text-generation conversational base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0

local-ai-chat★ 3

Local AI Chat App / ローカル AI チャットアプリ

Portable-Offline-LLM★ 3

Run a fully private, offline AI assistant from a USB flash drive — no installation, no internet, no admin rights required.

Qwen_Qwen3-Coder-30B-A3B-Instruct-GGUF★ 3

transformers gguf TensorBlock GGUF text-generation base_model:Qwen/Qwen3-Coder-30B-A3B-Instruct

ellmos-stack★ 3

Self-hosted local-first AI research stack with Ollama, n8n, USMC & GARDENER memory, task-master, KnowledgeDigest, and Docker Compose

OpenZero-Ministral3-8B-Runtime-Agent-GGUF★ 3

gguf llama-cpp ministral openzero agent-zero local-ai

Taiwan-LLM-7B-v2.0.1-chat-GGUF★ 3

transformers gguf text-generation zh arxiv:2311.17487 license:apache-2.0

KAT-Coder-V2.5-Dev-Cerebellum-GGUF★ 3

gguf GGUF qwen3 qwen quantized cerebellum

KAT-Coder-V2.5-Dev-oQ8e★ 3

mlx safetensors qwen3_5_moe text-generation conversational base_model:Kwaipilot/KAT-Coder-V2.5-Dev

internlm2-limarp-chat-20b-GGUF★ 3

gguf mergekit merge text-generation en dataset:lemonilia/LimaRP

gemma4-12b-mtp-assistant★ 3

gguf gemma4 mtp speculative-decoding draft-model llama.cpp

Pebble-10M-Chat-GGUF★ 3

gguf llama.cpp pebble mamba2 hybrid small-language-model

Qwen3-Coder-30B-A3B-IT-Heretic-Q5_K_M-GGUF★ 3

transformers gguf heretic uncensored decensored abliterated

Sharp-MiniCPM5-2B-MLX-oQ6e★ 3

mlx safetensors llama omlx apple-silicon oq

USB-GPT★ 3

Luna - a portable, offline AI assistant that lives on a USB drive. Plug into any Windows, Mac, or Linux computer and chat privately in your browser. No cloud. No accounts. No data left behind. Powered by Ollama + Open WebUI.

GhostSwitch★ 3

Local AI assistant: one Qwen3.5 model plus a library of LoRA experts, picked for each message and checked in code. Runs on a 16 GB laptop, OpenAI-compatible API.

ollama-pterodactyl-egg★ 3

A production-ready Pterodactyl egg for Ollama. Run a local LLM API server (LLaMA 3, Qwen, Gemma) directly on your Pterodactyl panel with zero configuration.

Cyber-Tiel-Coder-35B-A3B-MTP-TTQ-GGUF★ 3

llama.cpp gguf quantized imatrix moe mtp

Qwen3.8-Flash-Next-Coder-cut240-GGUF★ 3

gguf qwen4_exp moe expert-pruning apple-silicon code

jllama-cli★ 2

A command-line interface for Ollama LLM models, designed for seamless integration with shell scripts and command-line workflows.

llama2★ 2

Using open-source Llama2 as an alternate to OpenAI

are-self-api★ 2

Brain-inspired AI reasoning engine — local LLM swarms on consumer hardware. Django + Celery + Ollama. MIT licensed.

AIMobileEdgeChat★ 2

AI Mobile Edge chat in offline model with Google Gemma4 AI Model

api-provider-for-ollama★ 2

AI Provider for Ollama Local for Wordpress 7.0 AI Conectors

llama-chat★ 2

Lightweight web UI for llama.cpp with dynamic model switching, chat history & markdown support. No GPU required. Perfect for local AI development.

rax-offline-cli★ 2

🚀 Advanced offline AI assistant with one-click installation. Complete privacy, no GPU required. Built by RaxCore.

Acuity★ 2

A llama.cpp based chat interface with a sandboxed environment for the LLM

KovalskyAI★ 2

🧠 A fully self-hosted, universal AI chat interface powered by local LLMs. Designed for both desktop and laptop use — supports CPU and GPU (CUDA) execution. Features real-time streaming responses, clean minimalist UI, and no external server dependencies. Ideal for building your own private, offline AI assistant.

QORA-LLM-3B★ 2

Pure Rust inference engine for the SmolLM3-3B language model. No Python runtime, no CUDA, no external dependencies. Single executable + quantized weights = portable AI on any machine.

lithe★ 2

Small Language Models

langgraph-llama-cpp-starter★ 2

🤖 Build intelligent, offline LLM agents with LangGraph and llama-cpp-python using this starter template for local, private tool-calling applications.

owllm★ 2

Local-first AI workstation. Run open-weight models, fine-tune, orchestrate multi-agent teams. No cloud required.

TunnelPanda★ 2

Tunnel Panda is a modular, secure reverse-proxy that streams local Ollama API and Chrome DB via Cloudflare Tunnel of yours to make it publicly accessible. Now we do have an GUI interface for the configuration.

openwebui-workbench★ 2

Portable OpenWebUI workbench with offline model packages, tools, filters, skills and import artifacts.

Pullama★ 2

Reliable, resumable model puller for Ollama with integrity guarantees

RigMatch.AI★ 2

Speed Dating With Local AI Models

1bit_llama3_instruct_xmad_chatbot★ 2

gradio region:us

DeepSeek-R1-Distill-Qwen-1.5B-Demo-GGUF★ 2

docker region:us

EleutherAI-gpt-j-6B★ 2

gradio region:us

EleutherAI-gpt-neox-20b★ 2

gradio region:us

EleutherAI-gpt-neo-1.3B★ 2

gradio region:us

GPT4All★ 2

gradio region:us

declare-lab-flan-gpt4all-xl★ 2

gradio region:us

anon8231489123-vicuna-13b-GPTQ-4bit-128g★ 2

gradio region:us

chavinlo-gpt4-x-alpaca★ 2

gradio region:us

GPT4ALL★ 2

gradio region:us

GPT4ALL-Falcon★ 2

gradio region:us

Uraion-Agent-Small★ 2

gguf safetensors llama-cpp lm-studio ollama local-llm

gemma-4-12B-it-assistant-MTP-GGUF★ 2

gguf gemma gemma-4 mtp speculative-decoding llama.cpp

Yi-1.5-6B-Chat-GGUF★ 2

gguf text-generation license:apache-2.0 endpoints_compatible region:us conversational

magnum-v3-9b-chatml-GGUF★ 2

gguf text-generation base_model:anthracite-org/magnum-v3-9b-chatml base_model:quantized:anthracite-org/magnum-v3-9b-chatml license:gemma endpoints_compatible

Mistral-7B-v0.3-Chinese-Chat-GGUF★ 2

transformers gguf llama-factory orpo text-generation en

Sailor2-1B-Chat-GGUF★ 2

gguf multilingual sea sailor sft chat

Mistral-Le-Chaton-Fat★ 2

mlx safetensors qwen3_5 8bit qwen3.5 lm-studio

Qwen1.5-110B-Chat-GGUF★ 2

transformers gguf qwen2 text-generation chat en

Llama-2-7B-Chat-GGUF★ 2

transformers gguf facebook meta pytorch llama

TinyLlama-1.1B-Chat-v1.0-GGUF★ 2

transformers gguf mistral quantized 2-bit 3-bit

TinyLlama-1.1B-Chat-v1.0-GGUF★ 2

transformers gguf llama text-generation GGUF conversational

internlm2_5-1_8b-chat-GGUF★ 2

gguf text-generation base_model:internlm/internlm2_5-1_8b-chat base_model:quantized:internlm/internlm2_5-1_8b-chat license:other endpoints_compatible

h2o-danube3-4b-chat-GGUF★ 2

transformers gguf gpt llm large language model h2o-llmstudio

Qwen1.5-MoE-A2.7B-Chat-GGUF★ 2

gguf chat TensorBlock GGUF text-generation en

Sailor2-20B-Chat-GGUF★ 2

gguf mistral quantized 2-bit 3-bit 4-bit

Yi-1.5-9B-Chat-GGUF★ 2

transformers gguf llama text-generation base_model:01-ai/Yi-1.5-9B-Chat base_model:quantized:01-ai/Yi-1.5-9B-Chat

Qwen1.5-1.8B-Chat-GGUF★ 2

transformers gguf qwen2 text-generation chat en

TinySlime-1.1B-Chat-v1.0-GGUF★ 2

gguf OpenAccess AI Collective MPT axolotl PyTorch Transformers

Agents-A1-4B-MLX-4bit★ 2

mlx safetensors qwen3_5 mlx-lm text-generation conversational

Replete-Coder-Instruct-8b-Merged-GGUF★ 2

gguf text-generation license:apache-2.0 endpoints_compatible region:us conversational

claude-failover★ 2

Hit your Claude usage limit or Claude is down? Flip one command and your claude -p scripts keep running on a local MLX model on your Mac. Flip back when the limit resets. Claude stays primary.

local-llm★ 2

A workbench for running large Mixture-of-Experts LLMs locally on consumer hardware with a tight VRAM budget.

yapshell★ 2

A funky CLI that talks to Qwen 3.6 via Ollama with animations and attitude.

zn-llm★ 2

Run large language models locally on Windows. A native Rust desktop app and local LLM API server for GGUF models fully offline, no Python, no Docker, no cloud account. It measures your GPU and the model file, then loads a configuration that fits. Tuned on an 8 GB laptop card.

More in Agent frameworks & infrastructure