Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 9 of 15

AI_GUI★ 2

A unified local AI workspace — chat with GGUF models (Qwen, Llama, Phi), generate images, and share your AI remotely. Features agentic web search.

Nexus-LLM★ 2

A unified .NET abstraction layer for multiple Large Language Model (LLM) providers. Nexus-LLM simplifies integration with OpenAI, Anthropic, Google,Nexus and other LLM services through a common interface, making it easy to switch between providers or support multiple providers in your application simultaneously.

Qwen.Qwen3-Coder-Next-GGUF★ 2

gguf text-generation base_model:Qwen/Qwen3-Coder-Next base_model:quantized:Qwen/Qwen3-Coder-Next endpoints_compatible region:us

Rombos-Coder-V2.5-Qwen-7b-GGUF★ 2

gguf code qwen qwen-coder codeqwen text-generation

CodeLlama-70b-Instruct-hf-GGUF★ 2

transformers gguf llama text-generation llama-2 code

stable-code-instruct-3b-GGUF★ 2

transformers gguf causal-lm code text-generation en

Agents-A1-ONYX-GGUF★ 2

gguf ollama llama.cpp imatrix moe reasoning

Chaos★ 2

Run models that do not fit in your RAM. Keeps the always-read weights resident, streams the routed experts from disk.

KAT-Coder-V2.5-Dev-MXFP4_MOE-GGUF★ 2

gguf text-generation base_model:Kwaipilot/KAT-Coder-V2.5-Dev base_model:quantized:Kwaipilot/KAT-Coder-V2.5-Dev endpoints_compatible region:us

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF★ 2

gguf quantized GGUF imatrix quantization imat

Yi-1.5-34B-Chat-GGUF★ 2

gguf text-generation base_model:01-ai/Yi-1.5-34B-Chat base_model:quantized:01-ai/Yi-1.5-34B-Chat license:other region:us

Qwen2.5-Coder-14B-Instruct-8bit★ 2

transformers safetensors qwen2 text-generation code codeqwen

escha-qwen3.8-27b-w2-docker★ 2

Run Qwen3.8 27B-parameter reasoning model with 128k context on a single RTX 3090 or 4090, locally, with an OpenAI-compatible API.

Qwen1.5-110B-Chat-GGUF★ 2

gguf chat text-generation en license:other endpoints_compatible

qwen3.8-9b-hermes-function-calling-v1-GGUF★ 2

gguf qwen3.5 rocmfpx function-calling text-generation base_model:JamieBradfield/qwen3.8-9b-hermes-function-calling-v1

Local-LLM-Hardware-Explorer★ 2

static region:us

Gemma-4-blr★ 2

On-device AI utility assistant powered by Gemma + LiteRT-LM with Wikipedia summarization and QR code generation.

Qwen3-Coder-30B-A3B-Instruct-3bit★ 2

mlx safetensors qwen3_moe text-generation conversational base_model:Qwen/Qwen3-Coder-30B-A3B-Instruct

Llama-68M-Chat-v1-GGUF★ 2

gguf text-generation ggml quantized q2_k q3_k_m

ollama-private-llm★ 2

gradio region:us

llm-zero-gpu-playground★ 2

gradio mcp-server region:us

kineticLLM★ 2

Run local LLMs on limited VRAM safely. Dynamic model sharing and OpenAI API drop-in replacement.

ollama-agents★ 2

Interactive AI agent built with Ollama that enables conversational interactions with web search capabilities. Features multi-turn conversations, real-time web research, and robust error handling using local LLM models.

Agents-A1-4B★ 2

litert-lm litert litertlm on-device edge hybrid

ollama-agent-kit★ 2

Minimal Node.js agent loop for Ollama: local LLMs that read/write/edit files, run shell commands, and call MCP tools. MCP client and server, two dependencies, fully offline.

MiniCPM5-2B-chat-mlx-4bit★ 2

mlx safetensors llama text-generation apple-silicon 4-bit

ollux★ 2

Ollux — a local AI assistant runtime for Ollama-powered LLM workflows.

OllamaHarness★ 2

Run local coding agents with full tool access and browser-based monitoring using an uncensored Ollama model on your hardware.

localize★ 2

Local LLM does Claude's bulk work: reads logs and files, researches the web, writes test-gated code.

horizon★ 2

Horizon: a local-first AI workspace with built-in model loading, tools, artifacts, profiles and agent integrations.

llm★ 2

LLM, Local LLM Manager, a TUI for setting up LLM service

Nexus-Web-LLM★ 2

NEXUS Web LLM is a browser-based local inference app for compatible GGUF and LittleBit models, featuring custom WebGPU acceleration, metadata-driven configuration, 100 model profiles, automatic memory planning, speed tuning, and runtime diagnostics.

hybrid-edge-ai-agent-android-arm-pwa★ 1

static huggingchat region:us

my_local_AI_assistant★ 1

gradio region:us

LLMs-Stable-Vicuna-13B★ 1

gradio region:us

totally-not-an-llm-AlpacaCielo2-7b-8k-GGML★ 1

gradio region:us

can-it-run-llm★ 1

streamlit region:us

ikaankeskin-llma-2-7b-kkplatypus★ 1

gradio region:us

LLM-34B-GGUF-Chat-V1★ 1

docker region:us

mlc_llm_Modify★ 1

static region:us

cerebras-Cerebras-GPT-1.3B★ 1

gradio region:us

DanielPinheiro-gpt4all★ 1

gradio region:us

chavinlo-gpt4-x-alpaca★ 1

gradio region:us

gpt2.0★ 1

gradio region:us

anon8231489123-vicuna-13b-GPTQ-4bit-128g★ 1

gradio region:us

Gpt4All★ 1

gradio region:us

gpt2★ 1

gradio region:us

chavinlo-gpt4-x-alpaca★ 1

gradio region:us

GPT4ALL1★ 1

gradio region:us

NorPaca_GPT★ 1

streamlit region:us

Gpt4all★ 1

gradio region:us

nomic-ai-gpt4all-j★ 1

gradio region:us

chavinlo-gpt4-x-alpaca★ 1

docker region:us

gpt2-xl★ 1

gradio region:us

nomic-ai-gpt4all-13b-snoozy★ 1

gradio region:us

EleutherAI-gpt-j-6b★ 1

gradio region:us

GPT4ALL★ 1

gradio region:us

GPT4ALL★ 1

gradio region:us

Qwen-AgentWorld-35B-A3B-MLX-2bit★ 1

mlx safetensors qwen3_5_moe apple-silicon agentworld 2bit

Qwen-AgentWorld-35B-A3B-MLX-oQ3.5★ 1

mlx safetensors qwen3_5_moe qwen world-model agent

Qwen-AgentWorld-35B-A3B-4bit★ 1

mlx safetensors qwen3_5_moe mlx-lm quantized 4-bit

SuperQwen-AgentWorld-35B-A3B-abliterated-6bit★ 1

mlx safetensors qwen3_5_moe mlx-lm quantized 6-bit

Agents-A1-4bit★ 1

mlx safetensors qwen3_5_moe mlx-lm quantized 4-bit

Qwen-AgentWorld-35B-A3B-MLX-oQ4★ 1

mlx safetensors qwen3_5_moe qwen world-model agent

Qwen2.5-3B-Instruct-Q4_0-GGUF★ 1

gguf chat llama-cpp gguf-my-repo text-generation en

TinyLlama-1.1B-Chat-v1.0-GGUF★ 1

transformers gguf llama text-generation base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0 base_model:quantized:TinyLlama/TinyLlama-1.1B-Chat-v1.0

glm-4-9b-chat-hf-Q4_K_M-GGUF★ 1

transformers gguf chatglm llama-cpp gguf-my-repo text-generation

Qwen3.5-0.8B-Chat-MLX★ 1

mlx safetensors qwen3_chat qwen3.5 text-generation on-device

Llama3.1-8B-Chinese-Chat-GGUF★ 1

gguf mistral quantized 2-bit 3-bit 4-bit

TinyLlama-1.1B-Chat-v1.0-int8-ov★ 1

transformers openvino llama text-generation conversational base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0

DeepSeek-V2-Lite-Chat-ReMoE-GGUF★ 1

llama.cpp gguf ollama deepseek-v2 deepseek-v2-lite moe

Qwen3.5-4B-ChatMDE-v1.0-GGUF★ 1

gguf llama-cpp qwen3.5 conversational persona imatrix

glm-4-9b-chat-imatrix-GGUF★ 1

transformers gguf imatrix glm-4-9b-chat text-generation en

THUDM.glm-4-9b-chat-1m-GGUF★ 1

gguf text-generation base_model:zai-org/glm-4-9b-chat-1m base_model:quantized:zai-org/glm-4-9b-chat-1m endpoints_compatible region:us

gemma-4-12b-coreml-iphone-practical-chat★ 1

coreml ios iphone gemma multimodal text-generation

Qwen-14B-Chat-GGUF-Quantization★ 1

gguf qwen text-generation zh en arxiv:2309.16609

TinyLlama-1.1B-Chat-v1.0-GGUF-MacQuantized★ 1

gguf quantization imatrix tinyllama apple-silicon llama.cpp

internlm2_5-7b-chat-1m-IMat-GGUF★ 1

gguf quantized GGUF quantization imat imatrix

Qwen.Qwen1.5-0.5B-Chat-GGUF★ 1

gguf text-generation base_model:Qwen/Qwen1.5-0.5B-Chat base_model:quantized:Qwen/Qwen1.5-0.5B-Chat endpoints_compatible region:us

Llama-3-Groq-70B-Tool-Use-GGUF★ 1

gguf facebook meta pytorch llama llama-3

Yi-1.5-34B-Chat-iMat-GGUF★ 1

gguf yi 01-ai instruct finetune chatml

Trendyol-LLM-8b-chat-v2.0-GGUF★ 1

gguf text-generation tr license:llama3 endpoints_compatible region:us

Qwen3-Coder-30B-A3B-MXFP4-MOE-GGUF★ 1

gguf mxfp4 mxfp4-moe text-generation en multilingual

Kimi-K2.7-Code-MLX-4bit-hiprec★ 1

mlx safetensors kimi_k25 moe kimi kimi-k2

DeepSeek-Coder-V2-Lite-Instruct_GGUF★ 1

gguf llama.cpp quantized text-generation lightweight lmstudio

OpenZero-Fusion-Qwen3-4B-Agentic-GGUF★ 1

llama.cpp gguf qwen3 local-ai agentic-ai coding-assistant

OpenZero-Qwen3-1.7B-Agentic-GGUF★ 1

llama.cpp gguf qwen3 local-ai agentic-ai coding-assistant

Llama-3-Groq-8B-Tool-Use-GGUF★ 1

gguf facebook meta pytorch llama llama-3

Yi-Coder-9B-Chat-GGUF★ 1

gguf llama text-generation base_model:01-ai/Yi-Coder-9B-Chat base_model:quantized:01-ai/Yi-Coder-9B-Chat license:apache-2.0

GRAFT-1B-Agentic★ 1

gguf agent tool-calling function-calling on-device llama.cpp

ZY-S1-GGUF★ 1

gguf zyr zy-s1 hrm prefixlm cpu

ABEJA-Qwen3-14B-Agentic-256k-v0.1-MLX-8bit★ 1

mlx safetensors qwen3 text-generation conversational ja

Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-NInfer★ 1

ninfer qwen3.8 groupwise-int iq3_xxs gsq rco

gemma-4-12B-it-qat-assistant-MTP-Q4_0-GGUF★ 1

llama.cpp gguf gemma gemma-4 qat mtp

internlm2_5-1_8b-chat-GGUF★ 1

gguf internlm2 text-generation custom_code base_model:internlm/internlm2_5-1_8b-chat base_model:quantized:internlm/internlm2_5-1_8b-chat

MiniCPM5-2B-chat-mlx-16bit★ 1

mlx safetensors llama text-generation apple-silicon conversational

tiny-agent-gguf★ 1

llama.cpp gguf cpu-inference q4_k_m qwen3 offline

DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF★ 1

gguf quantized GGUF quantization imat imatrix

Yi-1.5-6B-Chat-GGUF★ 1

transformers gguf llama text-generation base_model:01-ai/Yi-1.5-6B-Chat base_model:quantized:01-ai/Yi-1.5-6B-Chat

colgrep-agent-2B-imatrix-GGUF★ 1

gguf local-llm llama.cpp lm-studio quantized ollama

More in Agent frameworks & infrastructure