Agent frameworks & infrastructure
Local LLM runtimes
1,401 agents · most starred first
Local LLM runtimes, page 9 of 15
A unified local AI workspace — chat with GGUF models (Qwen, Llama, Phi), generate images, and share your AI remotely. Features agentic web search.
A unified .NET abstraction layer for multiple Large Language Model (LLM) providers. Nexus-LLM simplifies integration with OpenAI, Anthropic, Google,Nexus and other LLM services through a common interface, making it easy to switch between providers or support multiple providers in your application simultaneously.
gguf text-generation base_model:Qwen/Qwen3-Coder-Next base_model:quantized:Qwen/Qwen3-Coder-Next endpoints_compatible region:us
gguf code qwen qwen-coder codeqwen text-generation
transformers gguf llama text-generation llama-2 code
transformers gguf causal-lm code text-generation en
gguf ollama llama.cpp imatrix moe reasoning
Run models that do not fit in your RAM. Keeps the always-read weights resident, streams the routed experts from disk.
gguf text-generation base_model:Kwaipilot/KAT-Coder-V2.5-Dev base_model:quantized:Kwaipilot/KAT-Coder-V2.5-Dev endpoints_compatible region:us
gguf quantized GGUF imatrix quantization imat
gguf text-generation base_model:01-ai/Yi-1.5-34B-Chat base_model:quantized:01-ai/Yi-1.5-34B-Chat license:other region:us
transformers safetensors qwen2 text-generation code codeqwen
Run Qwen3.8 27B-parameter reasoning model with 128k context on a single RTX 3090 or 4090, locally, with an OpenAI-compatible API.
gguf chat text-generation en license:other endpoints_compatible
gguf qwen3.5 rocmfpx function-calling text-generation base_model:JamieBradfield/qwen3.8-9b-hermes-function-calling-v1
static region:us
On-device AI utility assistant powered by Gemma + LiteRT-LM with Wikipedia summarization and QR code generation.
mlx safetensors qwen3_moe text-generation conversational base_model:Qwen/Qwen3-Coder-30B-A3B-Instruct
gguf text-generation ggml quantized q2_k q3_k_m
gradio region:us
gradio mcp-server region:us
Run local LLMs on limited VRAM safely. Dynamic model sharing and OpenAI API drop-in replacement.
Interactive AI agent built with Ollama that enables conversational interactions with web search capabilities. Features multi-turn conversations, real-time web research, and robust error handling using local LLM models.
litert-lm litert litertlm on-device edge hybrid
Minimal Node.js agent loop for Ollama: local LLMs that read/write/edit files, run shell commands, and call MCP tools. MCP client and server, two dependencies, fully offline.
mlx safetensors llama text-generation apple-silicon 4-bit
Ollux — a local AI assistant runtime for Ollama-powered LLM workflows.
Run local coding agents with full tool access and browser-based monitoring using an uncensored Ollama model on your hardware.
Local LLM does Claude's bulk work: reads logs and files, researches the web, writes test-gated code.
Horizon: a local-first AI workspace with built-in model loading, tools, artifacts, profiles and agent integrations.
LLM, Local LLM Manager, a TUI for setting up LLM service
NEXUS Web LLM is a browser-based local inference app for compatible GGUF and LittleBit models, featuring custom WebGPU acceleration, metadata-driven configuration, 100 model profiles, automatic memory planning, speed tuning, and runtime diagnostics.
static huggingchat region:us
gradio region:us
gradio region:us
gradio region:us
streamlit region:us
gradio region:us
docker region:us
static region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
streamlit region:us
gradio region:us
gradio region:us
docker region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
gradio region:us
mlx safetensors qwen3_5_moe apple-silicon agentworld 2bit
mlx safetensors qwen3_5_moe qwen world-model agent
mlx safetensors qwen3_5_moe mlx-lm quantized 4-bit
mlx safetensors qwen3_5_moe mlx-lm quantized 6-bit
mlx safetensors qwen3_5_moe mlx-lm quantized 4-bit
mlx safetensors qwen3_5_moe qwen world-model agent
gguf chat llama-cpp gguf-my-repo text-generation en
transformers gguf llama text-generation base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0 base_model:quantized:TinyLlama/TinyLlama-1.1B-Chat-v1.0
transformers gguf chatglm llama-cpp gguf-my-repo text-generation
mlx safetensors qwen3_chat qwen3.5 text-generation on-device
gguf mistral quantized 2-bit 3-bit 4-bit
transformers openvino llama text-generation conversational base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0
llama.cpp gguf ollama deepseek-v2 deepseek-v2-lite moe
gguf llama-cpp qwen3.5 conversational persona imatrix
transformers gguf imatrix glm-4-9b-chat text-generation en
gguf text-generation base_model:zai-org/glm-4-9b-chat-1m base_model:quantized:zai-org/glm-4-9b-chat-1m endpoints_compatible region:us
coreml ios iphone gemma multimodal text-generation
gguf qwen text-generation zh en arxiv:2309.16609
gguf quantization imatrix tinyllama apple-silicon llama.cpp
gguf quantized GGUF quantization imat imatrix
gguf text-generation base_model:Qwen/Qwen1.5-0.5B-Chat base_model:quantized:Qwen/Qwen1.5-0.5B-Chat endpoints_compatible region:us
gguf facebook meta pytorch llama llama-3
gguf yi 01-ai instruct finetune chatml
gguf text-generation tr license:llama3 endpoints_compatible region:us
gguf mxfp4 mxfp4-moe text-generation en multilingual
mlx safetensors kimi_k25 moe kimi kimi-k2
gguf llama.cpp quantized text-generation lightweight lmstudio
llama.cpp gguf qwen3 local-ai agentic-ai coding-assistant
llama.cpp gguf qwen3 local-ai agentic-ai coding-assistant
gguf facebook meta pytorch llama llama-3
gguf llama text-generation base_model:01-ai/Yi-Coder-9B-Chat base_model:quantized:01-ai/Yi-Coder-9B-Chat license:apache-2.0
gguf agent tool-calling function-calling on-device llama.cpp
gguf zyr zy-s1 hrm prefixlm cpu
mlx safetensors qwen3 text-generation conversational ja
ninfer qwen3.8 groupwise-int iq3_xxs gsq rco
llama.cpp gguf gemma gemma-4 qat mtp
gguf internlm2 text-generation custom_code base_model:internlm/internlm2_5-1_8b-chat base_model:quantized:internlm/internlm2_5-1_8b-chat
mlx safetensors llama text-generation apple-silicon conversational
llama.cpp gguf cpu-inference q4_k_m qwen3 offline
gguf quantized GGUF quantization imat imatrix
transformers gguf llama text-generation base_model:01-ai/Yi-1.5-6B-Chat base_model:quantized:01-ai/Yi-1.5-6B-Chat
gguf local-llm llama.cpp lm-studio quantized ollama