Agent frameworks & infrastructure
Local LLM runtimes
1,401 agents · most starred first
Local LLM runtimes, page 14 of 15
Route a Hermes Agent's local Ollama traffic through the native /api/chat API (honors num_ctx + correct tool-call streaming) instead of the OpenAI-compat /v1 endpoint.
Lightweight web admin panel for llama.cpp server management
htop for local AI inference — curses-based TUI monitor for Ollama, llama-swap/vLLM and ComfyUI: models, GPU and server
Ollama Target adapter for Waken — routes waken Events to a local/self-hosted Ollama daemon.
A private reasoner for your terminal. Direct conversation or grounded multi-agent research, GPU-native and fully local. No API keys, no inference servers.
Local-first LLM instance framework: deploy, specialize, and chat with grounded, cited model instances.
GoNext CLI — the gonext terminal plus the local worker that runs agent / OCR / PDF / embedding jobs on your Mac (Ollama / MLX / OpenAI-compatible).
Internal component of Sutra SDK — applications should install sutra-sdk. The Edge Harness. Offline-first autonomous agent runtime for local SLMs (Phi-3, Gemma, quantized GGUF/ONNX) with durable friction telemetry and on-device vector memory.
A comprehensive set of agentic tools for LM Studio: file operations, web fetching/search, memory, SSH, and an autonomous vibe_bridge keep-alive loop.
Transformers.js provider for Vercel AI SDK - Run 🤗 Transformers directly in the browser.
A corporation of character AI agents running on local LLMs — each agent with its own persona and app set. Name reservation; developed privately.
Bharat-Tiny-LLM - Offline Hindi/Hinglish AI assistant for Apple Silicon. `bharat chat` and go. 33.8% token compression via Brahmi injection.
CHP capability adapter — local LLM inference via Ollama (primary) or llama.cpp (fallback)
Installer and CLI for the local DiffusionGemma coding agent runtime
ethical, local-only inference wrapper for llama.cpp
🦙 Exposed Ollama Instance Hunter, Proxy & API Manipulation Tool
Small GB10/GX10 helpers for loading LLMs without slow cold-page CUDA H2D transfers.
Keeps your local LLM contexts warm: persistent KV-cache proxy for llama.cpp's llama-server
A secure Windows-native DLL LLM tool for enterprises.
LangChain integrations with Apple Foundation Model
Ollama model components as a standalone Langflow Extension Bundle.
Find, download (over BitTorrent + webseeds), and SHA-256-verify LLM models from the LLM Torrents catalog.
CLI tool to manage local LLM models and role-based server lifecycle
LoRA adapter bridge between Unsloth and Ollama for dynamic adapter loading.
The LLM stack you can read and understand: Gemma 4 and Llama on CUDA, built from explicit C++23 components.
omllm
A unified Python interface and API server for running local Large Language Models.
Deploy local LLMs behind your own managed Anthropic-compatible API server - vLLM, llama.cpp, or SGLang in isolated environments, with API keys, workspaces, and usage tracking.
A terminal user interface (TUI) for SDG Agents — powered by local AI models
Pinned Vulkan llama-server build for Skulk's served GGUF engine (Linux x86_64)
LangChain.js adapters for @localmode — drop-in local inference for existing LangChain apps
Standardized compatibility checker for local Ollama model tool-calling support
Pre-flight checker for local Ollama models
Placeholder to reserve this name — not the real package.
In-process llama.cpp VLM inference engine for MinerU
Recommend, download, and run Ollama models based on your system specs and use case
A local-first, agentic shell for Ollama — chat, tools, and an MCP-style agent loop that never phones home.
Aclade Local AI Agent Connector
Force local Ollama models to output strict, valid JSON. Auto-strips markdown fences and retries on failure.
DeepSeek Harness model provider for virtualized local CLI agents
Pre-flight connectivity checks for the n8n + Ollama training stack.
Is your llama.cpp decode memory-bandwidth-bound? Measure it, in one command.
Local LLM serving for a single RTX 3090, with a browser control panel
Work out which AI models your computer can run, fine-tune, or train.
Autopilot for local AI: evidence-driven planning, controlled setup, and measured verification.
Fail-safe client discipline for local LLMs on memory-constrained machines: preflight memory checks, starvation-aware retries, thinking-mode-off, token budgets.
RAMP: Resource-Aware Model Proxy - an elastic local LLM daemon that auto-calibrates model size to available RAM, VRAM and disk
Web Endpoint as PI Package for PI Agent (pi.dev), with Ollama API and OpenAI API compability.
Search, download, and import HuggingFace GGUF models into Ollama
Native Ollama integration for Hyle.
Run local llama.cpp servers from a config file, and work out which model to run and how
LLM plugin for models served by oMLX
Lightweight, hardware-independent local GGUF parser and Ollama substitute server with Hugging Face Hub integration.
GGUF runtime, model manager & OpenAI server utilizing device resources for Android Termux & ARM64
A package to combine GBNF grammar and Beam Search for probablistic grammar pruning with configurables beams (both breadth- and depth-first)
Find out which open-source LLMs can run on your hardware.
Inference library for running local LLMs on modern consumer GPUs
Local NVIDIA LLM experimentation workbench
Live prefill/decode progress for a local Ollama server and basic observability metrics
Free local LLM on your Mac: Apple Intelligence's on-device Foundation Models and Private Cloud Compute from Python, plus text to speech and speech to text in macOS's own voices and recognition. Guaranteed JSON (pydantic), streaming, sessions. No API key, no account.
Native GGUF and safetensors inference runtime for Qwen, Laguna, K2-Horizon, Muse, DeepSeek-V4, Gemma and BailingMoE3 models
Hugpy storage: model download queue and daemon, Hugging Face transport and token, resumable transfers, model sync, physical inventory and on-disk model layout
A hunch engine plugin: local models served by Ollama, read through their token probabilities
Find, size, install and verify open-weight LLMs for llama.cpp on your own machine.
LLM plugin for models served by Unsloth Studio
Мы предоставляем Вам удобную возможность - развернуть модель на облаке или на компьютере локально и использовать ее с другой точки мира! Работает это просто - сначала Вызапускаете модель на любом железе, получаете ключ доступа (Local_CreateCode()) и сопряжаете по нему устройства. Далее - работайте по документации (https://llmconstructor-pylibrary.onrender.com/docs).
Find local AI models that fit your hardware, with memory and speed estimates.
Call the Ollama local API by function.Please install Ollama before use.
Dialog transformer for OVOS, designed to return responses from Ollama
Private language model inference with a local Responses API and remote matrix execution
Inference engine for large language models on consumer GPUs with deep optimization for older hardware
Scaffold a local-LLM agent for an RGB Lightning Node with @kaleidorg/mind: npm create @kaleidorg/mind my-agent
LangChain chat model (ChatQVAC) that runs local QVAC completions, for use in LangChain runnables and LangGraph graphs.
WebLLM chat models and embeddings for LangChain, LangGraph, and Deep Agents.
带插件系统的 CLI 大语言模型推理接口
CLAP: Chat Local And Persistent - Local LLM conversation tool with Ollama
Explicit tool-enabled Ollama agent capability for Crewplane
OpenAI-compatible local LLM server, CLI and benchmark over dasLLAMA
CLI to chat with local GGUF LLMs, with MCP tool-calling and SQLite conversation history.
Manage local llama.cpp model servers with a JSON manifest and an interactive TUI.
An integration package connecting llama.cpp's llama-server and LangChain, with reasoning, timings, structured output and tool calling
Hybrid System-One Local Decision Engine (JevCompat 32/32 MUST, AnyJev L0 Prefill, StartLux Read-or-Think Cascade, OpenJev-RLCD Trainer)
One command runs the best local AI your PC can handle: measured model picks + a tuned llama.cpp (AMD, NVIDIA, Intel, Apple)
Hybrid System-One Local Decision Engine (JevCompat 32/32 MUST, AnyJev L0 Prefill, StartLux Read-or-Think Cascade, OpenJev-RLCD Trainer)
Discover, verify and wire local MLX models on Apple Silicon
User-owned Ollama servers in interactive NERSC allocations
Interactive CLI to run Ollama LLMs on Kaggle/Colab GPUs: GPU detection, fit and tokens/sec estimates, library search, public tunnel.
Ollama (local OpenAI-compatible) plugin for PolyLLM.
Run bigger models on smaller machines — RAM-adaptive, GPU-aware LLM inference manager
A live terminal panel for llama.cpp's llama-server on Windows: GPU and shared memory, RAM, requests
llama.cpp backend core for the Windows ML Runtime
JustFit: long-context 27B language-and-vision serving on 24 GB Apple silicon Macs (native, no Python at runtime)
Standalone Ollama and ChatGPT OAuth clients with a shared Python API and CLI
Local DeepSeek twin — a faithful, stateful local DeepSeek Platform API your real `@ai-sdk/deepseek` talks to unmodified. The model is stubbed (deterministic), but the protocol envelope is vendor-faithful and so are DeepSeek's OpenAI-DIVERGENT REFUSALS: 42
GGUF runtime, model manager & OpenAI server utilizing device resources for Android Termux & ARM64
Emulates Bedrock Mantle with Ollama's OpenAI-compatible API
Local AI agent platform with full under-the-hood visibility — multi-agent orchestrator, web UI, channels, pluggable engines.
Acorn — 本地 AI 运行时:把大模型压缩成单个 .acorn 文件(内嵌权重+分词器+对话模板),在你的 CPU 上完全离线地聊天、微调、瘦身。零依赖核心,自研 INT4 内核,越小越好且附质量报告。无账号、无云端、无遥测。
LangChain integration for llamay, a local LLM runtime
Ollama chat, RAG and embeddings backend for DashAI