Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 14 of 15

hermes-ollama-native

Route a Hermes Agent's local Ollama traffic through the native /api/chat API (honors num_ctx + correct tool-call streaming) instead of the OpenAI-compat /v1 endpoint.

llama-wrangler

Lightweight web admin panel for llama.cpp server management

ollama-mtop

htop for local AI inference — curses-based TUI monitor for Ollama, llama-swap/vLLM and ComfyUI: models, GPU and server

waken-ollama

Ollama Target adapter for Waken — routes waken Events to a local/self-hosted Ollama daemon.

reasoning.run

A private reasoner for your terminal. Direct conversation or grounded multi-agent research, GPU-native and fully local. No API keys, no inference servers.

lore-llm

Local-first LLM instance framework: deploy, specialize, and chat with grounded, cited model instances.

@tiens.nguyen/gonext-cli

GoNext CLI — the gonext terminal plus the local worker that runs agent / OCR / PDF / embedding jobs on your Mac (Ollama / MLX / OpenAI-compatible).

@moolam/edge-agent

Internal component of Sutra SDK — applications should install sutra-sdk. The Edge Harness. Offline-first autonomous agent runtime for local SLMs (Phi-3, Gemma, quantized GGUF/ONNX) with durable friction telemetry and on-device vector memory.

vibe-lm

A comprehensive set of agentic tools for LM Studio: file operations, web fetching/search, memory, SSH, and an autonomous vibe_bridge keep-alive loop.

@browser-ai/transformers-js

Transformers.js provider for Vercel AI SDK - Run 🤗 Transformers directly in the browser.

agent-corporation

A corporation of character AI agents running on local LLMs — each agent with its own persona and app set. Name reservation; developed privately.

bharat-tiny-llm

Bharat-Tiny-LLM - Offline Hindi/Hinglish AI assistant for Apple Silicon. `bharat chat` and go. 33.8% token compression via Brahmi injection.

chp-adapter-local-llm

CHP capability adapter — local LLM inference via Ollama (primary) or llama.cpp (fallback)

diffusiongemma-agent

Installer and CLI for the local DiffusionGemma coding agent runtime

ethicallama

ethical, local-only inference wrapper for llama.cpp

evil-ollama

🦙 Exposed Ollama Instance Hunter, Proxy & API Manipulation Tool

gb10-load-llm

Small GB10/GX10 helpers for loading LLMs without slow cold-page CUDA H2D transfers.

hearth-llm

Keeps your local LLM contexts warm: persistent KV-cache proxy for llama.cpp's llama-server

kinako-llama-cpp

A secure Windows-native DLL LLM tool for enterprises.

langchain-apple-fm

LangChain integrations with Apple Foundation Model

lfx-ollama

Ollama model components as a standalone Langflow Extension Bundle.

llmt-cli

Find, download (over BitTorrent + webseeds), and SHA-256-verify LLM models from the LLM Torrents catalog.

local-llm-ctl

CLI tool to manage local LLM models and role-based server lifecycle

lora-ollama-bridge

LoRA adapter bridge between Unsloth and Ollama for dynamic adapter loading.

mila-llm

The LLM stack you can read and understand: Gemma 4 and Llama on CUDA, built from explicit C++23 components.

omllm

omllm

omni-local-llm

A unified Python interface and API server for running local Large Language Models.

outo-llms

Deploy local LLMs behind your own managed Anthropic-compatible API server - vLLM, llama.cpp, or SGLang in isolated environments, with API keys, workspaces, and usage tracking.

sdg-agents

A terminal user interface (TUI) for SDG Agents — powered by local AI models

skulk-llama-server-vulkan

Pinned Vulkan llama-server build for Skulk's served GGUF engine (Linux x86_64)

@localmode/langchain

LangChain.js adapters for @localmode — drop-in local inference for existing LangChain apps

caniollama

Standardized compatibility checker for local Ollama model tool-calling support

ollama-doctor

Pre-flight checker for local Ollama models

fastpluggy-ollama-connector

Placeholder to reserve this name — not the real package.

mineru-llama-cpp

In-process llama.cpp VLM inference engine for MinerU

ollama-advisor

Recommend, download, and run Ollama models based on your system specs and use case

ollama-shell

A local-first, agentic shell for Ollama — chat, tools, and an MCP-style agent loop that never phones home.

aclade-agent

Aclade Local AI Agent Connector

ollama-json-fixer

Force local Ollama models to output strict, valid JSON. Auto-strips markdown fences and retries on failure.

dsh-llm-agent-virtualization

DeepSeek Harness model provider for virtualized local CLI agents

dc-preflight-llm-check

Pre-flight connectivity checks for the n8n + Ollama training stack.

llama-roofline

Is your llama.cpp decode memory-bandwidth-bound? Measure it, in one command.

lllm3090

Local LLM serving for a single RTX 3090, with a browser control panel

llmcalculator

Work out which AI models your computer can run, fine-tune, or train.

llmrig

Autopilot for local AI: evidence-driven planning, controlled setup, and measured verification.

local-llm-preflight

Fail-safe client discipline for local LLMs on memory-constrained machines: preflight memory checks, starvation-aware retries, thinking-mode-off, token budgets.

ramp-llm

RAMP: Resource-Aware Model Proxy - an elastic local LLM daemon that auto-calibrates model size to available RAM, VRAM and disk

@drdoc-com/pi-openai-api-wrapper

Web Endpoint as PI Package for PI Agent (pi.dev), with Ollama API and OpenAI API compability.

hf2ollama-python-cli-tool

Search, download, and import HuggingFace GGUF models into Ollama

hyle-ollama

Native Ollama integration for Hyle.

llama-mdl

Run local llama.cpp servers from a config file, and work out which model to run and how

llm-omlx

LLM plugin for models served by oMLX

pocket-o-llama

Lightweight, hardware-independent local GGUF parser and Ollama substitute server with Hugging Face Hub integration.

termux-llamacpp

GGUF runtime, model manager & OpenAI server utilizing device resources for Android Termux & ARM64

llama-cpp-beam-search

A package to combine GBNF grammar and Beam Search for probablistic grammar pruning with configurables beams (both breadth- and depth-first)

canirunllm

Find out which open-source LLMs can run on your hardware.

exllamav3

Inference library for running local LLMs on modern consumer GPUs

lllm2

Local NVIDIA LLM experimentation workbench

ollama-watch

Live prefill/decode progress for a local Ollama server and basic observability metrics

apple-llm

Free local LLM on your Mac: Apple Intelligence's on-device Foundation Models and Private Cloud Compute from Python, plus text to speech and speech to text in macOS's own voices and recognition. Guaranteed JSON (pydantic), streaming, sessions. No API key, no account.

flyweight-llm

Native GGUF and safetensors inference runtime for Qwen, Laguna, K2-Horizon, Muse, DeepSeek-V4, Gemma and BailingMoE3 models

hugpy-storage

Hugpy storage: model download queue and daemon, Hugging Face transport and token, resumable transfers, model sync, physical inventory and on-disk model layout

hunch-engine-ollama

A hunch engine plugin: local models served by Ollama, read through their token probabilities

llamafit

Find, size, install and verify open-weight LLMs for llama.cpp on your own machine.

llm-unslothstudio

LLM plugin for models served by Unsloth Studio

llmchub

Мы предоставляем Вам удобную возможность - развернуть модель на облаке или на компьютере локально и использовать ее с другой точки мира! Работает это просто - сначала Вызапускаете модель на любом железе, получаете ключ доступа (Local_CreateCode()) и сопряжаете по нему устройства. Далее - работайте по документации (https://llmconstructor-pylibrary.onrender.com/docs).

modelscout-llm

Find local AI models that fit your hardware, with memory and speed estimates.

ollama-local-api

Call the Ollama local API by function.Please install Ollama before use.

ovos-ollama-dialog-transformer-plugin

Dialog transformer for OVOS, designed to return responses from Ollama

pllm.run

Private language model inference with a local Responses API and remote matrix execution

pocketllm

Inference engine for large language models on consumer GPUs with deep optimization for older hardware

@kaleidorg/create-mind

Scaffold a local-LLM agent for an RGB Lightning Node with @kaleidorg/mind: npm create @kaleidorg/mind my-agent

@space-uy/qvac-langgraph

LangChain chat model (ChatQVAC) that runs local QVAC completions, for use in LangChain runnables and LangGraph graphs.

langchain-web-llm

WebLLM chat models and embeddings for LangChain, LangGraph, and Deep Agents.

airllm-easy-tui

带插件系统的 CLI 大语言模型推理接口

clap-llm

CLAP: Chat Local And Persistent - Local LLM conversation tool with Ollama

crewplane-ollama

Explicit tool-enabled Ollama agent capability for Crewplane

dasllama

OpenAI-compatible local LLM server, CLI and benchmark over dasLLAMA

dragon-cli

CLI to chat with local GGUF LLMs, with MCP tool-calling and SQLite conversation history.

eary-start-llama

Manage local llama.cpp model servers with a JSON manifest and an interactive TUI.

langchain-llama-cpp

An integration package connecting llama.cpp's llama-server and LangChain, with reasoning, timings, structured output and tool calling

lightpath-microllm

Hybrid System-One Local Decision Engine (JevCompat 32/32 MUST, AnyJev L0 Prefill, StartLux Read-or-Think Cascade, OpenJev-RLCD Trainer)

make-localllm-easier

One command runs the best local AI your PC can handle: measured model picks + a tuned llama.cpp (AMD, NVIDIA, Intel, Apple)

microllm-jev

Hybrid System-One Local Decision Engine (JevCompat 32/32 MUST, AnyJev L0 Prefill, StartLux Read-or-Think Cascade, OpenJev-RLCD Trainer)

mlx-agent

Discover, verify and wire local MLX models on Apple Silicon

nersc-ollama-manager

User-owned Ollama servers in interactive NERSC allocations

notebook-llm-cli

Interactive CLI to run Ollama LLMs on Kaggle/Colab GPUs: GPU detection, fit and tokens/sec estimates, library search, public tunnel.

polyllm-ollama

Ollama (local OpenAI-compatible) plugin for PolyLLM.

ramllm

Run bigger models on smaller machines — RAM-adaptive, GPU-aware LLM inference manager

win-llmtop

A live terminal panel for llama.cpp's llama-server on Windows: GPU and shared memory, RAM, requests

windowsml-llama-core

llama.cpp backend core for the Windows ML Runtime

justfit-llm

JustFit: long-context 27B language-and-vision serving on 24 GB Apple silicon Macs (native, no Python at runtime)

rosa-llm

Standalone Ollama and ChatGPT OAuth clients with a shared Python API and CLI

@volter/twin-deepseek

Local DeepSeek twin — a faithful, stateful local DeepSeek Platform API your real `@ai-sdk/deepseek` talks to unmodified. The model is stubbed (deterministic), but the protocol envelope is vendor-faithful and so are DeepSeek's OpenAI-DIVERGENT REFUSALS: 42

amv-llamacpp

GGUF runtime, model manager & OpenAI server utilizing device resources for Android Termux & ARM64

localstack-extension-bedrock-mantle

Emulates Bedrock Mantle with Ollama's OpenAI-compatible API

sharedagents

Local AI agent platform with full under-the-hood visibility — multi-agent orchestrator, web UI, channels, pluggable engines.

acorn-llm

Acorn — 本地 AI 运行时:把大模型压缩成单个 .acorn 文件(内嵌权重+分词器+对话模板),在你的 CPU 上完全离线地聊天、微调、瘦身。零依赖核心,自研 INT4 内核,越小越好且附质量报告。无账号、无云端、无遥测。

langchain-llamay

LangChain integration for llamay, a local LLM runtime

dashai-ollama

Ollama chat, RAG and embeddings backend for DashAI

More in Agent frameworks & infrastructure