Agent frameworks & infrastructure
Local LLM runtimes
1,401 agents · most starred first
Local LLM runtimes, page 10 of 15
gguf llama.cpp qwen3 arabic tool-use agent
gguf gemma4 llama.cpp q4_0 qat speculative-decoding
gguf gemma4 llama.cpp q4_0 qat moe
gguf llama.cpp mtp speculative-decoding gemma gemma-4
transformers safetensors gguf llama text-generation gpt
gguf atomic-chat qwen qwen3 qwen3-coder imatrix
gguf qwen2 tool-use mcp function-calling llama-cpp
gguf quantized autoround qwen3.8 qwen 27b
transformers gguf imatrix SeaLLM3-7B-Chat text-generation en
Ollama review: compare features, pricing, use cases, access model, and alternatives for this Model Serving agent in 2026. Tool for running large language models locally
Run AI models on your computer for free and keep your data private.
Run powerful AI models directly on your computer, privately.
Ollama LLM provider (ILlm, extends OpenAIProvider — Ollama's OpenAI-compatible /v1 API) for @mcp-abap-adt/llm-agent.
Hardware accelerated language model chats on browsers
Local LLM runtime for Node.js: GGUF inference, KV-cache compression, SONA self-learning with LoRA, speculative decoding and an HTTP serve command.
LM Studio SDK
Intelligent CLI tool with AI-powered model selection that analyzes your hardware and recommends optimal LLM models for your system
Ollama integration for LangChain.js
Gemma model-family provider for Robota using OpenAI-compatible local endpoints
Conclave AI Ollama agent — local inference via Ollama's OpenAI-compatible endpoint. Zero cost, zero API key.
Use Ollama with the Vercel AI SDK, implementing the official Ollama API. This provider has minimal dependencies and is web-compatible out of the box.
Vercel AI Provider for running LLMs locally using Ollama
Vercel AI SDK Provider for Ollama using official ollama-js library
Ollama adapter for TanStack AI local LLM chat, tool calling, and structured outputs.
Apple LLM provider for Vercel AI SDK
Browser AI model provider for Vercel AI SDK (Chrome & Edge)
Maiar: A Composable, Plugin-Based AI Agent Framework
AI Agent framework powered by node-llama-cpp
Ollama tamil helps you to build ollama api but with tamil interface
Python client for the Ollama API
Python client library for function calling interactions with Ollama REST API
Ollama backend adapter for the AccuralAI LLM orchestration pipeline.
Foundry Local integration for Microsoft Agent Framework.
MLX integration for the Agent Framework
Ollama integration for Microsoft Agent Framework.
Microsoft Agent Framework Onnx.
Ollama local LLM + embedding provider for AgentForge — llama / mistral / qwen / mxbai-embed on localhost
ailia LLM
Airflow operator for managing Ollama and ngrok
AirLLM runs 70B large language models on a single 4GB GPU without quantization, distillation or pruning. Kimi K3 2.8T on under 4GB, Qwen3.8-Flash-Next 125B on 6GB, DeepSeek-V3 671B on ~12GB. Train Qwen3.8-Flash-Next under 6GB.
A Python package for interacting with AI models
A simple and efficient python library for fast inference of GGUF Large Language Models.
A simple and efficient python library for fast inference of GGUF Large Language Models.
A python package to apply opensource LLM in local CUDA environment
Official Python bindings for AutoAgents Llama.cpp backend (CUDA)
Official Python bindings for AutoAgents Llama.cpp backend (Metal)
Official Python bindings for AutoAgents Llama.cpp backend (CPU)
Official Python bindings for AutoAgents Llama.cpp backend (Vulkan)
A chat completion client extension using Llama CPP, integrating with AutoGen for AI-powered chat interactions.
Benchmark your local LLMs.
A benchmarking tool for LM Studio inference
Tries to pick the best local llm for your computer and environment.
Large Language Model Develop Toolkit
Run Claude Code locally on the Bonsai 8B 1-bit MLX model.
When you think about light weight LLMs, call brollm!
cLLM is an Open-source library that use llama-cpp-python and llama.cpp and provide a Low and High level API and allow developer to be more pythonic.
chatglm llm
ChatLLaMA: Open and Efficient Foundation Language Models Runnable In A Single GPU
A python module for conversational AI with ollama
A minimal, typed Python interface for Ollama that simplifies the existing ollama library
Esta librería fue creada por IA Sistema de Intres
Run ollama on Google Collab or Kaggle
Compact AI, any device. No API key. No cost.
Access Llama in your terminal.
Windows support for copilot-ollama
Framework for efficient local LLM interaction
GPU-Accelerated LLM Terminal for Apple Silicon
Local LLM Council powered by Ollama
Custom CrewAI LLM provider for Ollama's native REST API — no OpenAI shim, NDJSON streaming, tool calling, cloud auth
cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.
cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.
cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.
cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.
cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.
Datasette enrichment for analyzing row data using a locally hosted - or remote - ollama instance
A CLI tool to deploy and manage LLMs using Ollama.
Functions to parse Ollama options from string and get info on Ollama options.
dora-llama-cpp-python
Python package wrapping llama.cpp for on-device LLM inference
Zero-friction GPU bootstrap for llama-cpp-python on Windows.
一个用于调用Ollama模型的Python工具包
elizaOS Ollama Plugin - Local LLM client for text and object generation
Embeddings interface for Ollama
A plugin for Enthusiast that provides an Ollama connector.
ESA Local-LLM is a llama.cpp server in Docker with OpenAI Style Endpoints.
A professional Python library for Ollama integration with support for multiple data sources
A lightweight Python wrapper for Ollama AI models.
fastllama
Find the best local AI model for your GPU — terminal UI
llama-index llms llama cpp integration
llama-index llms ollama integration
Use local LLMs in your Python apps, with GPU acceleration and zero dependencies.
Open weights large language model (LLM) from Google DeepMind.
Deprecated: This package has been renamed to genkit-ollama.
Download LLMs to use locally.
Wrapper for simplified use of Llama2 GGUF quantized models.
Build GPT Disk Images on Local Drives
Python bindings for GPT4All
Command-line interface using GPT4ALL bindings