Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 10 of 15

Arabian-Agent-Qwen3-0.6B-GGUF★ 1

gguf llama.cpp qwen3 arabic tool-use agent

gemma-4-E4B-it-qat-assistant-q4_0-exact-gguf★ 1

gguf gemma4 llama.cpp q4_0 qat speculative-decoding

gemma-4-26B-A4B-it-qat-assistant-q4_0-exact-gguf★ 1

gguf gemma4 llama.cpp q4_0 qat moe

gemma-4-E4B-it-assistant-GGUF★ 1

gguf llama.cpp mtp speculative-decoding gemma gemma-4

Danube_3-500M_Chat_GGUF★ 1

transformers safetensors gguf llama text-generation gpt

qwen3-coder-30b-a3b-GGUF★ 1

gguf atomic-chat qwen qwen3 qwen3-coder imatrix

limbic-tool-use-0.5B-32K-GGUF★ 1

gguf qwen2 tool-use mcp function-calling llama-cpp

Sakura-Humanlike-Chat-2.0-AutoRound-GGUF★ 1

gguf quantized autoround qwen3.8 qwen 27b

SeaLLM3-7B-Chat-imatrix-GGUF★ 1

transformers gguf imatrix SeaLLM3-7B-Chat text-generation en

ollama

Ollama review: compare features, pricing, use cases, access model, and alternatives for this Model Serving agent in 2026. Tool for running large language models locally

LM Studio

Run AI models on your computer for free and keep your data private.

Ollama

Run powerful AI models directly on your computer, privately.

@mcp-abap-adt/ollama-llm

Ollama LLM provider (ILlm, extends OpenAIProvider — Ollama's OpenAI-compatible /v1 API) for @mcp-abap-adt/llm-agent.

@mlc-ai/web-llm

Hardware accelerated language model chats on browsers

@ruvector/ruvllm

Local LLM runtime for Node.js: GGUF inference, KV-cache compression, SONA self-learning with LoRA, speculative decoding and an HTTP serve command.

@lmstudio/sdk

LM Studio SDK

llm-checker

Intelligent CLI tool with AI-powered model selection that analyzes your hardware and recommends optimal LLM models for your system

@langchain/ollama

Ollama integration for LangChain.js

@robota-sdk/agent-provider-gemma

Gemma model-family provider for Robota using OpenAI-compatible local endpoints

@conclave-ai/agent-ollama

Conclave AI Ollama agent — local inference via Ollama's OpenAI-compatible endpoint. Zero cost, zero API key.

ollama-ai-provider-v2

Use Ollama with the Vercel AI SDK, implementing the official Ollama API. This provider has minimal dependencies and is web-compatible out of the box.

ollama-ai-provider

Vercel AI Provider for running LLMs locally using Ollama

ai-sdk-ollama

Vercel AI SDK Provider for Ollama using official ollama-js library

@tanstack/ai-ollama

Ollama adapter for TanStack AI local LLM chat, tool calling, and structured outputs.

@react-native-ai/apple

Apple LLM provider for Vercel AI SDK

@browser-ai/core

Browser AI model provider for Vercel AI SDK (Chrome & Edge)

@maiar-ai/model-ollama

Maiar: A Composable, Plugin-Based AI Agent Framework

agentium

AI Agent framework powered by node-llama-cpp

OllamaTamil

Ollama tamil helps you to build ollama api but with tamil interface

Python-Ollama

Python client for the Ollama API

RacoGPT

Python client library for function calling interactions with Ollama REST API

accuralai-ollama

Ollama backend adapter for the AccuralAI LLM orchestration pipeline.

agent-framework-foundry-local

Foundry Local integration for Microsoft Agent Framework.

agent-framework-mlx

MLX integration for the Agent Framework

agent-framework-ollama

Ollama integration for Microsoft Agent Framework.

agent-framework-onnx

Microsoft Agent Framework Onnx.

agentforge-ollama

Ollama local LLM + embedding provider for AgentForge — llama / mistral / qwen / mxbai-embed on localhost

ailia-llm

ailia LLM

airflow-ollama

Airflow operator for managing Ollama and ngrok

airllm

AirLLM runs 70B large language models on a single 4GB GPU without quantization, distillation or pruning. Kimi K3 2.8T on under 4GB, Qwen3.8-Flash-Next 125B on 6GB, DeepSeek-V3 671B on ~12GB. Train Qwen3.8-Flash-Next under 6GB.

allama

A Python package for interacting with AI models

allm

A simple and efficient python library for fast inference of GGUF Large Language Models.

allmdev

A simple and efficient python library for fast inference of GGUF Large Language Models.

applyllm

A python package to apply opensource LLM in local CUDA environment

autoagents-llamacpp-cuda

Official Python bindings for AutoAgents Llama.cpp backend (CUDA)

autoagents-llamacpp-metal

Official Python bindings for AutoAgents Llama.cpp backend (Metal)

autoagents-llamacpp-py

Official Python bindings for AutoAgents Llama.cpp backend (CPU)

autoagents-llamacpp-vulkan

Official Python bindings for AutoAgents Llama.cpp backend (Vulkan)

autogen-llama-cpp-chat-completion

A chat completion client extension using Llama CPP, integrating with AutoGen for AI-powered chat interactions.

benchllama

Benchmark your local LLMs.

benchmarkLLM

A benchmarking tool for LM Studio inference

bestllm

Tries to pick the best local llm for your computer and environment.

bigdl-core-llm-27
bigdl-llm

Large Language Model Develop Toolkit

bonsai-claude

Run Claude Code locally on the Bonsai 8B 1-bit MLX model.

brollm

When you think about light weight LLMs, call brollm!

cLLM-python

cLLM is an Open-source library that use llama-cpp-python and llama.cpp and provide a Low and High level API and allow developer to be more pythonic.

chatglm-llm

chatglm llm

chatllama

ChatLLaMA: Open and Efficient Foundation Language Models Runnable In A Single GPU

chatollama

A python module for conversational AI with ollama

clean-ollama

A minimal, typed Python interface for Ollama that simplifies the existing ollama library

colab-gradio-llm

Esta librería fue creada por IA Sistema de Intres

collab-ollama

Run ollama on Google Collab or Kaggle

compactllm

Compact AI, any device. No API key. No cost.

consolellm

Access Llama in your terminal.

copilot-ollama-windows

Windows support for copilot-ollama

corellm

Framework for efficient local LLM interaction

cortex-llm

GPU-Accelerated LLM Terminal for Apple Silicon

councillm

Local LLM Council powered by Ollama

crewai-ollama-cloud

Custom CrewAI LLM provider for Ollama's native REST API — no OpenAI shim, NDJSON streaming, tool calling, cloud auth

cyllama

cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.

cyllama-cuda12

cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.

cyllama-rocm

cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.

cyllama-sycl

cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.

cyllama-vulkan

cyllama is a comprehensive zero-dependencies Python library for local AI inference using the state-of-the-art llama, whisper, and stable-diffusion .cpp ecosystem.

datasette-enrichments-ollama

Datasette enrichment for analyzing row data using a locally hosted - or remote - ollama instance

deploy-llm

A CLI tool to deploy and manage LLMs using Ollama.

dm-ollamalib

Functions to parse Ollama options from string and get info on Ollama options.

dora-llama-cpp-python

dora-llama-cpp-python

easy-llama

Python package wrapping llama.cpp for on-device LLM inference

easyllama-divi845

Zero-friction GPU bootstrap for llama-cpp-python on Windows.

elias-ollama

一个用于调用Ollama模型的Python工具包

elizaos-plugin-ollama

elizaOS Ollama Plugin - Local LLM client for text and object generation

ellama

Embeddings interface for Ollama

enthusiast-model-ollama

A plugin for Enthusiast that provides an Ollama connector.

esa-local-llm

ESA Local-LLM is a llama.cpp server in Docker with OpenAI Style Endpoints.

evolvishub-ollama-adapter

A professional Python library for Ollama integration with support for multiple data sources

ezollama

A lightweight Python wrapper for Ollama AI models.

fastllama

fastllama

fitmyllm

Find the best local AI model for your GPU — terminal UI

flying-delta-llms-llama-cpp

llama-index llms llama cpp integration

flying-delta-llms-ollama

llama-index llms ollama integration

fxn-llm

Use local LLMs in your Python apps, with GPU acceleration and zero dependencies.

gemma-llm

Open weights large language model (LLM) from Google DeepMind.

genkit-plugin-ollama

Deprecated: This package has been renamed to genkit-ollama.

getllms

Download LLMs to use locally.

gguf-llama

Wrapper for simplified use of Llama2 GGUF quantized models.

gpt-image

Build GPT Disk Images on Local Drives

gpt4all

Python bindings for GPT4All

gpt4all-cli

Command-line interface using GPT4ALL bindings

More in Agent frameworks & infrastructure