Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 12 of 15

llm-llama-server

Interact with llama-server models

llm-llamacpp-plugin

LLM plugin providing access to llama.cpp server models

llm-llamafile

Access llamafile localhost models via LLM

llm-lmstudio

LLM plugin for models served by LMStudio (local LLM studio HTTP API)

llm-local-client

Agentic interface to local LLMs with tree-based context management.

llm-mlc

LLM plugin for running models using MLC

llm-mlx

Support for MLX models in LLM

llm-mpt30b

Plugin for LLM adding support for the MPT-30B language model

llm-neofetch-plus

Advanced system information tool for local LLM usage

llm-ollama

LLM plugin providing access to models running on an Ollama server

llm-rs

Unofficial python bindings for llm-rs. 🐍❤️🦀

llm-rs-cuda

Unofficial python bindings for llm-rs.

llm-rs-metal

Unofficial python bindings for llm-rs. 🐍❤️🦀

llm-rs-opencl

Unofficial python bindings for llm-rs. 🐍❤️🦀

llm-smollm2

SmolLM2-135M-Instruct.Q4_1 for LLM

llm-text-generation

Run large language models.

llm-transformers

Plugin for llm adding support for 🤗 Hugging Face Transformers

llmfastdev

A Python client for interacting with llamafile local LLM instances

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

llmflex

A python package for developing AI applications with local LLMs.

llmocal

Professional open source client for running large language models locally

llmpm

LLM Package Manager — download, run, and share AI models from the command line

llmready

A CLI tool to check if your local hardware can run a Hugging Face LLM.

llmscan

A fast, modern CLI that shows which local LLMs fit on your machine

llmstack-cli

One command. Full LLM stack. Zero config.

llmster

llmster

llmtoolset

Tools using ollama and torch that are easy and nice to have for working with local LLMs

llmwalk

Explore the answer-space for any prompt and any MLX-supported model.

llmworker

A lightweight Python toolkit for LLM text and image workflows with Ollama.

lmagent-plus

Local-first AI agent orchestrator — manages llama.cpp, local models, and cloud APIs

local-llama

A flask server for a chrome extension that querys llm models

local-llm

Local-LLM is a llama.cpp server in Docker with OpenAI Style Endpoints.

local-llm-checker

Find which local LLMs can run on your system

local-llm-kit

One Python API for offline LLMs and local servers: Ollama, AirLLM, SGLang, vLLM, LM Studio and llama.cpp

local-llm-wrapper

Local-first LLM wrapper (import local_llm_wrapper) with pluggable transports (Ollama, Apple, optional Claude Code CLI) and a stable text-in text-out API.

locallm

An api to query local language models using different backends

locollm

Local Collaborative LLMs -- Frontier AI on a Student Budget

mallama

Browser UI for Ollama • Local LLM Interface • Web Chat Client for Local AI Models

mattsollamatools

Tools for working with Ollama

mdllama

A command-line interface for Ollama API

merlin-llm

Merlin — a fast local LLM for agentic coding on Apple Silicon

metaflow-ollama

An EXPERIMENTAL Ollama decorator for Metaflow

microgpt

Lightweight GPT implementation designed for resource-constrained environments

mlx-augllm

('A library for augmenting large language models using MLX',)

mlx-gpt-oss

Minimal OpenAI-compatible server for GPT-OSS models on Apple Silicon with MLX

mlx-llm

Large Language Models (LLMs) applications and tools running on Apple Silicon in real-time with Apple MLX

mygpt

A locally runnable LLM

myllmcli

llmcli 模型下载

nanollama

Nano Llama

nucleusiq-ollama

Ollama inference provider for the NucleusIQ AI agent framework (official ollama Python SDK).

observer-ollama

HTTPS proxy with CORS support for Ollama

odoo-addon-ai-oca-native-generate-ollama

This module replaces AI from html_editor to use an Ollama server instead of OpenAI through Odoo IAP.

oh-llama

Opinionated, Ollama Client based Python Tools.

ollama

The official Python client for Ollama.

ollama-agentic

A beautiful, agentic CLI for Ollama — run local LLMs with auto tool-calling, memory, and more

ollama-aid

Ollama model management, trends viewing, testing & external runner assistant

ollama-api

A Python client for the Ollama API

ollama-bar

Macos status bar app that allows you to start/stop/view ollama from the menu bar.

ollama-batch

Run text LLM prompts over a list of texts

ollama-call

A simple Python wrapper for Ollama API

ollama-classifier

Text classification with constrained output and confidence scoring across multiple LLM backends

ollama-cli

Simple command line tool that reads a text from stdin and pipes it to Ollama. One can set all Ollama options on commandline as well as define termination criteria in terms of maximum number of lines, paragraphs, or repeated lines.

ollama-client

Lightweight CLI client for a remote Ollama server

ollama-client-lib

Object-oriented library for interacting with Ollama Cloud

ollama-data-tools

Tools for working with Ollama model data

ollama-downloader

A library and Hugging Face model downloader for Ollama.

ollama-flow

A Python library for the Ollama API.

ollama-inspect

Ollama Inspect — minimal GGUF inspector web server

ollama-installer

Install Ollama from a Python CLI.

ollama-instructor

Validate structured outputs from LLMs with Ollama and automatic retries.

ollama-llm
ollama-local-serve

Local LLM infrastructure with monitoring dashboard for distributed AI applications

ollama-manager

Manage Ollama models from your commandline!

ollama-models

Python client for the ollama-models API — search and list Ollama model weights

ollama-panongbene

CLI tools to export and import Ollama models (.tar.gz)

ollama-remote

Expose OLLAMA models to the internet on a remote server via tunneling

ollama-scout

Scan your hardware and find compatible Ollama LLMs

ollama-serve

A Python package that ensures the Ollama server is running and ready with the models your project expects.

ollama-service-deepsec

Provide interaction service to LLM with Ollama.

ollama-switch

A package to manage Ollama service and models.

ollama-tool-cli

CLI tool for Ollama - backup, restore, update, and list models

ollama-toolkit

A comprehensive Python client library for the Ollama API

ollamafreeapi

A lightweight client for interacting with LLMs served via Ollama

ollamapy

A simple Python package for Ollama utilities with built-in AI vibe tests

ollamate

Ollamate is a simple AI assistant for local LLMs hosted on your machine using Ollama. it will support some custom functions and a RAG system.

ollamawebui

OllamaWebUI-A Simple Flask WebUI, You Can Chat With Ollama On Web|Flask聊天界面,可以自定义设置Ollama Api(python学霸公众号)

ollamax

A package to manage Ollama service and models.

ollamazip

Bundle Ollama models into single portable archives for transfer between machines.

onnxllm

OnnxLLM: A collection of Onnx Large Language Models.

openai-agents-python-providers

Ollama and llama.cpp providers for the OpenAI Agents SDK

opencode-llama-cpp-launcher

One command launcher for running OpenCode with a local llama.cpp model.

opencode-llmstack

Multi-tier local LLM stack: llama-swap + FastAPI auto-router + opencode wiring.

openllama

openllama

orchard-llm

High-performance LLM runtime for Apple Silicon

org.slashlib.py.inference.ollama

python inference handler for agents

ovos-solver-llamacpp-plugin

A question solver plugin for ovos

ovos-solver-llmcpp-plugin

A question solver plugin for ovos

oxillama

Python bindings for OxiLLaMa — Pure Rust LLM inference engine

parllama

Terminal UI for Ollama and other LLM providers

patchs-unsloth-llamacpp

A patch to enable llama.cpp as acceleration backend on windows platform

More in Agent frameworks & infrastructure