Agent frameworks & infrastructure

Local LLM runtimes

1,401 agents · most starred first

Local LLM runtimes, page 11 of 15

gpt4all-j

Python bindings for the C++ port of GPT4All-J model.

gpt4all-pypi-part-001

Package with binary files and subfolders

gpt4all-pypi-part-002

Package with binary files and subfolders

gpt4all-pypi-part-003

Package with binary files and subfolders

gpt4all-pypi-part-004

Package with binary files and subfolders

gpt4all-pypi-part-005

Package with binary files and subfolders

gpt4all-pypi-part-006

Package with binary files and subfolders

gpt4all-pypi-part-007

Package with binary files and subfolders

gpt4all-pypi-part-008

Package with binary files and subfolders

gpt4all-pypi-part-009

Package with binary files and subfolders

gpt4all-pypi-part-010

Package with binary files and subfolders

gpt4all-pypi-part-011

Package with binary files and subfolders

gpt4all-pypi-part-012

Package with binary files and subfolders

gpt4all-pypi-part4

Package with binary files and subfolders

h2ogpt
haiku-ollama

A command-line interface for interacting with Ollama AI models

hexonit-llm

Ultra-fast local LLM inference with zero-config hardware-optimized speculative decoding.

hf2ollama

Download a HuggingFace model and convert it to GGUF for Ollama.

hippo-llm

Local LLM inference + embedding & search in one package. Run 30B on consumer hardware, RAG without ChromaDB.

ik-llama-cpp-python

Python bindings for ik_llama.cpp — high-performance llama.cpp fork

inferno-llm

Ignite your AI experience with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other cutting-edge language models - all running locally with blazing-fast performance.

ipex-llm

Large Language Model Develop Toolkit

ipex-llm-fix

ipex-llm fix version

ipython-ollama

IPython magics for running Ollama models within the shell

jan883-ollama

Tools to interact with LLM models hosted on Ollama.

jllm

Running Large Language Model easily, faster and low-cost.

jupyter-anywidget-webllm

Jupyter anywidget for sideloading webllm wasm

koda-llm

Run LLMs locally. One command.

langchain-llama
langchain-llamacpp-chat-model
langchain-localai

An integration package connecting LocalAI and LangChain

langchain-mlx

Add your description here

langchain-nexa-ai

An integration package connecting NexaAI and LangChain

langchain-ollama

An integration package connecting Ollama and LangChain

langchain-onnx
langchain-openllm
langchain-openlm
langchain-qualcomm-inference-suite

An integration package connecting Qualcomm AI Inference Suite and LangChain

langchain-xinference

An integration package connecting Xinference and LangChain

langextract-llamacpp

LangExtract provider plugin for llama.cpp

langgraph-ollama-local

Local agent building at scale using Ollama - comprehensive LangGraph patterns for local LLMs

legionforge-llm-valet

Cross-platform LLM lifecycle manager with resource-pressure and gaming-detection autopause

library-gpt4all

Библиотека для запуска графического интерфейса с нейросетью GPT4All

llama-agentic

Local agentic AI CLI powered by llama.cpp

llama-bsl

Script to communicate with Texas Instruments CC13xx/CC2538/CC26xx Serial Boot Loader (Fork of cc2538-bsl.py)

llama-buddy

CLI wrapper for llama.cpp providing an ollama-like experience

llama-core

gguf connector core built on llama.cpp

llama-cpp-cffi

Python binding for llama.cpp using cffi

llama-cpp-conv

llama.cpp (GGUF) conversion for pypi

llama-cpp-guidance

A guidance compatibility layer for llama-cpp-python

llama-cpp-py

Python wrapper for running the llama.cpp server

llama-cpp-py-sync

Auto-synchronized Python bindings for llama.cpp using CFFI ABI mode

llama-cpp-pydist

A Python package for Llama CPP.

llama-cpp-python

Python bindings for the llama.cpp library

llama-cpp-python-ampere

Python bindings for the Ampere® optimized llama.cpp library

llama-cpp-python-binary

Python bindings for the llama.cpp library

llama-cpp-python-bundled

Python bindings for the llama.cpp library

llama-cpp-python-cross

Python bindings for the llama.cpp library

llama-cpp-python-kirp

Python bindings for the llama.cpp library

llama-cpp-python-win

Python bindings for the llama.cpp library

llama-deck

Llama-Deck is a command-line tool for quickly managing and experimenting with multiple versions of llama inference implementations. It can help you quickly filter and download different llama implementations) and llama2-like transformer-based LLM models. We also provide some images based on some implementations, which can be easily deploy and run through our tool. Inspired by llama2.c project.

llama-embedder

LLama.cpp embedder library python bindings

llama-index-llms-ipex-llm

llama-index llms ipex-llm integration

llama-index-llms-llama-cpp

llama-index llms llama cpp integration

llama-index-llms-llamafile

llama-index llms llamafile integration

llama-index-llms-lmstudio

llama-index llms lmstudio integration

llama-index-llms-localai

llama-index llms localai integration

llama-index-llms-mlx

llama-index llms mlx integration

llama-index-llms-ollama

llama-index llms ollama integration

llama-index-multi-modal-llms-ollama

llama-index multi modal llms ollama integration

llama-index-multi-modal-llms-openvino

llama-index multi_modal_llms openvino integration

llama-index-packs-ollama-query-engine

llama-index packs ollama_query_engine integration

llama-index-packs-zephyr-query-engine

llama-index packs zephyr_query_engine integration

llama-ipfs

Enable IPFS model loading for llama-cpp-python

llama-memory

Easy deployment of quantized llama models on cpu

llama-rs-python

Unofficial python bindings for llama-rs. 🐍❤️🦀

llama-tokens

A Quick Library with Llama 3.1/3.2 Tokenization - source https://github.com/jeffxtang/llama-tokens

llama-tools-aisee

Utility for building and running llama.cpp

llama-tools-jaliya

Utility for building and running llama.cpp

llama2

llama2

llama2-py

Inference Llama 2 in one file of pure Python

llama2-wrapper

Use llama2-wrapper as your local llama2 backend for Generative Agents / Apps

llama3-package

A Python package to interact with Llama 3 locally using Ollama.

llamac2py

llamac2py is a Python package that provides a wrapper for running inference using the Llama-2 Transformer model. The package includes a C executable (run.c) from Karpathy's llama2.c that performs the inference, and the wrapper module (wrapper.py) allows easy integration of the C code into Python scripts.

llamacpp

Python bindings for @ggerganov's llama.cpp

llamacpp-cli

Ollama-like CLI wrapper around llama.cpp

llamacpypy

Python bindings for llama.cpp

llamaedge

Python library for LlamaEdge

llamamlx-rs-llamasearch

LlamaMlx-RS

llamastudio

FastAPI backend & desktop launcher for running local models via llama.cpp

llm-apple

LLM plugin for Apple Foundation Models (Apple Intelligence)

llm-autotune

39% faster TTFT, 67% less KV cache, zero config — autotune optimises local LLMs on Ollama, LM Studio, and MLX

llm-checker

CLI tool to check hardware for local LLMs

llm-contextlens

Compress your local LLM KV cache with 5.3× memory reduction - Install: pip install llm-contextlens

llm-coreml

A plugin for https://github.com/simonw/llm for running local CoreML .mlpackage model files.

llm-falcon-model

Microlib for the Falcon LLM

llm-fit-check

One line: which open-source LLM works best on your local machine today.

llm-gguf

Run models distributed as GGUF files

llm-gpt4all

Plugin for LLM adding support for GPT4ALL models

llm-llama-cpp

LLM plugin for running models using llama.cpp

More in Agent frameworks & infrastructure