Agent frameworks & infrastructure
Local LLM runtimes
1,401 agents · most starred first
Local LLM runtimes, page 11 of 15
Python bindings for the C++ port of GPT4All-J model.
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
Package with binary files and subfolders
A command-line interface for interacting with Ollama AI models
Ultra-fast local LLM inference with zero-config hardware-optimized speculative decoding.
Download a HuggingFace model and convert it to GGUF for Ollama.
Local LLM inference + embedding & search in one package. Run 30B on consumer hardware, RAG without ChromaDB.
Python bindings for ik_llama.cpp — high-performance llama.cpp fork
Ignite your AI experience with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other cutting-edge language models - all running locally with blazing-fast performance.
Large Language Model Develop Toolkit
ipex-llm fix version
IPython magics for running Ollama models within the shell
Tools to interact with LLM models hosted on Ollama.
Running Large Language Model easily, faster and low-cost.
Jupyter anywidget for sideloading webllm wasm
Run LLMs locally. One command.
An integration package connecting LocalAI and LangChain
Add your description here
An integration package connecting NexaAI and LangChain
An integration package connecting Ollama and LangChain
An integration package connecting Qualcomm AI Inference Suite and LangChain
An integration package connecting Xinference and LangChain
LangExtract provider plugin for llama.cpp
Local agent building at scale using Ollama - comprehensive LangGraph patterns for local LLMs
Cross-platform LLM lifecycle manager with resource-pressure and gaming-detection autopause
Библиотека для запуска графического интерфейса с нейросетью GPT4All
Local agentic AI CLI powered by llama.cpp
Script to communicate with Texas Instruments CC13xx/CC2538/CC26xx Serial Boot Loader (Fork of cc2538-bsl.py)
CLI wrapper for llama.cpp providing an ollama-like experience
gguf connector core built on llama.cpp
Python binding for llama.cpp using cffi
llama.cpp (GGUF) conversion for pypi
A guidance compatibility layer for llama-cpp-python
Python wrapper for running the llama.cpp server
Auto-synchronized Python bindings for llama.cpp using CFFI ABI mode
A Python package for Llama CPP.
Python bindings for the llama.cpp library
Python bindings for the Ampere® optimized llama.cpp library
Python bindings for the llama.cpp library
Python bindings for the llama.cpp library
Python bindings for the llama.cpp library
Python bindings for the llama.cpp library
Python bindings for the llama.cpp library
Llama-Deck is a command-line tool for quickly managing and experimenting with multiple versions of llama inference implementations. It can help you quickly filter and download different llama implementations) and llama2-like transformer-based LLM models. We also provide some images based on some implementations, which can be easily deploy and run through our tool. Inspired by llama2.c project.
LLama.cpp embedder library python bindings
llama-index llms ipex-llm integration
llama-index llms llama cpp integration
llama-index llms llamafile integration
llama-index llms lmstudio integration
llama-index llms localai integration
llama-index llms mlx integration
llama-index llms ollama integration
llama-index multi modal llms ollama integration
llama-index multi_modal_llms openvino integration
llama-index packs ollama_query_engine integration
llama-index packs zephyr_query_engine integration
Enable IPFS model loading for llama-cpp-python
Easy deployment of quantized llama models on cpu
Unofficial python bindings for llama-rs. 🐍❤️🦀
A Quick Library with Llama 3.1/3.2 Tokenization - source https://github.com/jeffxtang/llama-tokens
Utility for building and running llama.cpp
Utility for building and running llama.cpp
llama2
Inference Llama 2 in one file of pure Python
Use llama2-wrapper as your local llama2 backend for Generative Agents / Apps
A Python package to interact with Llama 3 locally using Ollama.
llamac2py is a Python package that provides a wrapper for running inference using the Llama-2 Transformer model. The package includes a C executable (run.c) from Karpathy's llama2.c that performs the inference, and the wrapper module (wrapper.py) allows easy integration of the C code into Python scripts.
Python bindings for @ggerganov's llama.cpp
Ollama-like CLI wrapper around llama.cpp
Python bindings for llama.cpp
Python library for LlamaEdge
LlamaMlx-RS
FastAPI backend & desktop launcher for running local models via llama.cpp
LLM plugin for Apple Foundation Models (Apple Intelligence)
39% faster TTFT, 67% less KV cache, zero config — autotune optimises local LLMs on Ollama, LM Studio, and MLX
CLI tool to check hardware for local LLMs
Compress your local LLM KV cache with 5.3× memory reduction - Install: pip install llm-contextlens
A plugin for https://github.com/simonw/llm for running local CoreML .mlpackage model files.
Microlib for the Falcon LLM
One line: which open-source LLM works best on your local machine today.
Run models distributed as GGUF files
Plugin for LLM adding support for GPT4ALL models
LLM plugin for running models using llama.cpp