Agent frameworks & infrastructure
Local LLM runtimes
Local LLM runtimes run language models on your own hardware instead of calling a hosted API. Ollama handles model downloads and serving, while llama.cpp provides the underlying C/C++ inference engine for GGUF weights. They differ in scope: some are bare runtimes, others bundle chat interfaces or starter setups for private, offline use.
1,401 agents · most starred first
Top 100 of 1,401
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
LLM inference in C/C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
Official inference framework for 1-bit LLMs
The Self-hosted AI Starter Kit is an open-source template that quickly sets up a local AI environment. Curated by n8n, it provides essential tools for creating secure, self-hosted AI workflows.
Sub-50mb agentic, voice and vision intelligence for consumer devices.
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
AI Browser
ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.
Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.
AI You Control: Choose your models. Own your data. Eliminate vendor lock-in.
The all-in-one local AI studio for your desktop: chat, image and video generation and a coding agent in one free, open source app. Windows and Linux. No Docker, no terminal, no cloud required.
Open source inference server that runs the best local models for your hardware, plugged into the agent you already use. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.
Hit your Claude Code usage limit? Keep going free and offline on a local model: Qwen 3.8, Gemma 4, and Qwen 4 the day it ships. Runs on Apple Silicon Macs (MLX), Linux and Windows (llama.cpp) with no API key. Your code stays home.
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.
Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.
Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.
The Swiss Army Knife of Offline AI. Chat, Speak, and Generate Images - Privacy First, Zero Internet. Download an LLM and use it on your mobile device. No data ever leaves your phone. Supports text-to-text, vision, text-to-image
Replace Copilot local AI
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Turn your PC, Mac, or Linux box into a private AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Includes MLX Core macOS app with chat, agent mode, and tool calling.
Local AI chat + coding agent for Apple Silicon, powered by Gemma 4 via MLX / Supports Ollama
Fully-featured web interface for Ollama LLMs
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
Chat with AI large language models running natively in your browser. Enjoy private, server-free, seamless AI conversations.
Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer.
streamlit region:us
mac code — Claude Code, but it runs on your Mac for free. 35B AI agent at 30 tok/s via Apple Silicon flash-paging. $0/month.
The most advanced, fully offline client-side AI suite on Android today.
MLX Studio - Easiest way to run LLM's on your Mac. All in one engine.
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
Build AI agents that run 100% on-device. Sub-100ms latency on Qualcomm NPU. Zero cloud dependency.
One-command installer for a self-hosted AI stack on your own server: n8n, Ollama, Open WebUI, OpenClaw, Dify, Flowise, Supabase, ComfyUI, Qdrant & 30+ tools. Docker Compose, automatic HTTPS via Caddy, built-in monitoring. Free Zapier/Make alternative.
Spark-x2.6 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models
gguf qwen3_next unsloth qwen qwen3 text-generation
transformers llama facebook meta pytorch llama-2
Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.
Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.
Llama 3+ inference in pure Java
MLX Omni Server is a local inference server powered by Apple's MLX framework, specifically designed for Apple Silicon (M-series) chips. It implements OpenAI-compatible API endpoints, enabling seamless integration with existing OpenAI SDK clients while leveraging the power of local ML inference.
On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.
Local LLM, image&video&music generator
Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
Private local model studio, AI chat, and agent workspace powered by WebGPU and WebLLM.
Private local model studio, AI chat, and agent workspace powered by WebGPU and WebLLM.
Ollama alternative for Rockchip NPU: An efficient solution for running AI and Deep learning models on Rockchip devices with optimized NPU support ( rkllm )
The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.
Run any agent on any model, models stored as OCI images
Run LLaMA (and Stanford-Alpaca) inference on Apple Silicon GPUs.
An easier way to use AI on your own computer. A free, open-source alternative to Claude and ChatGPT that runs open-weight models like Gemma and gpt-oss locally on Mac, Windows and Linux. Private, and works offline.
Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~41 in fast mode, 512K context. Plus GLM-5.3-Flash, Qwen3.8 and LFM3-Omni. MoE experts tiered across VRAM, pinned RAM and NVMe, with Xolotl Direct NVMe->VRAM. OpenAI + Anthropic API.
本地优先的全栈 AI 创作工作站:AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块;vLLM / llama.cpp / Transformers 多引擎本地推理,显存智能调度、舱壁隔离自愈、仅供学习使用,欢迎交流指正。(最新进度(除插件系统)受到协议限制,暂时无法提交到仓库)
Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon
transformers gguf llama facebook meta pytorch
static region:us
[EMNLP 2024 Demo] TinyAgent: Function Calling at the Edge!
Run AI ✨ assistant locally! with simple API for Node.js 🚀
Run local LLMs like Gemma, Qwen, and LLaMA on Android for offline, private, real-time chat and question answering with LiteRT and ONNX Runtime.
LiteRT and LiteRT-LM sample apps, model recipes, agent skills and utilities.
TUI for Ollama and other LLM providers
✨ AI interface for tinkerers (Ollama, Haystack RAG, Python)
gguf GLM 4.7 Flash thinking reasoning NEO Imatrix MAX Quants
Run many AI agents at once on your Mac, up to 3x faster.
A tool for generating function arguments and choosing what function to call with local LLMs
ROCmFPX Family for AMD Hardware and Processors. More quants and special agent quants
Criando um chatbot inteligente 100% offline e no navegador
Spark-x2.5 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models
QVAC - Local AI SDK and libraries for building private, cross-platform, peer-to-peer AI applications. Run LLMs, speech-to-text, translation, and more locally on Linux, macOS, Windows, Android, and iOS.
Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
⚡ Edgen: Local, private GenAI server alternative to OpenAI. No GPU required. Run AI models locally: LLMs (Llama2, Mistral, Mixtral...), Speech-to-text (whisper) and many others.
Setup and run a local LLM and Chatbot using consumer grade hardware.
Self-hosted AI studio on your own GPU: chat with your files (RAG), screen and browser agents, coding swarms, LoRA training, an MCP server for Claude Code and Cursor, and local video, image, music and voice generation. No telemetry.
The easiest way to run the fastest MLX-based LLMs locally
Chrome Extension to Summarize or Chat with Web Pages/Local Documents Using locally running LLMs. Keep all of your data and conversations private. 🔐
This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.
Large Language Model (LLM) module for the Spezi Ecosystem
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.
A web UI Project In order to learn the large language model. This project includes features such as chat, quantization, fine-tuning, prompt engineering templates, and multimodality.
Private AI on premise
gguf text-generation image-text-to-text vision gsq rco
True on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM inference, Speech-to-Text, Image Generation, Agent Framework and RAG — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.
Tiny toolkit for air-gapped LLMs on consumer-grade hardware
Load local LLMs effortlessly in a Jupyter notebook for testing purposes alongside Langchain or other agents. Contains Oobagooga and KoboldAI versions of the langchain notebooks with examples.
Add AI capabilities to your file system using Ollama, Groq, OpenAi and other's api
Belullama is a comprehensive AI application that bundles Ollama, Open WebUI, and Automatic1111 (Stable Diffusion WebUI) into a single, easy-to-use package.
transformers gguf llama facebook meta pytorch
A guided tour on how to use HuggingFace large language models on Macs with Apple Silicon
True on-device AI for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM, Speech-to-Text and Image Generation — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.
🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки
무료로 한국어🇰🇷 파인튜닝 모델 받아서 로컬 LLM 호스팅. LangServe, Ollama, streamlit + RAG
Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document parsing, embeddings, transcription, text-to-speech and MCP tools. Powered by Ollama, LiteLLM, AnythingLLM and more. Local-first with auto-configured authentication, lightweight stacks, optional HTTPS and NVIDIA GPU support. MIT licensed. Multi-arch: amd64, arm64.