Tracked Repositories

HuggingFace Transformers — state-of-the-art NLP/ML model library (~140K stars)

HuggingFace Diffusers — diffusion model inference & training (Stable Diffusion, Flux, etc.)

HuggingFace TGI — LLM serving (archived March 2026, read-only)

User-friendly local LLM runner built on llama.cpp (~167K stars)

High-performance LLM inference in C/C++ (CPU + GPU)

High-performance Whisper speech recognition in C/C++

Self-hosted ChatGPT alternative with built-in RAG, offline-capable (~104K stars)

Desktop AI app + SDK for running LLMs locally (~73K stars)

Most widely adopted open-source LLM serving engine; PagedAttention, continuous batching

vLLM community plugin for Intel Gaudi accelerators

Cross-platform ML pipeline framework (vision, audio, NLP)

AI Edge model gallery

LiteRT for language model inference

Sample apps using MediaPipe

Highly optimized neural network operators library (ARM, x86, WASM)

Google's Lite Runtime (successor to TensorFlow Lite)

Model visualization and exploration tool

LiteRT integration with PyTorch

Sample code for LiteRT

Quantization tooling for AI Edge models

Sample models for AI Edge

AI Edge APIs — upstream repo deleted (404), local copy retained

Array framework for ML on Apple silicon (Python)

Example models and applications using MLX

Tools for converting & running models with Core ML

LLM inference and fine-tuning with MLX

Example apps using MLX Swift

Swift bindings for MLX

Efficient data loading for MLX

LLM inference in Swift via MLX

C bindings for MLX

Gradio web UI for LLMs — multi-backend (llama.cpp, ExLlamaV2, transformers) (~43K stars)

Free, open-source OpenAI drop-in replacement — runs locally, no GPU required (~36K stars)

Run LLMs distributed across heterogeneous devices (Mac, iPhone, etc.)

Microsoft DeepSpeed — distributed training and inference (ZeRO, MII, FastGen)

Open Neural Network Exchange format specification

Microsoft's cross-platform, high-performance ONNX inference engine

LLM serving framework and home of Chatbot Arena (~37K stars)

MLC's universal LLM deployment engine (multi-backend)

Tile-based ML language and compiler

Community on-device LLM project

vLLM-style inference on Apple silicon via MLX

Internal fork of OpenEvolve

High-performance neural network inference for mobile (Android/iOS)

Tencent Neural Network — mobile and edge inference

NVIDIA's optimized LLM inference library (GPU)

NVIDIA's high-performance deep learning inference SDK (GPU)

High-throughput LLM/VLM serving with RadixAttention and structured generation

Single-file LLM executables via Cosmopolitan Libc — zero install, all platforms (~21K stars)

High-performance LLM inference in web browsers via WebGPU

Alibaba's neural network inference framework for mobile & edge

Apache TVM ML compiler — auto-tunes models for any hardware target

Apache TVM Foreign Function Interface for deep learning compilation

Audio models (TTS, ASR) with MLX

Vision-language models on Apple silicon via MLX

Swift audio inference using MLX

Text embedding models with MLX

Video model inference with MLX

ONNX-based runtime for ASR, TTS, VAD, and keyword spotting

NVIDIA Triton — production multi-model inference server (HTTP/gRPC, multi-backend)

Intel's toolkit for optimizing & deploying deep learning on Intel hardware

OpenVINO GenAI — generative AI layer with speculative decoding & KV-cache opt

DNN inference library & tutorials for NVIDIA Jetson

Intel IPEX-LLM — local LLM acceleration on Intel hardware (archived Jan 2026, read-only)

Unified SDK for running LLMs and multimodal models locally

High-throughput LLM serving with TurboMind engine (C++/CUDA)

Lightweight inference engine for mobile & embedded from PaddlePaddle

On-device Whisper inference for Apple platforms (Swift)

Python tooling for WhisperKit model optimization

On-device AI benchmarking framework

Swift playground for ArgMax SDK

Cactus core edge inference framework

React Native bindings for Cactus

Flutter bindings for Cactus

Kotlin/Android bindings for Cactus

Function-calling Gemma hackathon project

Demo chat app using Cactus

Org profile and community health files

Distributed LLM inference via Cactus

Fast model transfer protocol

Fine-grained model transfer protocol

Homebrew tap for Cactus

Open-source claw project

Open-source code project by Cactus

PyTorch's portable execution framework for on-device inference

High-performance EXL2-quantized inference for consumer NVIDIA GPUs

TensorFlow Lite for microcontrollers and embedded devices

On-device audio inference framework

Fluid Inference core runtime

Rust text processing library for inference

Mirai's on-device inference runtime

Mirai's LLaMA-based on-device model

Swift SDK for Uzu

Multimodal LLM inference framework for mobile & edge

ARM Neural Network SDK for ARM & Mali devices

NimbleEdge's deliteAI on-device inference framework

NimbleEdge fork of ExecuTorch with edge optimizations

Picovoice's on-device LLM inference engine

AMD's graph inference engine for MI-series GPUs

MLange sample applications

MLange extension library

iOS sample app for MLange

Swift LLM template for iOS

Flutter SDK for MLange

Android sample app for MLange

Kotlin LLM template for Android

React Native LLM template

Org profile and community health files

Zetic fork of Google AI Edge Torch

Convert ONNX models to PyTorch

MLange SDK documentation

iOS framework for MLange

iOS extension framework for MLange

Flutter LLM template

Whisper sample app using MLange

YOLOv8 demo using MLange

Simplified YOLOv8n demo using MLange