Inference Radar

TL;DR

vLLM reset its serving baseline: vLLM made its newer model runner the default and paired it with stronger structured-output, connector, ROCm, and multimodal safety work.1
SGLang scaled the control plane: SGLang hardened Rust routing, cache and offload paths, multi-accelerator support, and speech serving through SGLang-Omni.2
Ollama crossed into ChatGPT Desktop: Ollama added direct local model use inside ChatGPT Desktop on macOS while improving Codex and OpenAI Responses compatibility.3
llama.cpp pushed every backend at once: llama.cpp shipped a stable release and kept tuning Vulkan, CUDA, HIP, Metal, OpenCL, SYCL, Hexagon, and server paths.4
Kernel libraries raced ahead of frameworks: FlashInfer expanded Blackwell-era attention, MLA, MoE, KDA, and quantized KV support that downstream servers are already pulling on.5

This Week in Inference

The model story did not come from one confirmed public launch. It came from runtime teams racing to absorb fast-moving model families and modality mixes: Hugging Face added new model integrations and compatibility work in Transformers, SGLang expanded support for large Qwen and GLM-style serving targets, and vLLM extended Kimi, Gemma, Qwen, Molmo, OpenPangu, and multimodal paths across its server stack (Transformers, SGLang, vLLM).1 The practical read is simple: inference teams spent the week making model support survivable under batching, tool use, long context, multimodal payloads, and hardware-specific kernels.

Optimization work clustered around KV-cache shape, low-bit kernels, and speculative decode. vLLM moved connector and offload logic toward per-request safety, SGLang hardened HiCache and offload behavior, llama.cpp pushed backend-specific GPU and CPU kernels, FlashInfer expanded attention and MLA kernels for newer NVIDIA targets, and ROCm AITER plus ATOM advanced AMD attention, MoE, and offload paths (vLLM, SGLang, llama.cpp, FlashInfer, ROCm AITER).1 The work tells a clear story: serving speed now depends as much on cache placement, routing, and kernel selection as on raw matmul throughput.

Hardware and infrastructure work spread across the stack. NVIDIA projects advanced TensorRT-LLM and TensorRT-Edge-LLM, ROCm pushed MI-class serving kernels, Google moved LiteRT and XNNPACK for edge and Arm paths, Qualcomm hardened GenieX for Snapdragon, and Apple’s MLX ecosystem kept tuning kernels, memory accounting, model loading, and Swift inference (TensorRT-LLM, ROCm ATOM, LiteRT, GenieX, MLX).8 Industry moves were quiet in the supplied data, so the useful signal came from code: open-source inference teams are treating cloud, desktop, mobile, and browser targets as one deployment problem.

Deeper Dive

Everything below is for readers who want the full picture. Feel free to scroll.

Code Changes by Category

Cloud & Datacenter Serving

vLLM had the cleanest datacenter story: the team shipped v0.29.0, made Model Runner V2 the default, and tied the release to parser hardening, ROCm/AITER updates, KV connector cleanup, structured-output failures, multimodal shared-memory cache safety, and late-interaction cache key security work (vLLM release, GHSA cache fix).13 vllm-gaudi tracked upstream churn with HPU fixes for Qwen FP8 MoE, Gemma/OpenAI sync, FusedSDPA, MLA, MoE routing, compact-GDN state safety, and CI updates (vllm-gaudi).20

SGLang shipped v0.5.19 and moved both core SGLang and SGLang-Omni around Rust routing, HiCache, Blackwell, ROCm, NPU, XPU, GLM, Qwen, Kimi, diffusion, and speech-serving paths (SGLang release).14 ai-dynamo focused on distributed serving reliability, adding admission gating, batched QUIC transport, ZMQ thread controls, vLLM/SGLang disaggregation, LMCache paths, multimodal EPD, LoRA fixes, and Kubernetes operator API updates (Dynamo).21

NVIDIA TensorRT-LLM advanced VisualGen attention, FP8 and FP4 paths, KV-cache windows, a PyTorch executor ModelRunner contract, Anthropic Messages support, Nemotron VL serving, speculative decoding, guided decoding, Kimi K3, Qwen Flash Next, and MoE infrastructure (TensorRT-LLM VisualGen).22 TensorRT Inference Server focused on production hardening, fixing shared-memory null dereferences, SageMaker MME model identity confusion, Python backend QA, and deprecated developer-tool docs (Triton server).23

LMDeploy reworked cache planning, backend operator build specs, piecewise CUDA graph prefill, SparseMLA, online FP8 exclusions, TurboMind C++20 builds, tool-choice validation, multimodal rejection, and cache metrics (LMDeploy cache redesign).24 LightLLM focused on PD serving, decode request splitting, prompt-cache-aware node selection, KV-transfer memory reservation, decode-attention autotuning outside CUDA Graph capture, and NIXL race fixes (LightLLM PD split).25

LiteLLM was the gateway outlier by volume, with work on MCP OAuth scoping, router lookup, JWT cache deletion, auto-routing, provider compatibility, Responses API streaming, cost accounting, OTEL traces, Langfuse sessions, and Docker signature notes across multiple releases (LiteLLM).26 Ray Serve worked on replica and ingress reliability, HAProxy behavior, Serve LLM model sources, metrics aggregation, direct ingress lifecycle, and Ray Data retry correctness (Ray Serve).27

Local LLM Runtimes

llama.cpp shipped v0.4.0 and continued its high-frequency binary train across Vulkan, CUDA, HIP, Metal, OpenCL, SYCL, Hexagon, WebGPU, server, grammar, Jinja, multimodal, and model support (llama.cpp release).17 whisper.cpp synced ggml v0.23.0, inherited backend improvements, and fixed language autodetect cancellation, same-process determinism, dynamic-backend tests, server JSON, stale automation, and Ruby binding lifecycle checks (whisper.cpp ggml sync).28

Ollama shipped v0.34.0 with ChatGPT Desktop integration, Codex compatibility, tool-search support, compaction, Responses API fixes, MLX structured-output speedups, and scheduler race fixes (Ollama release).16 LocalAI had a large reliability week around Qwen agent templates, Python backend request shaping, SGLang and vLLM seed preservation, diffusers device detection, distributed multimodal staging, quantization progress, realtime voice profile switching, UI fixes, CVE dependency work, and backend pin updates (LocalAI).29

exllamav3 shipped v1.4.7 and v1.4.8, with CPU MoE acceleration, DRY sampling, recurrent slot-leak fixes, cache quantization for newer Flash models, pyproject.toml, expandable CUDA segments, and FlashAttention-2 removal (exllamav3 release).30 llamafile merged ZIP parser hardening for offset, size, ZIP64, and extra-field bounds, then restored build recipes for bundled llama.cpp tools (llamafile ZIP fix).31

WebLLM shipped v0.2.85, tightened JavaScript NaN validation for generation parameters and logit bias, updated the bundled web runtime, improved cache handling, and drew proposals for Chrome Built-in AI Prompt API compatibility and MCP demos (WebLLM release).32 Osaurus, Rapid-MLX, oMLX, ds4, OpenArc, vllm-mlx, uzu, and Wally all pushed local app and desktop-serving paths around routing, tool parsing, Apple Silicon execution, benchmark sharing, and hosted agent launch flows (Osaurus, Rapid-MLX, oMLX).33

Apple Silicon & MLX Ecosystem

Apple’s MLX core added M5 Ultra GEMM tuning, quantized matmul and gather fixes, NVFP4 global scales, SDPA/NAX speedups, stream/runtime fixes, CUDA robustness, autograd fixes, memory accounting, and Windows file handling (MLX M5 GEMM).36 mlx-lm added PLaMo, Talkie, XVERSE, Granite MoE Hybrid fixes, server and streaming work, cache-state fixes, evaluation cleanup, and CPU/pipeline fixes (mlx-lm).37

Core AI projects added Gemma 3n macOS export, ATEM tool parsing, Muse Glimmer VLM and speculative decoding export, Core AI Torch debug tooling, Core ML conversion fixes, and compression/palettization work (coreai-models).38 Blaizzy’s mlx-vlm added GLM-5 Next, PP-DocLayoutV3, MoGe-3, DINOv2, SAM video tracking, Mage-VL, MiniCPM-o TTS, and speculative verifier work, while mlx-audio added Nemotron VoiceChat, live STT sessions, VibeVoice ASR streaming, Breeze-TTS segmentation, MiMo-Audio, DialogueSidon, and CI fixes (mlx-vlm release, mlx-audio release).39

Rapid-MLX added hybrid sliding-window KV quantization, MTP telemetry, FLUX image precision paths, Desktop Computer Use workflows, loopback-only binds, credential protections, and private-by-default benchmark sharing (Rapid-MLX release).34 oMLX advanced Cluster v2, Qwen decode speed, SSD-backed PLE lookup, long-context scheduler safety, prefix-cache guardrails, and Apple memory pressure handling (oMLX Cluster).35

Mobile & Edge Frameworks

ExecuTorch absorbed the PyTorch 2.14 transition, reworked headers and C++20 handling, improved AOTI shims, hardened ROCm/macOS/build image paths, and expanded LLM runner, batched generation, SmolLM export, Vulkan delegate, Arm/Ethos-U, Qualcomm, NXP, XNNPACK, MLX, and Samsung backend paths (ExecuTorch PyTorch pin).41 Google’s LiteRT and LiteRT-LM moved on Gemma, MoE dynamic shapes, OpenVINO gather rewrites, QNN/HTP, streamed loading, ASR, Omni audio, NPU setup, constrained decoding, Apple/Swift builds, and metrics (LiteRT-LM release).42

ncnn fixed FP16 tanh correctness on ARM and RISC-V after SigLIP2-style failures, added Adreno cooperative-matrix workarounds, improved Apple A19/M5 detection, expanded BF16 support, fixed Vulkan BF16 without shaderInt16, and tightened CPU topology parsing (ncnn FP16 fix).43 Qualcomm’s GenieX shipped two releases while ai-hub-models added SmolLM2, native KV-cache infrastructure, Llama 3.2 SSD paths, QAIRT runtime controls, ModelScope pulling, Snapdragon accuracy workflows, Adreno OpenCL fixes, and VLM device selection (GenieX release).44

sherpa-onnx added Termux Android builds, Android Python wheels, Windows arm64 Go support, ONNX Runtime updates, OpenVINO Intel NPU support, silence-hallucination fixes across ASR models, diarization confidence scores, Arabic TTS assets, and download retry hardening (sherpa-onnx Termux).45 Cactus shipped a Parakeet audio encoder fix that added explicit input_features_mask handling and improved padded-frame behavior, while Needle moved on multi-generation engines, audio APIs, embeddings, WASM publishing, and finetuning seeds (Cactus release).46

FluidAudio fixed final-window ASR loss, custom vocabulary boosting, false replacements, Kokoro ANE voice-pack conversion, PocketTTS compute-unit routing, Swift package bloat, and diarization cancellation (FluidAudio).47 mllm added MiniCPM5-2B ARM CPU inference with configs, KAI W4A32 quantization, compatibility checks, GQA tests, KV-cache tests, and a pending prompt-preprocessor refactor (mllm MiniCPM5).48

Compilers, Runtimes & Graph Engines

Triton split sanitizer and membar passes, expanded ConSan and FPSan tracking, added Blackwell and Hopper FP4 layout work, atomic load/store ops, Gluon IR compilation, AMD/Gluon support, and layout-pipeline restructuring, while reverting two unstable rematerialization changes (Triton FP4).49 Apache TVM and tvm-ffi overhauled structural traversal and mutation APIs, upgraded TVM’s vendored FFI, fixed Relax/ONNX correctness, updated PTX tables, and hardened LLVM, Metal, and CUDA backend behavior (tvm-ffi structural work).50

OpenXLA and TensorFlow shared a heavy compiler week: XLA deferred CUDA VMM deallocation during stream capture, added CPU compile paths, prepared AMX_FP8 detection, rolled Triton and LLVM, refined Shardy behavior, and expanded PJRT/IFRT APIs (OpenXLA CUDA graph fix).51 JAX added native log2, fixed GPU min/max gradient drift, improved Pallas and Mosaic lowering, worked on free-threaded Python readiness, and pulled XLA and Triton updates (JAX log2).52

OpenVINO core added Intel GPU MLIR graph-compiler integration, BF16 SDPA, stateless LLM SDPA alignment, NPUW cache behavior, ONNX and PyTorch frontend fixes, GGUF accuracy work, and CPU/TEMPLATE hardening (OpenVINO GPU compiler).53 OpenVINO GenAI added LTX image-to-video, EAGLE3 KV-cache precision fixes, VLM input cleanup, PagedAttention fallback logging, Kokoro fixes, TTS benchmark metrics, and supported-model docs (OpenVINO GenAI LTX).54

ONNX fixed reference semantics for Unique, normalization, GatherElements, LpNormalization, ConstantOfShape, GlobalLpPool, MaxUnpool, and Resize, while ONNX Runtime shipped a patch release and added CUDA, WebGPU, WebNN, CPU/MLAS, GQA, paged attention, TopK, BF16 reduction, plugin EP, and build fixes (ONNX Runtime release).55 Modular pushed MAX quantization, NVFP4 checkpoints, MXFP6 coverage, QSA attention, fused projections, grouped NVFP4 tuning, GraphCompiler fusion, dKV metrics, Mojo compiler changes, and stdlib fixes (Modular).56

Models, Quantization & Optimization

Hugging Face shipped Transformers v5.17.0, led by HYV4 Preview support, Fun-ASR-Nano, Kimi Linear, vision RoPE/MRoPE fixes, GGUF generation work, chat fixes, quantization updates, and backend correctness (Transformers release).57 Optimum Intel added Qwen MTP/speculative decoding and Gemma OpenVINO fixes, Diffusers moved on modular video pipelines and offloading, and Candle removed candle-ug, switched tokenizer regex dependencies, and cleaned up Rust CI (Optimum Intel, Diffusers).58

FlashAttention focused on FA4 correctness for SM90, SM100, SM120, and Blackwell-era backward paths, including non-aligned head dimensions, scheduler window handling, CuTe JIT cache hashes, invalid varlen tile guards, and hdim-64 backward speedups (FlashAttention release).60 FlashInfer expanded Cake kernels, PrimTS contracts, FP8 and NVFP4 KV handling, DeepSeek/MLA support, MoE quant config, KDA, GDN, ReplaySSM, sampling, JIT cache, and CI (FlashInfer release).61

Intel neural-compressor added NVFP4_E5M3 and AutoRound QDQ support for vLLM paths, CuTe non-contiguous QDQ handling, Flux MXFP8 example updates, ChatGLM2 tokenizer trust behavior, and JAX serialization cleanup (neural-compressor NVFP4).62 ROCm AITER shipped two post releases with gfx942 and gfx950 wheels, while ATOM consumed AITER for GLM, Kimi, MiniMax, DeepSeek, KDA, MLA, MoE, RCCL, offload, and serving telemetry (AITER release).63

Community Pulse

SGLang had the largest visible momentum jump among core serving engines, with issue traffic around GLM cache corruption, Blackwell qualification, NVFP4 MoE memory retention, Kimi tool grammar, DFlash sizing, and B300 HiCache/CUDA graph hangs (SGLang issues).64 Open WebUI kept heavy community volume around tool servers, OpenAPI strictness, ingestion, citations, provider compatibility, cancellation, mobile Safari, and release test ownership (Open WebUI issues).65

Ollama’s community centered on MLX context behavior, YaRN, prefix-cache restore behavior, low-bit Qwen artifacts, tool-calling regressions, and ChatGPT Desktop/Codex workflows (Ollama issues).66 Apple MLX users pressed on distributed multi-Mac reliability, compile-cache resource use, memory leaks, server deadlocks, paged KV storage, Core ML packaging, and Swift LLM feature requests (MLX issues, mlx-lm issues).67

The edge community supplied strong deployment feedback. Qualcomm users reported Hexagon corruption, Adreno OpenCL crashes, Android JNI error swallowing, QAIRT performance gaps, Gemma VLM keepalive failures, and encoder device-selection bugs (GenieX issues).69 sherpa-onnx users focused on redistribution and commercial licensing for ASR, KWS, diarization, and mobile models, which shows that edge inference adoption now runs into packaging law as much as kernel speed (sherpa-onnx issues).70

Community Debates

Ollama’s MLX context-length debate was the week’s clearest product philosophy split. A proposal to enforce scheduler-selected context in MLX was rejected because maintainers want MLX to use trained context length, while contributors argued that this blocks long-context YaRN use cases for Qwen-style models (Ollama context PR).71

Mooncake drew a hard line between availability and durability in its Store contract. A proposal to fail PutEnd on OpLog admission errors did not land because maintainers want the HA cache model to preserve latency and availability even if recent acknowledged cache data may be lost (Mooncake Store debate).72

MLX maintainers pushed back on low-confidence and boundary-crossing contributions. Integer einsum fallback and shapeless Scan compile support closed amid process concerns around AI-generated submissions, contributor understanding, and solving issues before code changes (MLX einsum PR, MLX Scan PR).73

Google and OpenXLA showed strict contribution and import workflow boundaries. Gallery and LiteRT-LM closed several external PRs in favor of internal import flows, while OpenXLA closed multiple technically detailed PRs under its AI-generated-contribution policy (Gallery PR, OpenXLA policy PR).75

Worth Watching

KV stores are becoming shared infrastructure: Mooncake, Dynamo, vLLM, SGLang, LMDeploy, LightLLM, and oMLX all changed cache ownership, identity, offload, or scheduler semantics this week (Mooncake, vLLM connectors).77
Browser inference wants a standard API surface: WebLLM’s Prompt API compatibility issue could make open WebGPU models look like browser-native AI to app developers (WebLLM Prompt API issue).79
C++20 is spreading through inference builds: TensorRT-LLM, LMDeploy TurboMind, Triton Server, DeepSpeed, and PyTorch-adjacent projects all touched newer compiler and PyTorch header requirements (LMDeploy C++20, Triton Server C++20).80
Speculative decoding needs workload-specific proof: Rapid-MLX, exllamav3, SGLang, Ollama, and LightLLM all surfaced cases where MTP or draft paths can help, regress, or need narrower validation (Rapid-MLX MTP issue, LightLLM speculative issue).82
Speech is moving into the same serving stack as text: SGLang-Omni, vLLM-Omni, mlx-audio, sherpa-onnx, FluidAudio, LiteRT-LM, and Cactus all shipped or staged ASR, TTS, realtime, or audio-encoder work (SGLang-Omni, vLLM-Omni).15

Major Releases

vLLM shipped v0.29.0, centered on Model Runner V2 becoming the default, CUDA graph memory profiling for KV-cache auto-sizing, batch-sharded sampling, prompt embeds, speculation hidden-state extraction, and padded FULL cudagraph dispatch. The release also caps a week of parser, connector, ROCm, security, and multimodal hardening..13

SGLang shipped v0.5.19, with notes citing 786 PRs from 214 contributors and new model support including Qwen3.8 2.4T-A95B and Qwen3.8-27B. The release sits on a broader week of Rust routing, HiCache, multi-accelerator, diffusion, and Omni speech work..14

ggml shipped llama.cpp v0.4.0 plus a dense prerelease binary train from b10770 to b10883. The stable release highlighted Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, on-demand tensor reading, per-slot server context limits, video input options, and ggml 0.23.0..17

Ollama shipped v0.34.0, focused on direct Ollama model use in ChatGPT Desktop on macOS, Apple Silicon structured-output performance, and OpenAI-compatible support work. The release matches a week of Codex, tool-search, compaction, and MLX reliability work..16

Hugging Face shipped Transformers v5.17.0, led by HYV4 Preview support, described in the notes as a 780B-parameter MoE LM with 49B active parameters per token, top-8 routing, and 1M-token context. The same week pushed Qwen, Gemma, VLM, GGUF, generation, and quantization compatibility across Transformers, Optimum Intel, Diffusers, and Candle..57

Google AI Edge shipped LiteRT-LM v0.17.0, with optimized local attention for lower memory and longer contexts, Apple Silicon Metal residency support, Gemma 4 12B multimodal, MTP, extended-context support, and stability fixes. The broader Google week also moved LiteRT, LiteRT-Torch, XNNPACK, Gallery, MediaPipe, samples, and Model Explorer..42

NVIDIA shipped TensorRT-Edge-LLM v0.10.1 and TensorRT-LLM v1.3.0rc26. TensorRT-Edge-LLM added DGX Spark TP=2, JetSpec/Qwen3 speculative decoding, FP8 ViT attention, DART pruning, media caching, Nemotron 3 Super 120B-A12B NVFP4 MoE, Blackwell kernels, and server footprint reductions. TensorRT-Edge-LLM release notes.85

BerriAI shipped four LiteLLM releases, from v1.101.0-dev.2 through v1.102.0-dev.1, with release notes focused on cosign verification for signed Docker images. The code stream was far broader, spanning MCP OAuth, router correctness, provider compatibility, Responses API behavior, observability, billing, and UI safety. Latest release notes.86

ROCm shipped AITER v0.1.21.post1 and v0.1.21.post2, both publishing manylinux wheels for gfx942 and gfx950 across ROCm and Python combinations. The releases tracked a week of AITER attention, MQA, MLA, MoE, GEMM, FP8, MXFP8, and ATOM integration work for GLM, Kimi, MiniMax, and DeepSeek paths. Latest release notes.63

FlashInfer shipped v0.6.18.post1, a patch release following v0.6.18. The release followed heavy work on SM100/103/110/120/121 Cake kernels, PrimTS attention contracts, quantized KV fixes, DeepSeek/MLA, MoE, KDA, GDN, ReplaySSM, sampling, JIT cache, and CI..61

Dao-AILab shipped fa4-v4.0.0.beta30, a FlashAttention prerelease with SM80/SM120 invalid-varlen-tile guards, callable-default JIT hash fixes, SM100 non-32-aligned backward support, and SM100 hdim-64 backward speedups. The release focused on FA4 correctness and Blackwell-era training stability..60

Microsoft shipped ONNX Runtime v1.28.2, a patch release for Compile API serialization that prevents duplicate graph nodes, inputs, outputs, and value info in optimized models, including embedded and external initializer cases. The wider Microsoft week also advanced Foundry Local, Olive, ONNX reference semantics, and ONNX Runtime provider work..55

MLC AI shipped WebLLM v0.2.85, with OPFS synchronous access handles, committed cache records, a bundled TVM web runtime update, OpenAI-compatible created timestamps, service-worker startup handling, worker model-state recovery, structural-tag hang prevention, NaN generation-parameter rejection, and safer example output rendering..32

Blaizzy shipped mlx-vlm v0.7.0 and mlx-audio v0.5.2 plus v0.5.3. The releases focused on Qwen4 shard gathering, native quantized KV fixes, Apodex, Qwen3.8-Flash-Next MTP, VibeVoice-ASR streaming, Voxtral stopping, OmniVoice tokenizer handling, and Granite Speech kwargs. mlx-vlm release notes.39

Cactus Compute shipped cactus v2.2.0, centered on Parakeet padded-frame and feature-normalization fixes, a custom transpiler, graph-side mmap weight deduplication, and a new contributor callout. Needle did not ship a release but advanced multi-generation engines, audio input, embeddings, WASM publishing, and finetuning reproducibility..46

AMD Ryzen AI shipped FastFlowLM v1.0.5, adding hy-mt2:1.8b, CLI/server docs, benchmark fixes, long-prefill fixes, decode and prompt-cache optimizations, HRX build maturation, refreshed HRX engines, and contributor guidance. Lemonade consumed the new backend versions and extended HRX coverage..87

Qualcomm shipped GenieX v0.6.0 and v0.6.1. The releases covered Windows/Linux geniex-bench tutorials, QDC retry behavior, llama.cpp MTP speculative decode alignment, VLM accuracy prompt chat-template fixes, and Microsoft-signed Hexagon HTP catalog handling for Windows on Snapdragon. Latest release notes.44

kvcache-ai shipped Mooncake v0.3.14-rc1, a prerelease with 24 assets and an empty captured release body. The week’s code work was richer than the notes, covering Store HA, liveness, placement, soft pins, transport, TENT, CUDA VMM dma-buf ranges, build pipelines, and KTransformers hybrid DeepSeek serving..88

ai-dynamo shipped four experimental v1.5.0 dev snapshots for Kimi-K3, DeepSeek-V4-Pro-0813, Gemma-4-31B, and Inkling. The dev releases matched a week of runtime transport, router, EPP scheduling, vLLM and SGLang disaggregation, multimodal EPD, LoRA, Kubernetes operator, recipes, and observability work. Kimi-K3 dev release notes.89

RunanywhereAI shipped Wally v0.5.3, Wally v0.5.4, and cpp-desktop-v0.20.37. The releases turned RCLI into Wally, added hosted Codex launcher support, tarball install paths, Windows quoting fixes, nightly/dev installer support, default-model settings, ANE model rows, embedding input types, SDK updates, and quieter desktop C++ kit logs. Wally release notes.90

Osaurus shipped 0.24.5, 0.24.6, 0.24.7, and 0.25.0. The cadence focused on workspace agents, shared agents, team inference, memory warnings, MiniCPM5 controls, vMLX repins, MTP telemetry, project folders, knowledge links, Full Disk Access probes, and model-routing fixes. Latest release notes.33

Rapid-MLX shipped v0.13.4 and rapid-mac-v0.13.4. The releases added faster qualified Qwen artifacts, an experimental video workspace, private-by-default Community Benchmark sharing, memory fitting, model discovery, Markdown rendering, headless fixes, local video generation for LTX, Wan, and CogVideoX-Fun, queued jobs, progress, cancellation, restart-safe results, and early memory validation..34

try-mirai shipped uzu 0.5.23, 0.5.25, and 0.5.26, all automated package and binding metadata releases with little or no textual notes. The code behind them improved Metal build generation, stream memory use, IOReport power metering, tool-call parsing, Qwen and Muse behavior, Hugging Face download handling, file download groups, and binding docs. Latest release notes.91

turboderp-org shipped exllamav3 v1.4.7 and v1.4.8. The releases focused on CPU MoE performance, DRY sampler support, recurrent slot leaks, trellis quantization speed, pyproject.toml, cache quantization for DSv4-Flash/Qwen3.8-Flash-Next/GLM5.3-Flash, VRAM allocation, expandable CUDA segments, and FlashAttention-2 removal. Latest release notes.30

OpenArc shipped v2.0.5, its last release before upcoming versioning changes. It added generation support for seed, frequency_penalty, and presence_penalty, fixed generation-kwargs return behavior, and updated Docker GPU dependencies from level-zero to libze-intel-gpu1..92

Ollama Invades ChatGPT Desktop