Build software better, together

FareedKhan-dev / kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

Updated Oct 2, 2026
C

OpenMOSS / MOSS-TTS-Nano

Star

A 100M-parameter multilingual TTS model for real-time CPU inference, voice cloning, and 48 kHz stereo generation

multilingual text-to-speech realtime tts speech-synthesis english chinese streaming-audio onnx voice-cloning multi-modality cpu-inference voice-clone audio-tokenizer

Updated Sep 6, 2026
Python

feder-cr / jev

Sponsor

Star

jevos is an open-source alternative to Jev for yes/no decisions that runs on your laptop.

Updated Oct 9, 2026
C++

kennethleungty / Llama-2-Open-Source-LLM-CPU-Inference

Star

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

Updated Nov 6, 2023
Python

Vui Nano — a small, context-aware text-to-speech model trained on real conversations. 219M active params (305M total), Apache 2.0, voice cloning, streaming, runs on CPU (dependency-free C build). Ships with a full real-time voice assistant: WebRTC, ASR, local LLM, OpenAI Realtime API compatible.

Updated Oct 10, 2026
Python

lucienhuangfu / eLLM

Star

eLLM: Run Long-Horizon Inference Faster on CPUs Than on GPUs

cpu-inference long-horizon qwen llm-infernece rust-llm kimi-k3

Updated Oct 7, 2026
Rust

brontoguana / krasis

Star

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

transformer inference-engine inference-optimization mixture-of-experts cpu-inference large-language-models gpu-inference llm-inference high-performance-inference hybrid-inference gguf-model-support llama-cpp-alternative

Updated Sep 15, 2026
C++

facex-engine / facex

Star

Full face stack that runs entirely in the browser. Detection, 576-point 3D mesh, recognition, anti-spoof, smile — all WebAssembly, zero server. Apache 2.0.

computer-vision deep-learning c99 simd biometrics face-recognition low-latency avx2 production-ready face-verification face-embedding avx-512 zero-dependencies embedded-ai edge-ai onnx-runtime cpu-inference

Updated May 14, 2026
C

Dicklesworthstone / franken_ocr

Sponsor

Star

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

rust ocr neon simd moe quantization avx512 int8 cpu-inference int4 vision-language-model deepseek-ocr

Updated Oct 6, 2026
Rust

notch-up / diarize

Star

Speaker diarization for Python — "who spoke when?" CPU-only, no API keys, Apache 2.0. ~10.8% DER on VoxConverse, 8x faster than real-time.

python audio-analysis speech-to-text speaker-recognition speech-processing speaker-diarization spectral-clustering voice-activity-detection onnx speaker-embedding diarization apache-2 rttm cpu-inference meeting-transcription who-spoke-when

Updated May 6, 2026
Python

gabriele-mastrapasqua / qwen3-tts

Star

Pure C inference engine for Qwen3-TTS text-to-speech. No Python, no PyTorch — just C and BLAS. Supports 0.6B and 1.7B models, 9 voices, 10 languages.

multilingual c text-to-speech metal cuda emotion inference simd tts speech-synthesis pure-c inference-engine voicecloning cpu-inference voiceclone qwen qwen-tts qwen3-tts

Updated Sep 25, 2026
C

CoderLSF / fast-llama

Star

Runs LLaMA with Extremely HIGH speed

llama inference-engine cpu-inference llama2

Updated Nov 21, 2023
C++

kouhxp / gutsy

Star

Local decision model with calibrated probabilities: send a state and yes/no, choice or score questions, get a probability for every option. 0.8B GGUF on CPU, Jev-style API.

python classifier routing decision-model agents intent-classification content-moderation cpu-inference llm llama-cpp local-llm qwen gguf small-language-models tool-selection llm-judge jev jev-compatible

Updated Oct 5, 2026
HTML

saurabhv749 / supertonic3-voice-clone

Star

Train voice styles for Supertone/supertonic-3 model.

text-to-speech speech-synthesis speaker-embedding voice-cloning onnxruntime fast-tts tts-model cpu-inference supertone hindi-tts supertonic-3 voice-builder

Updated Jul 8, 2026
Python

FareedKhan-dev / glm-5.2-in-c

Star

GLM-5.2, a 744 billion parameter mixture of experts model, in a pure C inference engine: quantized to int4, experts streamed from disk, deployed and benchmarked. Generates in 16 GB of RAM.