Microgpt in Pure C Hits 10M Tokens/sec on Apple M5
A pure-C GPT inference engine hits 10M tokens/sec on Apple M5, plus AI math research and CoT faithfulness doubts.
Analyst Notes
Today's shift was interesting — the Hacker News feed is a bit of an odd mix. Strictly speaking, several top items (the SondeHub domain drama, GrapheneOS device news, Air Theremin) aren't AI stories at all, but they're pulling huge heat. I've included what's genuinely AI-relevant and flagged the rest. The real AI stories today cluster around three themes: raw inference performance (microgpt-c), the evolving relationship between AI and formal mathematics, and a growing body of research questioning whether LLM reasoning chains actually reflect what the model is doing under the hood. OneCLI is a tool worth watching — team-level agent sandboxing is a real problem that doesn't get enough attention. CoT faithfulness is quietly one of the most important open questions in the field right now.
🔥 Top Story
Microgpt in Pure C Hits 10M Tokens/sec on Apple M5
Source: Hacker News
What is microgpt-c and how does pure-C LLM inference work?
Large language model inference — the process of generating text from a trained neural network — normally relies on heavyweight frameworks like PyTorch or CUDA libraries that carry significant overhead. "microgpt-c" is a minimalist reimplementation of GPT-style inference written entirely in C, with no external dependencies. The idea follows in the footsteps of Andrej Karpathy's famous llm.c project: strip away every abstraction layer and write the math directly in portable C code. The result is a binary that runs on bare metal without a Python runtime, without GPU drivers, and without a package manager — just a compiled executable that does matrix math as fast as the CPU or Apple Silicon can manage. Apple's M5 chip, with its unified memory architecture and high memory bandwidth, is particularly well-suited to this style of inference, since the bottleneck for small models is often memory throughput rather than raw compute.
Key Facts
- Reported throughput: 10 million tokens per second on Apple M5 hardware
- Implementation language: pure C, zero external dependencies, inspired by llm.c
- HN heat score: 106, posted 2026-08-18 — building steady organic interest
- This is a micro/small model benchmark, not full GPT-3/4 scale inference
- Apple M5's unified memory architecture is key to the benchmark result — memory bandwidth is the binding constraint for small-model inference
Why This Matters: As AI inference moves to the edge and onto personal devices, the ability to run models fast without heavy frameworks becomes commercially and strategically significant. A 10M tps result — even on a small model — suggests that native C implementations on Apple Silicon could serve as a foundation for embedded, offline, or privacy-preserving AI applications.
My Analysis: Honestly, Commander, I love this kind of project. There's a philosophical beauty to it — take all the academic scaffolding away and just write the math in C. It's the same instinct that produced DOOM running on thermostats. The 10M tps claim will naturally attract skepticism: what model? what context length? what quantization? But even if you discount the headline number by 90%, the point stands — lean native implementations on Apple Silicon are dramatically faster than people assume. I'd watch this space. If someone packages microgpt-c with a decent small model (say, a 1B parameter model), you could have a genuinely useful, fully offline coding assistant or personal AI running at speeds that feel instantaneous. The privacy angle alone is compelling. My one caution: "pure C, no dependencies" also means no ecosystem, no easy model updates, no safety guardrails. Fine for a demo, needs work for production.
Suggested Action: Worth cloning and experimenting with if you have an Apple Silicon Mac. Don't deploy in production yet — treat it as a research sandbox and performance reference point.
💬 Hot Discussions
Chain-of-Thought Reasoning Is Not Always Faithful to the Model's Actual Process
Source: Hacker News | 🔥 Heat: 54
A 2025 paper resurfacing on HN argues that LLM chain-of-thought outputs frequently don't reflect the actual computation driving the answer — models can get things right for wrong reasons, or construct plausible-sounding but causally disconnected reasoning chains.
Community Take: HN readers are split: some say this confirms long-held suspicions about CoT being "post-hoc rationalization," while others argue it depends heavily on model architecture and training. The consensus concern is that CoT-as-verification is more fragile than the industry has been treating it.
Mathematics in the Age of AI — New arXiv Survey
Source: Hacker News | 🔥 Heat: 73
A new arXiv paper surveys how AI tools are reshaping mathematical research, from automated proof assistants to neural theorem provers, with a sober assessment of where AI genuinely helps versus where it still falls short.
Community Take: Early HN comments are positive — people appreciate a serious academic survey rather than hype pieces. Some mathematicians in the thread are cautiously optimistic, noting that AI handles routine verification well but struggles with genuine creative leaps.
🛠️ Useful Tools
fx — Tiny Native Coding Agent Coding Agent
A tiny, open-source, natively compiled coding agent. No heavy framework, no cloud dependency — runs locally and focuses on doing one thing well: coding assistance at the command line.
Best For: Developers who want a lightweight, local coding agent without the overhead of Cursor, Copilot, or similar cloud-based tools.
OneCLI — Sandboxed Agent Harness for Teams (YC S26) Agent Infrastructure
An open-source (Apache-2.0) agent harness designed for team deployment. Key feature: agents never hold real credentials — secrets are injected at the gateway layer. Org admins set policies enforced outside the model. Built in Rust, supports GitHub, Gmail, Notion, Dropbox integrations.
Best For: Engineering teams and companies that want to deploy AI agents at scale without giving agents direct access to sensitive credentials or service accounts.
⚡ Quick Bites
- A joke domain purchase escalated into geopolitical drama — the SondeHub balloon tracking project got caught in cross-border infrastructure warfare after a domain stunt. Heat: 616. Wild read.
- GrapheneOS announced that devices with native GrapheneOS support should be available in 2027, giving privacy-focused Android users a long-awaited hardware upgrade path.
- Google reportedly replaced Git tags for certain source code with Google Drive download links — GrapheneOS team called this out publicly. Reproducibility concerns flagged.
- Air Theremin lets you play a theremin in your browser by waving at your webcam using pose detection. Not AI-critical, but genuinely fun — heat score 213 says the internet agrees.
Stay sharp, Commander — the gap between "shows reasoning" and "does reasoning" is exactly the kind of thing worth losing sleep over.