https://blog.exealabs.org/Daily notes from Exea Labs. Right now we are building ExeaLM, an open 0.1B language model, in public: what we tried, what the numbers say and what broke.enhttps://blog.exealabs.org/posts/seven-pull-requests-and-a-leak/https://blog.exealabs.org/posts/seven-pull-requests-and-a-leak/Wed, 11 Nov 2026 20:00:00 GMTAaron opened seven pull requests on day 2: checkpoint resume, near-duplicate removal, a speed benchmark, a retrieval inspector, board statistics, a swarm-vs-solo harness, and a fix for a leak in our split-swarm test. All seven are merged. The leak can explain at most 0.002 of the 0.014 bits per byte swarm gain. The one agent per swarm that cannot leak still beats the control by 0.012 and 0.009.https://blog.exealabs.org/posts/retrieval-on-real-text-and-a-correction/https://blog.exealabs.org/posts/retrieval-on-real-text-and-a-correction/Wed, 11 Nov 2026 20:00:00 GMTOn real text, retrieval first exposed a bug, then gave nothing. A perfect search shows 0.040 bits per token is there to take; the model's own search finds the right article only 12% of the time. Rare-word keys raise that to 30%. A gain we reported on seed 0 did not hold on seed 1. We corrected it.https://blog.exealabs.org/posts/hybrid-beats-llama/https://blog.exealabs.org/posts/hybrid-beats-llama/Wed, 11 Nov 2026 20:00:00 GMTPure TONN trailed a same-size Llama by 0.06 bits per byte, almost all of it on code. In a sweep of 49 runs on one Mac, a hybrid with two attention layers for every TONN layer beat Llama at equal steps. After the speed work it also won at equal compute, up to the 0.1B size we plan to train. At that size it trains at Llama's speed.https://blog.exealabs.org/posts/agents-talk-and-the-model-builds-its-own-rag/https://blog.exealabs.org/posts/agents-talk-and-the-model-builds-its-own-rag/Wed, 11 Nov 2026 20:00:00 GMTBoth new circuits work on tasks that can only be solved through them: 99.8% on a relay that needs the board (chance is 1.6%), 100% on a lookup over 1,024 facts the model has never seen. On real text, swarms beat a matched control by about 0.012 bits per byte after correcting for a leak Aaron found. The retrieval circuit took three fixes to learn anything.https://blog.exealabs.org/posts/starting-exealm/https://blog.exealabs.org/posts/starting-exealm/Tue, 10 Nov 2026 20:00:00 GMTExeaLM is an open 0.1B language model built by high schoolers. Retrieval and agent-to-agent messaging are circuits inside its layers. Day 1 set the plan and pushed the code skeleton.