<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>larochelle.io</title>
    <link>https://larochelle.io/</link>
    <atom:link href="https://larochelle.io/feed.xml" rel="self" type="application/rss+xml"/>
    <description>Building a personal AI assistant and an Emacs phone. Notes as I go.</description>
    <language>en</language>
    <lastBuildDate>Mon, 13 Jul 2026 06:39:22 +0000</lastBuildDate>
    <item>
      <title>Reach for a skill before middleware</title>
      <link>https://larochelle.io/posts/2026-06-27-skills-over-middleware.html</link>
      <guid isPermaLink="true">https://larochelle.io/posts/2026-06-27-skills-over-middleware.html</guid>
      <pubDate>Sat, 27 Jun 2026 00:00:00 +0000</pubDate>
      <content:encoded><![CDATA[<p>
When I started building my agent, my bias was to cage it. Wrap the
model in a tool with a fixed shape. Add middleware that watches its
output and steps in. Write a guardrail that catches the bad case
before it reaches the user. This is how the industry made services
reliable: structure around the unreliable part. With a small local
model, the one I run at home, this reflex is stronger because the
model is weaker.
</p>

<p>
Most of what I have learned in the last few months is that the reflex
is wrong. <b>Reach for a skill before middleware</b>. Cage the model last,
only at the backstop, and only in a generic way.
</p>
<div id="outline-container-orgbcb3e3b" class="outline-2">
<h2 id="orgbcb3e3b">The cost of the cage</h2>
<div class="outline-text-2" id="text-orgbcb3e3b">
<p>
Two failures taught me the price.
</p>

<p>
The first was a loop detector. I built six patterns to stop a
small-model agent from running away, each with a real incident behind
it. In production it bounced a perfectly good research turn between
three of those patterns, stripped its tool calls, and dropped the user
into a canned "I couldn't settle on one, how would you like to
proceed?" The guardrail meant to prevent a runaway had created a worse
failure than the runaway. The fix was to <a href="https://github.com/pierrel/assist/pull/128">delete four of the six
patterns</a>, and a thousand lines with them, and let the agent run to a
<a href="https://github.com/pierrel/assist/blob/b310cf73c318dca235897419fb458387a5603dd6/assist/agent.py#L184-L188">coarse recursion limit</a> instead.
</p>

<p>
The second was an org format editing bug. Who would have thought that org
wasn't a large part of Qwen's training set? The model kept anchoring an
edit on the wrong line and splitting a file. My instinct was a
deterministic guard that rejects bad edits. I spent days trying to
"lightly" detect and surface a bad edit, and landed on doing the
agent's job for it before realizing that fights the whole point of the
project. What actually worked was a
<a href="https://github.com/pierrel/assist/pull/122">checkable rule on the tool's arguments, written into the instruction</a>,
not a wrapper bolted around it.
</p>

<p>
Both times my reflex was structure, and both times it was worse than what it
replaced.
</p>
</div>
</div>
<div id="outline-container-orga12fa91" class="outline-2">
<h2 id="orga12fa91">Reach for the skill</h2>
<div class="outline-text-2" id="text-orga12fa91">
<p>
The alternative is a skill. On the phone, "call Ana" has to turn a
name into a number, maybe disambiguate, confirm, and dial. The first
design added a tool to look up contacts and a server event to drive
the confirmation. Hand-holding the whole way. The dial is a dumb,
deterministic command: a number in, a call out, no
thinking. Everything fuzzy lives in a skill, a plain-English recipe
the agent follows: search my notes for a name near a number, pick,
confirm, call. No new tool. It runs on the small local model and it
actually works.
</p>

<p>
A skill is the opposite of a commitment. It is text. I can read it, change it,
and watch the behavior change, the same way I change anything else on the phone.
A tool is a fixed shape with code behind it and a contract that drifts from the
recipe that uses it. Given the choice, put the judgment in the text and let the
model do the thing it is actually good at.
</p>
</div>
</div>
<div id="outline-container-org8ffb334" class="outline-2">
<h2 id="org8ffb334">The backstop</h2>
<div class="outline-text-2" id="text-org8ffb334">
<p>
Leaning in has a backstop, the thin layer of hard structure you cannot skill your
way out of. A weak model under pressure falls back to its strongest prior, and
prose does not reliably outweigh it. I told an agent eight ways never to answer
from its own memory. When search failed it answered anyway and invented a version
number. It had not missed the rule, it had no other move. A "stop" handed back in
a tool result reads the same way, as one more piece of data, not a command.
</p>

<p>
So the backstop needs structure, and the craft is keeping it narrow. Guard only
the case that is never legitimate work: the same tool, the same arguments, over and
over, ended by a counter. "Many distinct searches" is not safe to guard, that is
also what thorough work looks like. For anything fuzzier reach for a coarse bound,
a timeout or a recursion limit, not a clever per-behavior heuristic.
</p>
</div>
</div>
<div id="outline-container-orgca48d9f" class="outline-2">
<h2 id="orgca48d9f">The rule</h2>
<div class="outline-text-2" id="text-orgca48d9f">
<ol class="org-ol">
<li>Skill</li>
<li>Checkable instruction</li>
<li>The narrowest structure that makes the bad state unreachable</li>
</ol>

<p>
Middleware is what you add once you have proven a recipe cannot do the
job, not what you start with.
</p>

<p>
It is also why the <b>functionality on this phone is skills and not
apps</b>. An app is someone else's code, a fixed shape I cannot open. A
skill is a paragraph of plain English describing how I want my device
to work. If the model is good enough to follow the paragraph, and most
of the time it is, then the paragraph is the better place to keep the
behavior.
</p>
</div>
</div>
]]></content:encoded>
    </item>
    <item>
      <title>Where we are now (June 2026)</title>
      <link>https://larochelle.io/posts/2026-06-21-where-we-are-now-june.html</link>
      <guid isPermaLink="true">https://larochelle.io/posts/2026-06-21-where-we-are-now-june.html</guid>
      <pubDate>Sun, 21 Jun 2026 00:00:00 +0000</pubDate>
      <content:encoded><![CDATA[<p>
<a href="https://larochelle.io/posts/2026-05-30-where-we-are-now.html">Last month's account</a> ended on a cliff: "[This project] is not a phone
in the way you mean. No cellular modem, no audio, no calls, no texts."
(Claude loves the drama)
</p>

<p>
It rings now.
</p>
<div id="outline-container-org6f9e6e0" class="outline-2">
<h2 id="org6f9e6e0">It rings</h2>
<div class="outline-text-2" id="text-org6f9e6e0">
<p>
I can now call out from it, and it can take a call coming in. When a
call arrives the screen shows who is calling and the keyboard turns into two
buttons, Accept and Decline. Answer, and those become Hang up. It is a cellular
call, over a real modem, on my own number.
</p>

<p>
However, the path that carries my voice out needs one more piece of
hardware I have not wired in. So today it is a phone you listen
on. That is a strange half of a phone to have first. It is also the
half that proves the hard parts work.
</p>

<p>
The keyboard grew up too. In May it had no numbers or symbols, which
meant I could not type a wifi password or a URL on the device. It has
them now. It's a small thing, but it was the line between a toy and
something I can set up without reaching for a laptop.
</p>
</div>
</div>
<div id="outline-container-org26c1935" class="outline-2">
<h2 id="org26c1935">What it cost</h2>
<div class="outline-text-2" id="text-org26c1935">
<p>
The reason this took a month and not an afternoon was not software. It was
power.
</p>

<p>
The modem kept dropping calls. Every time, the system logged the modem
falling off USB and coming back, mid-call. That looks exactly like a driver
bug, and I treated it like one for too long. It was a brownout. When the modem
keys up its radio to transmit it pulls a hard spike of current, and the five
volts was reaching it through the Pi, which cannot supply that spike. The rail
sagged and the modem reset. The fix was to feed the modem power straight from a
stiff source, in parallel, never through the Pi. With its own supply a call now
holds.
</p>

<p>
There was a dumber lesson in the same week. The call audio went silent and I
spent an hour sure it was a software bug, checking the sound config every way I
knew. The headphones were dead. A different pair worked on the first try.
Nothing in software can tell you a speaker is broken. A dead wire and a bug
that eats the audio look identical from where I was sitting.
</p>

<p>
The nicest part to build was the call screen itself. The bottom of the phone
is a fixed keyboard, and the obvious move was to put Accept and Decline up in
the call window. Instead I made the keyboard reactive. It is not "the
keyboard," it is the control surface, and what it shows should follow what is
on screen. A call replaces the keys with call controls. A normal screen shows
the keys. The agent and I built that on a branch, reviewed it, and merged it
like any other change.
</p>
</div>
</div>
<div id="outline-container-orgd313796" class="outline-2">
<h2 id="orgd313796">The agentic part</h2>
<div class="outline-text-2" id="text-orgd313796">
<p>
And it's all wired up through a phone call skill, and this is shaping
the way I'm thinking about this phone. The smartphone era brought us
apps: some functionality, UX, some data, and connection to a
service. But this was all contained, owned, and driven by a faceless
company with a team of engineers. And thousands of lines of code.
</p>

<p>
Now with AI agents, functionality can be contained within a skill:
plain english, describing how you, the user want to use your
device. Here's a snippet of how calls are handled on my phone:
</p>

<div class="org-src-container">
<pre class="src src-markdown"># Place a call

Your job is the *interpretive* part — turn what the user said into one concrete
phone number — then hand it to the deterministic `emacos-call` command. You
work through `eval_elisp`: **evaluate elisp to SEARCH for the number, then
invoke the `emacos-call` command to DIAL.** Stay on the call task.
</pre>
</div>

<p>
And yes, it works! I type "Call Ana" and it searches through my files,
finds her number, and calls.
</p>
</div>
</div>
<div id="outline-container-org576d4fa" class="outline-2">
<h2 id="org576d4fa">What's still rough</h2>
<div class="outline-text-2" id="text-org576d4fa">
<p>
It is held together with a wall charger and alligator clips. The proper power
board and a smaller antenna are in the mail. I cannot put my voice on a call
until a headset adapter arrives. Speech to text is still planned and not built,
so the keyboard is the only way in, and it is still slow to type on.
</p>

<p>
So I'm still very far from where I want to be.
</p>
</div>
</div>
<div id="outline-container-org9c44d76" class="outline-2">
<h2 id="org9c44d76">What's next</h2>
<div class="outline-text-2" id="text-org9c44d76">
<p>
Get my voice onto the call, which is the headset adapter plus the
software to run both directions of audio at once. Move the power off
the bench rig and onto the real board. Then back to input, where
speech would change how this feels more than anything else I could do.
</p>

<p>
But there's progress: In May it was a malleable Emacs handheld that
could not make a call. In June it makes calls and I can hear
them. That is still less than a phone, but I'll still call it a phone,
my phone.
</p>

<p>
For the why, read the <a href="https://larochelle.io/manifesto.html">manifesto</a>.
</p>
</div>
</div>
]]></content:encoded>
    </item>
    <item>
      <title>Where we are now (May 2026)</title>
      <link>https://larochelle.io/posts/2026-05-30-where-we-are-now.html</link>
      <guid isPermaLink="true">https://larochelle.io/posts/2026-05-30-where-we-are-now.html</guid>
      <pubDate>Sat, 30 May 2026 00:00:00 +0000</pubDate>
      <content:encoded><![CDATA[<p>
This is the first of what I plan to write every month or so: a plain account of what the two projects can actually do, today, together.
</p>

<p>
My bet is in the <a href="https://larochelle.io/manifesto.html">manifesto</a>. Software that is personal, private, and sovereign, with an agent that works for me, all running on hardware I own. I'm working on two projects to test it. <code>assist</code> is the agent. <code>emacsos</code> is the phone the agent reshapes.
</p>
<div id="outline-container-org806d1e3" class="outline-2">
<h2 id="org806d1e3">The two halves</h2>
<div class="outline-text-2" id="text-org806d1e3">
<p>
<code>assist</code> is a local-first agent. It runs against a small open-weight model (currently Qwen3.6-27B) on a box at home, so nothing I say to it leaves the house. It has the parts you would expect, tools and memory and skills and sub-agents for research and exploration, but tuned for a small model, where reliability is the hard part. Each conversation works inside its own git branch of a repo I choose, so the agent's edits stay isolated until I merge them. It has a web UI, a CLI, and an Emacs integration, all sharing one agent core.
</p>

<p>
<code>emacsos</code> is the phone. It is Emacs on a small touchscreen (sounds fun, right?), a 320x240 frame with a nine-key keyboard, on a Raspberry Pi. I chose this as the starting point because the phone is the most personal device I own, so it should be the most malleable. Everything on it is elisp and changeable at runtime. The catch is that editing elisp on a 320x240 screen with a nine-key keyboard is miserable. So I don't. The agent does!
</p>
</div>
</div>
<div id="outline-container-orgd2df0e4" class="outline-2">
<h2 id="orgd2df0e4">What's possible today</h2>
<div class="outline-text-2" id="text-orgd2df0e4">
<p>
The two are wired together. The phone has a chat page. I type a request, it streams to a small server sitting next to <code>assist</code>, and the agent answers token by token in the buffer. I can abort mid-answer.
</p>

<p>
The part I care about most works: I can change the phone by talking to it. "Set the cursor blue." The agent reads the current config, writes the new elisp, and applies it. Every change is a git commit, with the agent's own summary as the commit message. So the history of how my phone became what it is, is just <code>git log</code>. When a change is wrong, one tap reverts it. That rollback is not a feature I built. It is <code>git revert</code>, which I already understood.
</p>


<figure id="orga2fd378">
<img src="https://larochelle.io/assets/images/emacsos-phone.jpg" alt="EmacsOS on a Raspberry Pi: a small touchscreen showing a cursor-change chat with the T9 keyboard below, next to a USB battery pack.">

<figcaption><span class="figure-number">Figure 1: </span>The phone, mid-chat about a cursor change.</figcaption>
</figure>

<p>
All of this runs at home, on my hardware, against my model. The phone depends on no remote service to keep working. The config, the history, and the decisions stay on machines I own.
</p>
</div>
</div>
<div id="outline-container-org6074c6d" class="outline-2">
<h2 id="org6074c6d">What's still rough</h2>
<div class="outline-text-2" id="text-org6074c6d">
<p>
Let's be honest: this is all very painful, input being the weakest link. The keyboard has no number or symbol keys yet, which means I cannot type a wifi password or a URL on the device. Speech-to-text is planned but not built. The phone and the agent assume they are on the same network. Taking the phone out into the world, over a tunnel home, is designed but not done. And it is not a phone in the way you mean. No cellular modem, no audio, no calls, no texts. Those need hardware I have not connected yet.
</p>

<p>
So today it is a malleable Emacs handheld that I talk to my own agent through, and that the agent reshapes for me on command. That is less than a phone, and in the one way that matters to me, more.
</p>
</div>
</div>
<div id="outline-container-org2296b48" class="outline-2">
<h2 id="org2296b48">What's next</h2>
<div class="outline-text-2" id="text-org2296b48">
<p>
The next step just unblocked. Until this week the agent could not reliably read the current config after a reset, which made targeted edits fragile. That is fixed now. So the next experiment is conversational, targeted revert: "undo the cursor change but keep the font change," driven from chat, mapped to the right commit, reverted without disturbing the rest. After that the road runs through input, numbers first and then speech, and then the actual phone build: a modem, audio, and the slow work of making this thing ring.
</p>

<p>
None of it is finished. That is why I want to write it down every month. For the why, read my <a href="https://larochelle.io/manifesto.html">manifesto</a>.
</p>
</div>
</div>
]]></content:encoded>
    </item>
  </channel>
</rss>
