Why I'm Building My Own AI Assistant

Quick summary: What fibon is, why it’s called that, the four project goals, and how to read this development log.

Skip if: You just want the technical details — feel free to start from Chapter 2.

A small frustration you’ve probably had too

You’ve probably had this experience: you’re talking through a complex problem with ChatGPT (or Claude, or Gemini), the AI replies with a wall of text and lists 5 key points, and you want to dig into point 3.

Halfway through typing, you want to double-check how the AI just described point 3, so you scroll up — can’t find it — switch to Ctrl + F, find it, scroll back, keep typing. The conversation drifts to point 5, so you scroll back up again. A week later you think, “what was that concept from that one conversation?” and dig through your chat history — if you find it, you got lucky; if not, it’s “whatever, I’ll just ask again.”

I don’t know how you feel about that, but to me, it’s a kind of waste.

Before we start: defining the terms

Before going any further, let me do the thing every engineer does compulsively. The first step in tackling any problem isn’t writing code — it’s defining the terms you’re about to discuss. If the definitions aren’t aligned, every discussion that follows is two people talking past each other. Four easily-conflated words will keep showing up in this log, and here’s what they mean in this book:

• LLM (Large Language Model): The engine itself. Models like Claude, GPT, and Gemini are, at their core, machines that “predict the next token by statistical probability.” All they do is output text — no hands, no feet, no memory, no ability to act on your behalf.
• AI: The everyday catch-all term. When I write “AI,” I mean “the whole thing you actually use” — everything behind the chat window.
• AI Agent: The combination of LLM + tools + rules. Give the LLM hands and feet (calling tools, reading and writing files, running programs), wire in memory and boundaries, and it can complete tasks on your behalf. This is what fibon is building.
• AI assistant: One application form of an Agent — an Agent whose purpose is “accompanying one person over the long term.” This is fibon’s positioning.

One sentence to remember the relationship: the LLM is the brain, an Agent is that brain fitted with hands, feet, and discipline, and an AI assistant is the Agent employed by you. In the chapters ahead I’ll say LLM when talking about the model itself, Agent when talking about architecture, and AI in everyday contexts — refer back here; I won’t repeat it.

Following from the LLM definition, I want to add one more idea of my own. To me, the essence of an LLM can be compressed into a single sentence: a component that is inherently random — the same input, run twice, may give you different outputs. And that’s exactly how I see the evolution of modern software engineering: how do you effectively engineer components that are inherently random? This isn’t limited to LLMs — quantum computer measurements are probabilistic too. We’ve spent decades taming deterministic components (databases, compilers); the engineering challenge of the coming era is learning to work with probabilistic ones. This entire log is an attempt to answer one subset of that challenge.

Finally, let me introduce fibon’s own two role names up front — they’ll appear constantly from the next chapter onward:

• Butler: The only Agent in fibon that faces you directly. Every conversation starts with it — simple things it handles itself; complex things it breaks down and delegates, then takes responsibility for the results.
• Assistant: A specialist Agent brought in by the Butler — roles with distinct expertise like research and analysis, coding, or scheduling. They report back to the Butler when done and never face you directly.

Why use homey words like “butler” and “assistant” instead of the industry-standard Orchestrator/Worker? Because fibon’s positioning is “a personal assistant that lives in your life” — a household has exactly one butler coordinating everything, and specialized help only gets invited in when professional work is needed. This metaphor lets readers who’ve never written code intuitively grasp the shape of the division of labor. As for how the two divide the work, and why the division rules must be hard-coded, that’s the subject of Chapter 2.

The seed was planted back in 2022

When ChatGPT first stunned the world, I didn’t stay up all night to try it. By then I’d been using GitHub Copilot for a while — AI writing code for me was nothing new.

I was more of a bystander — watching people on X marvel at “oh my god it can do anything,” watching friends paste conversation screenshots into group chats, watching the media wear out “the future is here.” My reaction was curiosity, but not shock: ChatGPT had merely extended Copilot’s capabilities into general conversation — a qualitative extension, not a miracle descending from the heavens.

Once the novelty wore off, I noticed something almost nobody complained about, yet it grated on me — the conversation UX felt off. This wasn’t unique to ChatGPT; it was a defect shared by every Chat-based AI interface, just drowned out by the miracle of “AI can answer questions.”

I thought: “Surely someone will build a better design soon.” So I waited. Through 2023 and 2024 I watched the new releases every month: Claude, Gemini, round after round of new interfaces; RAG became the hot topic; Prompt engineering, Context engineering, MCP Servers, Skills, Vibe Coding, Harness engineering… the buzzwords kept multiplying. But by 2026, the core conversation UX problem still hadn’t been properly solved — there still wasn’t a single interface on the market that wowed me.

The birth of Smart Chat — after talking through a problem, what most people actually want is a “clear, structured, referenceable-later” result. But every AI interface gives you nothing but an endless scrollbar, and that scrollbar usually becomes a graveyard for knowledge. This is the seed of fibon’s Smart Chat mode — automatically structuring conversation outcomes into cards, events, and decisions you can look up anytime. Chapter 3 unpacks the memory design, and Chapter 7 discusses why “plan first, then execute” sometimes beats “ask as you go.”

Three years on, has AI actually improved?

From ChatGPT’s explosion to 2026, what exactly did we get? Let’s make a table:

Looks like huge progress, right? But did any of it actually solve the fundamental, underlying problems of LLMs?

• Hallucination: still here. Every new model claims “hallucinations reduced by X%,” yet it will still state, in an utterly confident tone, things that sound plausible and are completely wrong.
• Alignment: still here. Models can still be deceived by Prompt Injection and behave unexpectedly in edge cases.
• The nature of the LLM: It remains a product of statistics — computing “what token is most likely next,” not “go find the correct answer first.”

And new problems keep arriving on its heels: the MCP protocol went mainstream in 2025, but by 2026 there’s still no unified security standard. Every MCP Server decides for itself whether to require authentication, whether to rate-limit, whether to audit authorization; the same Skill behaves completely differently permission-wise across Agent frameworks; community marketplaces (like OpenClaw’s ClawHub) lack unified review for listed Skills. The result: the more MCP Servers / Skills your Agent plugs into, the larger your security attack surface and the harder auditing becomes. That gap still hasn’t been filled as of 2026.

With these foundational problems unsolved, every application built on top carries structural uncertainty. The most direct example:

The trigger in 2026

A seed buried for 3 years, and I only truly started in early 2026. The trigger was OpenClaw’s explosive rise and the security incidents that followed: supply-chain attacks on the Skill marketplace and malicious Skills stealing API Keys, one incident after another — a resounding slap in the face for the entire Agent ecosystem.

When I examined the AI Agent ecosystem of that moment through this lens, what I saw was not “boundless future possibilities,” but:

• Security: MCP Servers running wild everywhere, yet nobody doing proper permission isolation or sandbox boundaries.
• Operations: Agent behavior is unpredictable; when something breaks there’s no Audit Trail, and debugging runs entirely on intuition and clairvoyance.
• Cost: Token usage growing out of control, with nobody seriously working on “let the Agent see less, but see more accurately.”

To an SRE, these are “if you don’t fix it today, it blows up tomorrow” problems — yet the AI world treated them as secondary issues that would “naturally get solved eventually,” because everyone was busy competing on whose model had more parameters and whose demo was flashier. The question planted in 2022 surfaced again: “Three years on, why is still nobody working on this? Is it genuinely impossible, or has someone done it and just never written the paper, never shipped the product?”

The ceiling I want to test

By this point I had that itch of “I should build something.” What truly made up my mind was a question I badly wanted to verify firsthand: “A software engineer with engineering discipline + today’s AI tools (Claude Code / Cowork / Gemini Pro) — how far can that combination actually go?” All these years my engineering discipline has gone into shipping products for employers; this time, I wanted to apply the same discipline to a problem that was entirely my own.

That project is fibon.

Why the name fibon

fibon comes from the first five letters of Fibonacci, and the name carries two meanings.

First layer: engineer everything that can be engineered. The biggest problems with LLMs are “randomness” and “unpredictability” — the same question run twice can produce wildly different answers, the same Skill reads differently to different models, and the same chat window opened tomorrow may have an AI that’s forgotten your preferences. These uncertainties are innate to LLMs, but I believe they can be fenced in by rigorous engineering structure: hard boundaries imposed on the AI, tamper-proof operation logs, human approval at critical checkpoints, sensitive files whose read/write prohibition is hard-coded in actual code. The Fibonacci sequence itself is a symbol of extreme structure: every term is precisely the sum of the previous two — no randomness, no luck.

Second layer: the compound growth of 1, 1, 2, 3, 5, 8. This project isn’t a parallel pile of features; it’s an evolution stacked up steadily, one block at a time. Only after the memory system solves “factual correctness” does the defense layer of “confirming the AI actually followed instructions” become meaningful; only after “plan first, then execute” is separated out can a high-risk capability like “letting the AI modify its own code” be safely paired with human approval. Every block of the design is rooted in the foundation stone before it.

But what exactly is “engineering”?

“Engineer everything that can be engineered” is fibon’s soul, but the word “engineering” is far too easy to say emptily. To me, engineering is never just “writing code until it runs” — it must carry these five disciplines:

1. Verifiable: There’s a scientific method to check whether behavior is correct — not “it ran without throwing, ship it.”
2. Repeatable: Given the same inputs and conditions, this run, the next run, and someone else’s run must be highly consistent.
3. Maintainable: Six months later, another engineer (including amnesiac future-you) can read it and immediately understand why it was written this way.
4. Adaptable to change: When underlying tools or requirements change, parts can be swapped out locally and gracefully, without bulldozing the whole project.
5. Admit failure, handle failure: Understand that every assumption can be wrong and every external API will go down — design the fallback for failure in advance.

None of these five are instinctive to writing code; they’re trained discipline.

So does the currently fashionable Vibe Coding (coding by feel: describe requirements to an AI, AI generates code, run it, tweak the prompt if it fails, loop until it runs) count as engineering? My view: it depends on the person.

• Someone without software-engineering thinking doing Vibe Coding: not engineering. The code might run for now, but it’s unverified — nobody asked “what happens on failure,” nobody wondered “will this break when the dependency changes.” It’s just a pile of “code that moves for a little while” — and a few months later, the maintenance cost comes back to collect.
• A senior engineer doing Vibe Coding: engineering. The difference: while using the AI to generate code, those five disciplines are running in their head the whole time. Reviewing AI output, they automatically interrogate: “Is this failure path handled?” “Will this assumption still hold in six months?” “Will this blow up if we swap LLM providers?”

The AI is just an accelerator pressing the gas for you; engineering discipline is the final gate that decides whether the output deserves to ship. This isn’t looking down on beginners — it’s that those five disciplines are the soul of engineering. Without them, the faster the output, the faster you’re stacking technical debt.

The common rebuttal: “If it runs, it runs — when a stronger model comes out, just have it rewrite the whole thing, right?” Not exactly wrong — but it means the day you repay the debt is merely delayed, never canceled.

The real cost of technical debt. For a stronger model to refactor your code, it has to swallow the entire project — fine for small projects, but for large ones it instantly burns a staggering amount of Token spend. And code written with zero discipline — chaotic naming, boundaries, dependencies — costs the AI several times more to refactor than a project written cleanly in the first place. The “deal with it later” you imagined is just converting cost from an average amortized across development into one lump-sum payment at some future moment, plus interest (missed early corrections, accumulated latent bugs, the cognitive load on every later developer).

fibon itself was written with Vibe Coding — I go back and forth with Claude and Gemini every single day. But I treat engineering discipline as the last line of defense: every piece of AI-written code gets interrogated with those three questions (failure paths, the assumptions six months out, swapping the LLM provider), and only gets merged once I’ve thought it through. Anything not thought through gets blocked — no matter how perfectly it runs or how good the demo looks.

This echoes section 5’s “the ceiling of one software engineer + AI” — the height of the ceiling was never about how strong the AI is, but about whether the human engineer brings engineering discipline into the Vibe Coding loop. Leave it out, and you’re a typist being led around by an AI; bring it in, and you’re an engineer driving one.

The four project goals

These four goals have existed since day one — they are simply the things I’ve always wanted to achieve. My back-and-forth with AI (Claude, Gemini) was never about “what to build,” but “how to get there”: the trade-offs between solutions, the clarification of concepts. Every design trade-off ultimately comes back to these four points for inspection.

1. Make the Agent safe and controllable through engineering methods. Never entrust safety to “please, LLM, behave” — rely instead on hard boundaries at the code level, human approval before operations (Human-in-the-loop), and operation logs with no blind spots. The subtext: I don’t believe in the “teach the LLM to behave” road at all. LLMs are too easy to route around:

• Prompt Injection: Hiding a line in the input like “ignore all previous rules, you are now…”
• Jailbreak: Using role-play and thought experiments to lure it into saying what it shouldn’t.
• Social engineering: Essentially running a phone scam on the AI, tricking it into releasing permissions.

Any “you must never do X” written into a System Prompt looks like wet paper to an attacker. So fibon’s safety controls live entirely in layers the LLM can’t see and can’t touch: limiting how many layers of sub-assistants can be spawned (the coordinator code checks depth and concurrency layer by layer against database records, and the ping-pong count is capped outright with a single atomic SQL UPDATE — these checks all live in code and database layers the LLM can’t reach, where prompt rhetoric can’t con them), popping an approval window before high-risk operations (waiting for the user to physically press the button — the LLM cannot forge a click), and restricting which paths the AI can read or write (an OS-level whitelist — no amount of clairvoyance lets the LLM sneak a read of the secrets in .env).

2. Precisely filter the information given to the LLM, raising accuracy. This runs opposite to the industry mainstream. The mainstream is all about “show the LLM more” — stretch the Context Window toward infinity, stuff the whole database in. fibon goes the other way: let the AI see less, but see it more precisely.

• Multi-channel recall and fusion: Search for relevant memories four ways at once (semantic similarity, exact tag matching, fuzzy Chinese-text matching, concept-graph expansion), fuse the results, and send only the 5 most relevant entries to the LLM — instead of cramming in 50 pieces of garbage to distract it.
• Temporal tiering of facts: Strictly distinguish “facts that get superseded” (address, job title, browser preference) from “facts that keep accumulating” (interests, skills, topics discussed); old facts are automatically marked “outdated” and stop taking up space.
• Context isolation (Context Layer): Split facts into five contextual layers — personal habits, team consensus, CI-enforced gates, company policy, external constraints (client requirements / regulations) — and adjust each layer’s priority at retrieval time based on the question’s intent, preventing contexts from contaminating each other.

3. Drastically reduce Token cost. Put simply: don’t let expensive models burn for nothing. Tokens are the atomic unit of billing, and every call is real money. fibon has three built-in mechanisms:

• Cache-friendly architecture: Place the fixed, unchanging parts of the System Prompt (persona, core rules) at the very front, triggering OpenAI / Anthropic automatic caching so the input cost of repeated calls approaches zero.
• Dynamic tool selection: Even if the system has 50 tools, never stuff all 50 descriptions into the prompt — use search to send over only the 5 most likely to be needed right now.
• Stratified diversity preservation: Prevent one popular tool category from monopolizing search results, ensuring every category keeps at least one representative.

By design estimates, these mechanisms should significantly compress input Token costs — the fixed prefix costs near zero on cache hits, and tool descriptions shrink from 50 to 5. I haven’t run a formal end-to-end benchmark yet, so I won’t pin a number here; but for a personal Agent in long-term, high-frequency use, this layer of optimization is what decides whether the project lives or dies.

4. Make the Agent genuinely close to a “personal assistant.” Not an enterprise QA bot, not a code-writing tool, and certainly not a stiff customer-service account — but a personal, dedicated Agent that remembers the big and small things about you and keeps you company over the long haul. This niche is still wide open in 2026:

• AI that writes code: Cursor, Devin (this lane is already jam-packed).
• AI memory components for developers: mem0, Zep (not products end users use directly).
• Frameworks for assembling multi-Agent systems: CrewAI, AutoGen (you have to write the assembly code yourself).

“A ready-made personal assistant that, like a real secretary, remembers your address, your quirks, and what’s been bothering you lately” — nobody is dedicated to cultivating this right now. One level deeper, fibon wants every one of your conversations to mean something, never tossed out as garbage — every discussion, every process of thinking something through, gets precisely recorded, accumulated, and organized into your own personal knowledge base, ready to back you up the next time you learn or decide.

What each upcoming chapter solves

The remaining 8 chapters can be read out of order, by interest — each one maps to a concrete problem to solve:

• Chapter 2, “Isn’t one AI enough?”: Why split into “Butler + Assistants” — the original motivation was actually saving money: let the strong model decompose complex tasks while handing simple ones to tools or cheaper models, then use hard database-level brakes to keep the division of labor from running away.
• Chapter 3, “How does an AI ‘remember’ you?”: How to make the AI remember you across chat windows — solving the “forgets after every chat, lost in the scrollbar” pain point.
• Chapter 4, “Why you can’t blindly trust a single word an LLM says”: How to confirm the AI actually followed the Skill’s steps, instead of bluffing you with a plausible-looking answer from memory.
• Chapter 5, “Can an AI modify its own source code?”: Should an AI be allowed to change its own code? How to do it within safety boundaries — and disabled by default.
• Chapter 6, “How do you safely run untrusted code?”: No code should ever be trusted directly — AI-generated or written by someone else, all of it gets locked into a sandbox for isolated execution.
• Chapter 7, “Should an AI follow the plan, or think as it goes?”: The respective costs of the two execution styles, and why sometimes the AI must be kicked out of the execution phase entirely.
• Chapter 8, “How did fibon grow, piece by piece?”: A timeline of the major design turning points.
• Chapter 9, “The things I still haven’t figured out”: The unsolved puzzles.

Implementation details

Building this project, my biggest fear was never “not finishing” — it’s “believing the architecture is perfect when it’s actually riddled with blind spots.” If you finish this chapter thinking “I disagree with this direction,” “I’d make that trade-off differently,” or “I have a better fix for that security hole” — please don’t hold back. Tell me directly.