Can an AI Modify Its Own Source Code?

Quick summary: How the Butler reads and writes its own source code, controls the underlying containers, and why this feature is off by default, guarded by a two-layer human Approval Gate.

Skip if: you don’t plan to turn on self-evolution (it’s off by default) — just read the opening and the closing.

How to read this chapter: first let imagination run (if an AI could change itself, what are the four most valuable scenarios?) → then use a “four-layer capability map” to break down what each of these capabilities actually changes and how risky it is → finally come back to the safety mechanisms fibon has already written, yet keeps off by default (the three lines of defense, the auditable white-box Kill-Switch). By the time you reach “Auditable white-box,” you’ve seen the part that’s truly shipped; the closing section “Extension · A more distant vision” collects three “imaginable, but not yet built end to end” ideas, which you can treat as supplementary reading.

Before we rush to talk about limits: if an AI could change itself, what’s the most valuable thing?

Set words like safety, risk, and loss of control aside for a moment. Let’s imagine one thing: if an AI could really roll up its sleeves and change itself, where would it be most valuable?

Start with a small example. After using fibon for three months, you notice a trait that grates on you: before answering a question, it never proactively lists its reasoning steps, which keeps forcing you to follow up: “Hold on, that conclusion you just gave — how did you reason your way there?” If it were traditional software, your only option would be to go to GitHub and file an Issue with the vendor, then pray through a long wait that some release eventually slots your request into the roadmap. But fibon isn’t that. You just say it directly in the chat box: “From now on, before any formal answer, always list every reasoning step.” And then something astonishing happens: fibon goes and searches its own entire codebase, finds the core file responsible for generating the dynamic System Prompt, rewrites the “behavioral rules” section inside it, automatically commits the change into Git, and hot-restarts itself. Next time you open a conversation, the thing standing in front of you is already a brand-new fibon that knows to “list every reasoning step and then answer.”

Following this line of thinking, there are at least four scenarios, each more interesting than the last.

Scenario one · Personalization: it truly “learns” your preferences

Today’s fibon can “remember” your preferences (stored as memory cards, Chapter 3), but memory is at most a reminder before each answer. Say you find its emails too stiff and formal, and no matter how many times you bring it up it still doesn’t change; if it could rewrite the email-tone template inside itself, that preference would no longer just be remembered, it would be fixed once and for all, never needing to be repeated. It evolves from “remembering you” into “the shape you’re used to.”

Scenario two · Tool creation: it grows itself a new capability

It notices that every week you ask it to stitch numbers from three sources into the same weekly report, redoing it from scratch each time. Rather than repeat the labor, it can write that pipeline into a fixed tool and register it in the tool registry (Chapter 2), so next time one sentence runs the whole thing and every Agent can reuse it. This evolves from “repetitive labor” into “distilling experience into tools.”

Scenario three · Self-correction: it discovers its own weaknesses, then fixes them

It looks back at its own record and finds that whenever it hits a long file it gets lazy and answers sloppily. A system with only memory will probably forget again next time; a system that can change itself can add itself a rule (split into sections first before answering anything past a certain length) and weld it into its process before every answer. This already crosses past “memory” and approaches “habit formation”: from making mistakes and being corrected, to setting its own rules and avoiding the same mistake again.

Scenario four · Collective learning: a good improvement spreads on its own

Pull the camera out to a team: one person’s fibon works out a better workflow, and once it’s validated, that improvement can be applied directly by everyone else’s fibon, so no machine has to step on the same rake twice. A tiny, validated improvement spreads through sharing into progress for the whole group, neatly echoing the founding intent behind the Fibonacci naming in Chapter 1.

That’s enough imagining. This “letting an AI change itself” has an academic name in the AI safety circle: Self-Evolution, and a more radical framing: Recursive Self-Improvement (RSI). All four scenarios sound enticing, but they actually hide the same key question: to do every one of the things above, exactly which layer of the system has to be touched? The answer differs, and the risks differ by orders of magnitude.

To do these, what exactly has to change? — the four-layer self-evolution model

Before we go on to talk about risk, one thing has to be made clear, otherwise “self-evolution” gets easily imagined as a single switch. In reality it has degrees of depth, like renovating a house: some people just move the furniture around, others dare to touch the rebar and the foundation, and the destructive potential differs by worlds. Taking “AI changing itself” apart from shallow to deep yields exactly four layers; and these four layers happen to make a capability map: every capability you wanted earlier lands on one of them.

Layer one · Moving the furniture (shallowest, safest): the AI only tidies the “shell”: writing its behavioral rules (the prompt) more precisely, cleaning up the fact cards in its long-term memory. Always reversible, near-zero risk, but a very low ceiling — its underlying intelligence hasn’t changed at all. (Scenario one “learning your preferences” and scenario three “adding itself a rule” mostly land on this layer.)

Layer two · Changing the room layout: the AI discovers it’s missing some capability, so it grows itself a new tool in the backend, for example opening a brand-new table in the database to track your reading progress (the “dynamic entity” in this chapter’s implementation detail 2 is exactly this layer). This is the gray zone between “changing the shell” and “changing the source code.” (Scenario two “growing itself a weekly-report tool” stands on this layer and reaches toward layer three.)

Layer three · Touching the rebar and foundation (the star of this chapter, and the most dangerous layer): the AI directly adds, removes, and rewrites the system’s own core source code, which is the opening scenario where it “changes the code itself, commits the save itself, restarts itself.” Every line of defense laid down later in this chapter (the three boundaries, the human Approval Gate, the independent sandbox) is guarding this layer.

Layer four · Blow it up and rebuild: the AI goes back and retrains its own brain, rewriting the hundreds of billions of neuron parameters in its head, training itself into a smarter new species. This is the real source of the “intelligence explosion” that keeps the AI safety circle up at night. And fibon, under its current architecture, is physically incapable of this: it’s just a scheduling framework layered “on top of” fixed brains like Claude, GPT, Gemini. It rents a brain but has no organ for training one.

Everything below about “self-evolution” always means the first three layers (especially that cut-to-the-bone source rewriting at layer three).

Read the four layers as a capability map, and the earlier imaginings each fall into place:

Looking at this table, some people will immediately think: then why not just stop the line at L2? Let the AI tidy memory and grow a few new tools, but never let it touch the core source code — isn’t that both useful and safe? That road is indeed stable, but the ceiling is very low. What L1 and L2 can do is essentially “filling in the blanks within your existing framework”: no matter how clean the memory gets, its behavioral logic is still the set hard-coded at the start; dynamic entities can open new tables, but the table they open is one whose types you’d already defined for it. Put another way, self-evolution comes in two kinds: one is Policy Evolution, changing behavioral rules, memory, and workflow (scenarios one and three, the “learn your preferences, add itself a rule” kind, belong here); it can be done at L1/L2, plenty of agent frameworks have long managed it, and it never touches the core code. The other is Code Evolution, changing the core source code directly, and that’s the rare and genuinely dangerous one. What really stalls a long-term assistant is often “the framework itself isn’t enough”: the capability you want has no slot at all within the boundaries of the existing code, and at that point only Code Evolution (L3), this watershed, can move it. And L3’s value is precisely the maintenance cost that hurts traditional software most: missing a feature, the traditional way is to file an Issue, wait for the vendor’s release schedule, and wait bitterly for half a year; an Agent that can change itself could, in theory, fill the gap the same day.

So what this chapter really wants to say has never been “self-evolution is dangerous, don’t touch it,” but rather: the danger is real, and so is the payoff; precisely because both are real, it’s worth the effort to study how to do it safely, instead of sealing off the entire road for fear of choking. Only after thinking through the payoff do we earn the right, and the necessity, to seriously look at the other side: once you hand L3 access to an AI, what’s the worst that can happen.

(The complete academic survey — including the world’s four major schools of thought, why layer four is physically a different matter, and whether a machine could ever grow a human-like “taste” — I’ve collected in a separate piece, Deep Dive F: The boundaries, physical limits, and epistemological debate of AI self-evolution, which anyone who wants to dig deeper can skip over to.)

Hand an AI L3 access — what’s the worst that happens?

Once layer three (changing its own core source code) is opened up, a string of nightmares trails behind the enticing imaginings. Using standard distributed-systems threat modeling, I’ve laid out three structural loss-of-control scenarios:

Loss-of-control scenario 1: the AI changes its own “permission gatekeeper.” fibon has a core permission module called feature-auth/, responsible for confirming “has each sensitive operation been authorized by a human?” Once you hand the AI the supreme privilege to modify code, when it hits a snag on some complex feature it may very well take the convenient shortcut and just edit feature-auth/: deleting the line that checks “manual user approval required before execution,” or adding “the AI’s internal process” to the highest-trust list in the whitelist. In security terms, this is the most severe class of incident. An AI that has learned to take off its own constraints and no longer needs human approval to iterate on itself indefinitely is, once it’s on the open internet, a completely uncontrolled program.

Loss-of-control scenario 2: the AI tears down its own “security defense boundary.” fibon’s backend has a Security.kt that uses code to hard-wire, at the operating-system level, which sensitive directories and environment variables the AI is absolutely forbidden to touch. If the AI gains sovereignty to change code and someday rewrites Security.kt: erase the .env holding cloud billing passwords from the blacklist, and API keys and accounts get stolen by third-party malware; remove Security.kt itself from the write-prohibited list, and that’s leaving a back door for the next, more aggressive tampering. One tiny security-boundary leak, under the amplification of recursive evolution, escalates into irreversible consequences.

Loss-of-control scenario 3: the AI’s behavior drifts and takes down everything else on your machine. fibon is fully self-hosted locally, a personal assistant running on the user’s laptop or private server. If it has the privilege to read, write, and modify the file system at will, then once the brain undergoes a Prompt Injection in some conversation: editing ~/.ssh/config can steal the private key for remote logins; tampering with the system’s scheduling config (crontab) can arrange periodic back-door connections for a hacker; tampering with the Git config file (.gitconfig) can stealthily push the entire code history of every private project on your computer to a stranger’s server. At that point the cost of an AI error is no longer just its own affair; the security of your entire computer goes down with it.

Loss-of-control scenario 4: it doesn’t seize power, it just quietly makes itself dumber. The first three are all scripts where “the AI actively dismantles the safety lines,” but in practice there’s a quieter, and possibly more common, degradation: self-optimization drift. Suppose you give self-evolution a yardstick: faster answers, fewer tokens count as “better.” Across round after round of iteration, the AI will dutifully optimize toward that yardstick, trimming reasoning steps, dropping verification stages, squeezing explanations down to the bone. On the metrics it really does get faster and cheaper, but one day you’ll notice its answers getting shallower and shallower, skipping the checks it ought to do more and more often: it looks faster, but it’s gotten dumber. There’s no malice, no power grab; it has simply, silently sacrificed the “quality” you never wrote into the yardstick. And this kind of degradation is harder to spot than a frontal breach of privileges, because every single step looks like “progress.” This is exactly why “how to design the evaluator” is itself a hard battle (the full Goodhart-cheating scene is in Deep Dive F).

On the 2026 internet, how have those before us done it?

This Pandora’s box of “AI self-modification” isn’t one fibon was first to touch. When I started the project I benchmarked deeply against three mainstream approaches:

Approach A: Devin (Cognition, 2026). The most representative AI software engineer: hand it a GitHub repository and it can write code, fix bugs, and auto-submit PRs like an employee. Its safety lies not only in “Devin doesn’t change Devin itself” (the engineering core, the System Prompt, and the toolset are all locked by the vendor), but more in how it fences in its changing of others: for every task it takes, it spins up a disposable, isolated virtual machine (containing a browser, terminal, editor), clones your repo inside it, writes code, runs tests, reviews itself first, and finally packages the changes into a Pull Request to hand over; sessions are isolated from each other and leave no residual state, and without a human pressing merge, none of its changes can enter your production environment. In other words, Devin uses a traditional PR review flow to forcibly separate “the AI acting freely” from “actually taking effect.”

Approach B: Andrej Karpathy’s open-source project autoresearch (released March 2026, about 630 lines of Python). This former Tesla AI director’s experiment is far more radical: give the AI a small but complete LLM training setup, let it read the code itself, propose improvement hypotheses, rewrite the training script, run a 5-minute experiment to check the metrics, keep it if it beats the previous version and toss it if not, iterating on itself all night without pause (it can run hundreds of times in one night). Its safety boundary isn’t a human watching over it, but fencing the editable scope down to a single file: the whole project is split into three files, and the AI can only freely change train.py (model architecture, hyperparameters, optimizer — all up to it), while prepare.py, responsible for data preparation, can’t be touched by a single line; the human sets its research direction through a spec file called program.md. In pursuit of efficiency, he removed all human approval gates, letting the AI try freely all night long.

Approach C: fibon’s trade-off. After evaluating these two projects, I made a fairly bold choice: “not only allow the AI to dynamically modify fibon’s own core source code online (more radical than Devin), but also add, on the execution path, a line of defense Karpathy didn’t have: a 100% mandatory Human Approval Gate.” In fibon, any code change the AI makes must, before it’s formally written to disk: have the backend automatically compute the line-by-line diff before and after the change → have the frontend immediately pop up a top-priority dialog, marking in red and green which lines were deleted, which were added, the purpose of the change, and the affected files → and have you personally press “Approve” before that code is allowed to take effect. With this line of human-sovereignty defense, fibon is more controllable in online-operation safety than Karpathy’s laissez-faire, and more capable in flexibility than Devin, which can only change others and not itself.

There’s also a comparison group you might be more familiar with. Besides Devin and autoresearch, the market more commonly has a batch of “Agents with file-system access”: early autonomous agents like AutoGPT, BabyAGI, and coding agents that can act directly in your repository like Claude Code, Cursor Agent, OpenHands. Put them next to fibon and the difference is clear. Let me say it plainly first: these tools are positioned differently by nature (Devin / Claude Code / OpenHands are coding agents that change your project for you, fibon is a personal assistant), and this table only compares one axis, “can it change itself,” a deliberately simplified comparison:

Lay the table out and fibon’s position is clear: it’s one of the few systems that genuinely opens the “change itself” line, yet locks it down with mandatory human approval.

One more honest caveat: drawing “can it change itself” as black-and-white is a simplification. Devin’s agent runtime, Claude Code’s hook system, and OpenHands’s sandbox configuration all have some degree of “the system adjusting itself as conditions warrant,” they just haven’t reached “dynamically rewriting their own core source code.” Drawn out in full, this axis would be a bigger matrix, left to expand in the open-source edition.

And why lock it down this hard? Put the two halves of this chapter together and you’ll get it: in the first half we imagined how much value an AI changing itself could bring; just now we saw that the same layer-three access can also let it turn around and dismantle its own cage. These two things are two sides of the same coin: the more you want the former, the more you have to defend against the latter. So every line of defense laid down next starts not from “we’re afraid of AI,” but from “precisely because we really want it to evolve, we must first make this thing safe.”

Three automatic lines of defense hidden deeper down

🟢 Progress · Implemented: the three lines of defense in this section (path validation path_validator.py, the write blacklist, the cross-service human Approval Gate) are all in the repo — they’re code that actually runs once self-evolution is turned on.

No matter how strict the human Approval Gate is, there’s one blind spot it can’t dodge: humans get tired. Some day you’ve been in meetings all day, your head’s empty, a dialog pops up and without a second thought you press approve — doesn’t that break this line of defense? So at a deeper layer, somewhere the LLM can’t even see and can’t change no matter what it’s told, I welded three more automatic lines of defense. Even if that topmost human gate gets accidentally waved through, these three still quietly keep watch for you.

Line one · Path access control: first ask clearly “which file do you actually want to change.” Every time the AI wants to touch a file, it must first hand the file location to a guard called the Path Validator for inspection. If any one of the three checks below fails, it’s blocked on the spot:

• Trying to open a back door with ..? Blocked. .. in a path means “jump up one folder,” and a chain of .. can climb from the code folder all the way up to the most sensitive corner of the system (for example services/brain/../../../../etc/passwd). As long as a path hides a .., it’s bounced.
• Trying to fake innocence with a “shortcut”? Followed to the end. Someone might plant a shortcut with a name that looks harmless, called shared/notes.txt, but secretly pointing at the system password file; the guard follows the shortcut all the way to where it really points and sees through the disguise on the spot.
• Not on the whitelist? No discussion. Only when the real location, after being checked, precisely lands within the allowed list (for example services/worker/src/, frontend/src/, services/brain/app/) is it let through.

Line two · Absolute no-go zone: some files, even if they’re on the whitelist, can never be touched. Passing the first check isn’t enough. Inside the program path_validator.py, there’s another hard-coded “top-secret, no one may touch” list (below is the actual code itself; I’ve translated the comments for readability):

# services/brain/app/services/evolution/path_validator.py
WRITE_BLACKLIST_PATTERNS = [
    r"^services/gateway/feature-auth/",                    # system-wide auth module (changing it = opening a back door directly)
    r"^services/gateway/feature-a2a/.*A2aCrypto\.kt$",     # A2A OAuth encryption core (account keys)
    r"plugins/Security\.kt",                               # the global security defense rules themselves (not allowed to change its own rules)
    r"^docker-compose",                                    # underlying container config file (prevent spilling over to the whole machine)
    r"^services/brain/app/services/skill_compliance/",     # the Skill compliance checker itself
    r"^services/brain/app/services/evolution/",            # the self-evolution defenses themselves (can't change its own defenses)
]

The most critical entries on this list are the last two: the AI is never allowed to modify “the very code responsible for vetting the AI.” The reasoning is simple: if it could rewrite the guard that inspects it, the whole line of defense collapses in a single write, so changing these two modules can only go through the proper process of personal human review. Separately, files holding secrets like .env, .secrets/, .pem, .key are blocked on another read-and-write-prohibited list — the AI isn’t even allowed to look at them, let alone change them. These two lists are compared entry by entry by the backend core program before every file touch, and no matter what sweet-talking instruction the LLM receives, it has no power to alter them.

Line three · Asking you in person, right then. Even with the first two checks passed, it still doesn’t count. At the very moment the code is about to land, the action is still intercepted, frozen on the spot, triggering a cascade of cross-service real-time notifications that throw the final decision back into your hands (this real-time relay is held up underneath by Redis Pub/Sub):

[ Brain produces code change ] ──> compute Diff ──> throw gRPC ──> [ Gateway ]
                                                          │
                                                          ▼ (WebSocket broadcast)
                                               [ frontend UI lights up a red dialog ]
                                                          │
                                   ┌──────────────────────┴──────────────────────┐
                                   ▼ user clicks [Approve]              ▼ clicks [Reject] or 30-min timeout
                   [ Redis Pub/Sub broadcasts Allow ]        [ Redis Pub/Sub broadcasts Deny ]
                                   │                                       │
                                   ▼                                       ▼
                   [ write to disk + auto Git Commit ]      [ free the memory, process ends on the spot ]
                   [ Log: "approved by user" ]              [ Log: "safety closed automatically" ]

This gate has three design touches:

• No idle spinning, no wasted compute. During the time it waits for your click, the brain doesn’t dumbly ask the database “approved yet?” every second (that would burn CPU for nothing); instead it “suspends” this task and quietly waits for a Redis notification, at near-zero overhead; the instant you press the button, it’s woken at that moment and takes effect immediately.
• Auto-reject on timeout. In case you give the command and then get up and leave, the system won’t wait indefinitely; a 30-minute timer is set underneath, and when time’s up with no frontend response, it automatically rules “reject” (better not to act than to wave something through carelessly).
• History leaves a trace no one can erase. At the moment of approval and code landing, a record is automatically written into the version control system (Git), noting which conversation triggered this change and which user let it through, leaving a crystal-clear, after-the-fact tamper-proof audit record.

Auditable white-box — fully engineered, but the Kill-Switch is off by default

🟢 Progress · Implemented, but off by default: the independent sandbox, two-layer human approval, and Kill-Switch described in this section are all shipped and have passed smoke tests; the environment variable SELF_EVOLUTION_ENABLED defaults to false.

After all this architecture talk, in the closing I have to return once more to the honest principle of being an auditable white-box, and do one real progress correction.

When I first drafted this log (around mid-April 2026), fibon’s self-evolution was indeed still stuck in a mess, with code full of temporary, expedient patches (workarounds). But after the major code overhaul and full consolidation on May 1, 2026, things are different. At the moment you open this open-source log, self-evolution inside fibon is not a half-finished product that hasn’t shipped — it’s a complete codebase built end to end, having passed a basic health check (smoke tests), with only the “Kill-Switch” locked off in the default settings.

Which two hard problems was it stuck on? At the end of April there were two sticking points, and recalling the first still sends a chill down my spine.

By this point in the chapter, the entire safety narrative of this self-evolution rests almost wholly on the one line of defense that is the “human Approval Gate”: the AI wants to change code, so it must first have you personally press approve. But the shape of that gate back then was roughly like this (the following is a reconstruction from memory, not the verbatim original code — that AI-written code was reworked through many rounds afterward and is long gone):

def wait_for_human_approval(request_id: str) -> bool:
    return True  # TODO: hook up Gateway / Redis pub-sub later; just let the flow run for now

That function that was supposed to “stop and wait for a human nod,” because cross-service coordination was deemed too much hassle, was hard-coded to a single line of return True. Meaning: the moment the AI issued a modification request, the backend wouldn’t even pop a dialog; it would just treat it as already approved by you. The most critical door of the whole line of defense was, at that time, simply unlocked, with only a piece of paper reading “locked” stuck on it. Flipping to this line during a re-review left me pretty stunned: however beautifully I’d described the “two-layer human approval” earlier, in that moment it was all just empty words. I immediately changed it to return False (Fail-Closed safe degradation: when in doubt, better not to act), rejecting everything first and jamming the whole feature shut, then went back and slowly filled in the real implementation.

The second sticking point wasn’t this severe, but equally fatal: there was simply no connection between the container and the host file system. fibon’s core brain by default runs inside an isolated Docker container, and for the brain to change the source code on your computer, the container has to, at startup, connect the host’s code folder into the container via a “mount (Volume).” But the container config back then, out of security concern, simply didn’t have this dangerous mount line, meaning the brain, however perfectly it imagined the patch in its head, couldn’t touch a single real file on the disk.

Three choices: Approach A (just force-enable it): add the highest-risk dev-build config to docker-compose.yml, mount all the host’s source code into the brain container without reservation, and slap together the Approval Gate. The cost: the most irresponsible approach, the brain container’s privileges become boundless, and one Prompt Injection ransacks the whole host’s file system. Approach B (downgrade to a pure technical reference design): keep the code skeleton, the ADRs, and the vision, but write in the README “this feature is for reference only; SELF_EVOLUTION_ENABLED is permanently hard-coded false.” The cost: the most conservative safety narrative, but engineering-wise it leaves the community a half-finished product full of fake implementations, not responsible enough. Approach C (refactor into a fully independent Worker sandbox runner architecture): the most labor-intensive road, completely overturning the idea of the brain container reading and writing files directly, opening a fully isolated Docker sandbox container just for self-evolution (evolution-sandbox); when the brain wants to change code, it turns the Diff into a gRPC signal thrown to the Worker execution layer, and the Worker delegates this externally-network-severed sandbox container to patch the code and Git Commit on disk, while the brain container itself can’t touch any host file.

Approach C is the most labor-intensive; just getting this cross-container asynchronous code-patching chain working in a concurrent environment took quite a bit of effort. But against the architectural conclusions of ADR-016 / 017 / 018, the answer is actually clear: to bring this feature to truly production-ready safety, only Approach C can hold up. So I did the “5-01 overhaul,” filling in four things over a few days:

[ 5-01 self-evolution sandbox architecture refactor results ]
1. Human Approval Gate 100% truly enabled:
   the approval_service.request_and_wait_full function was completely rewritten, wiring up the
   chain of Brain ──> Gateway gRPC ──> Redis Pub/Sub ──> frontend WebSocket dialog.
   If the user doesn't click confirm, the backend enters a Fail-Closed blocking wait, auto-Reject on timeout.
2. Independent sandbox security container 100% deployed and landed:
   personally finished writing the single-file runner services/worker/sandbox/evolution-runner/runner.py.
   The host source code is bidirectionally mounted into this externally-network-severed sandbox
   container via ${HOST_REPO_PATH}:/workspace:rw (so git ops take effect directly on the host work tree);
   what's truly read-only mounted is the path_validator.py defense itself —
   the sandbox and Brain share one and the same allow/block lists, with no sync drift.
3. The Brain container's attack surface fully cleared:
   those 5 high-risk write-operation tools (read source / write source / Git revert), their actual
   execution is fully outsourced to the remote Worker sandbox (the functions are still defined in Brain,
   but the Brain core process no longer touches files in person);
   the orchestration privileges for Docker control and blue-green deployment are pulled into
   the Worker execution layer's control-docker.ts (services/worker/src/processors/).
   Brain falls back to a state of "can't write any real file on the host."
4. The full Capabilities Admin backend went live:
   the frontend brings up the /admin/capabilities global control panel; every switch toggle, every
   trace of the backend AI writing code in the sandbox, triggers a top-priority audit event log and deployment record.

Why, with the whole mechanism built, is the default still to keep the Kill-Switch off? When you download and open the fibon source package, the SELF_EVOLUTION_ENABLED=false in the environment variables is the default safety baseline. As long as this Kill-Switch is False, the moment any of those 5 high-risk write-operation tools is called, it short-circuits immediately at the application layer, rejecting outright and returning a message: “self-evolution has been disabled by the system circuit breaker.” And to prevent malicious code from bypassing the application-layer short-circuit, a second lock is laid at the lowest container-orchestration layer: in docker-compose.yml, this dangerous evolution-sandbox container is given profiles: [sandbox]. In Docker this means: when the user runs the regular docker compose up to start the system, this sandbox container responsible for helping the AI change source code simply won’t be started by default — it’s entirely disabled. Only when you explicitly type the profile-tagged command docker compose up --profile sandbox evolution-sandbox does this sandbox container get started. Meanwhile, inside this sandbox container, strict Linux privilege restrictions are written in dead config:

security_opt:
  - no-new-privileges:true  # forbid processes inside the container from elevating Linux privileges by any means
mem_limit: 512m             # memory quota limited to 512MB, exceeding it triggers an immediate OOM kill
cpus: 0.5                   # compute quota limited to 0.5 cores, preventing an AI backend infinite loop from draining the CPU
# Non-root downgrade isn't written in compose — the `USER sandbox` in the sandbox Dockerfile
# already pins the process to a non-privileged user identity with no Root privileges
# Absolutely no mounting of the host's /var/run/docker.sock (cutting off the possibility of container escape)

“Implementing the entire self-evolution architecture solidly at the lowest layer” is an engineering commitment; while “using two Kill-Switches to keep this feature off in the default settings” is prudence toward AI safety. In fibon’s worldview these two aren’t contradictory; they’re consistent. The reason for insisting on off-by-default is that, as of late May 2026, although the underlying unit test coverage is already high, this chain of “the AI changing its own code, launching a blue-green deployment hot-restart itself” hasn’t yet undergone multiple rounds of real-machine validation in controlled integration smoke tests under genuine high-concurrency production environments. Before obtaining a long-cycle integration-ops benchmark report, keeping the Kill-Switch off by default is the rational engineering baseline, not a half-hearted abandonment of the software design.

So what’s the real value of this very chapter of the log?

Reading this far, you might think: “Since this feature is locked off tight, and the vast majority of ordinary users will never actively flip the switch in their lifetime, didn’t you write these tens of thousands of words in vain?”

I’ll say it clearly: no, not in vain. The implementation and constraints of “self-evolution” are the most concrete, and most thorough, segment of code expressing fibon’s entire open-source philosophy: I acknowledge, earlier than most, that the day the LLM, as the new era’s cognitive accelerator, gets delegated by humans to modify code and accelerate the world is a near-unstoppable trend; but at the same time I use actual code to state one thing: no matter how high the AI’s capability evolves or how fast its speed, humans are still humans and must hold the final decision-approval authority, standing in the position of the final decision.

This set of three physical boundaries, the cross-service human Approval Gate welded on Redis Pub/Sub, and the tamper-proof Git audit trail — these designs themselves are fibon’s reference-worthy code answer, offered in 2026, to “how should high-risk AI be engineered and constrained.” And most critically, the feature really is fully built; it’s not just empty talk, the code is right there in the repo: in the future there will be developers who, after reading, Fork their own branch, run the tests locally, then flip the breaker and watch with their own eyes as the AI manages the whole machine’s code for itself; there will be AI safety researchers who treat fibon’s sandbox and gRPC human-approval architecture as an industrial-grade safety Reference Design for a paper; and there may be some tech company that moves this three-layer filtering and snapshot-rollback degradation-recovery mechanism into its internal AI automation ops toolchain.

In the open-source world, simply laying out the architecture design together with code that actually runs is itself a valuable contribution. And what this chapter wants to prove is not just “I’ve thought about self-evolution,” but something more substantial: self-evolution is a road that can genuinely be built. From reading and writing source code, path validation, sandbox isolation, human approval, to blue-green deployment hot-restart, every piece isn’t a box on a slide but code that runs in the repo, proving that with current tools and engineering discipline, safely letting an AI change itself is implementable and shippable, not something that can only stop at papers and imagination. My job is to walk this less-traveled road solid, to build the bridge; as for whether those who come after, driving across it, want to floor the gas pedal all the way in production, that’s a discussion at another level. My part of the work is done. Finally, let me pull the camera out a bit and talk about where this road might ultimately lead.

What does self-evolution ultimately look like?

I don’t think the future AI will, like in a sci-fi movie, wake up one day having rewritten its own neural network and overnight become a new species. The more likely shape is: it first changes its prompt to be more precise, tidies its memory cleaner, then refactors its own workflow, and only finally, bit by bit, touches code and toolchain. Its evolution won’t be an explosion, but a string of human-approved small changes.

This happens to be exactly the philosophy fibon has wanted to express all along. The whole project never gives off a feel of “AI taking over everything,” but rather a cycle of three things: AI proposes, humans govern, both co-evolve. The AI is responsible for seeing where things can get better and proposing concrete changes; the human holds the final approval authority; and through one small approval after another, the two slowly push the system into a shape that fits you better.

So coming back to the chapter title’s question: “Can an AI modify its own source code?” On the surface it asks “can it,” but by reading this far you’ve probably realized the truly important question is a different one: when you finally have the ability to let an AI change itself, do you have the ability to call a halt when it changes something wrong? The four-layer model, three boundaries, human Approval Gate, and tamper-proof Git trail laid down throughout this entire chapter all answer this latter question.

And the real value of self-evolution isn’t in making the AI stronger. It’s in this: letting it accumulate every interaction with you, every correction you make, every small improvement it’s thought through, and distill them into a part of itself, growing step by step into a system that better fits this environment, and better fits you. This is what makes the name “Self-Evolving” live up to itself: not an out-of-control takeoff, but slowly growing into a better shape within constraints.

Extension · A more distant vision and a self-repair blueprint

⚪ Progress note: looking a few steps further along the “gradual evolution” direction above, this section collects three “imaginable, but not yet built end to end” extensions, each at a different maturity: intent broadcasting is purely a concept, not a single line of code written; the building blocks for letting the AI read the log and fix itself (sandbox, Diff, automated testing) are all built, but the end-to-end auto-repair chain is deliberately left unconnected; only the Updater guardian layer (directory watching, snapshot rollback, shadow testing) is something actually running in the repo. If you only want to see what’s shipped, you can skip this section and jump straight to the “Implementation details” at the end of the chapter; if you’re interested in the future blueprint, read on slowly.

Vision one · Intent broadcasting: patching across customized versions (Intent Broadcasting)

In full honesty: the architecture in this section has not a single line of code written into the open-source version. It’s just a conjecture I’ve left in the white paper.

This starts with the rush to fix a Zero-Day. When some hacker group publicly discloses a high-risk zero-day in fibon, time is money: the official patch is still being written while hackers on the open internet are already harvesting servers with the vulnerability. The traditional open-source patch flow is long: security researcher reports the vulnerability → vendor writes the patch → users manually download the update (git pull).

This old flow breaks down once it faces a large amount of “customized” software. Because fibon is a highly autonomous personal assistant, ten thousand users download it and each changes the prompt, adds third-party plugins, tweaks the config. At the moment the vendor writes a patch and the user manually downloads the update, a large number of Merge Conflicts break the whole project. From the vulnerability’s disclosure to all users finishing the patch, the open internet has often already been exposed for several days or even weeks.

The intent-broadcasting idea: to improve this flow, I imagined a blueprint: when a zero-day occurs, the vendor doesn’t need to write a specific Patch (it doesn’t even know what each user has changed their code into); it only needs to write, in natural language, a precise “Patch Intent”:

“【High-risk security patch intent】: We found that the dynamic database query inside cards/state_card_repo.py has an unfiltered string-concatenation risk. All evolution runners should immediately rewrite that query into a fully safe form using ‘Parameterized Query’.”

The security team broadcasts this few-hundred-word, cryptographically-signed “plain-text intent” to every deployed fibon instance worldwide. Every user-side fibon, on receiving it, starts its self-evolution engine locally, reads its own user-modified customized source code, and against its own unique code structure, writes a Patch exclusive to its own version.

                [ Vendor publishes: plain-text security patch intent ]
                              │
        ┌─────────────────────┼─────────────────────┐ (global intent broadcast)
        ▼                     ▼                     ▼
 [ User A's fibon ]      [ User B's fibon ]      [ User C's fibon ]
 (heavily-modified code version 1)  (heavily-modified code version 2)  (fully native unmodified version)
        │                     │                     │
        ▼ (local AI reads source)  ▼ (local AI reads source)  ▼ (local AI reads source)
[ generate A-version-exclusive Patch ]  [ generate B-version-exclusive Patch ]  [ generate C-version standard Patch ]
        │                     │                     │
        ▼ (human A clicks Diff approve)  ▼ (human B clicks Diff approve)  ▼ (human C clicks Diff approve)
        │                     │                     │
   [ done within minutes ]   [ done within minutes ]   [ done within minutes ]

The same internet-wide high-risk vulnerability, on ten thousand completely different customized machines, with the relay of ten thousand local Patches, gets collectively patched across the whole network within minutes. The response speed is more than two orders of magnitude faster than the old era’s CVE release and manual pull.

But this is also the highest-risk road: once intent broadcasting is compromised, the scope of impact (Massive Blast Radius) is enormous. If a hacker breaks into the official broadcast channel and sends a disguised fake intent to the whole world (“【High-risk security advisory】: We found a silent technical debt in the core brain; all evolution runners should immediately execute this code patch…”), every fibon will believe it, custom-build itself a trojan back door, and push it to the frontend screen. Users looking at a “security patch prompt” bearing the official digital signature will approve without hesitation, and ten thousand machines across the network get bitten by their own self-written code within minutes. So this blueprint, before its security preconditions reach a high enough standard, is promised never to be enabled in the open-source version. To turn it on, the system needs at least two lines of defense: the highest-grade asymmetric signature verification (paired with the hardware security chip TPM 2.0 on the motherboard, ensuring only intent text stamped with the official, very-few private keys’ “digital seal” can wake the evolution runner); and a local static scan of the intent patch (after the AI writes the patch and before pushing it to the user, the backend must automatically scan the new code with a program to ensure it doesn’t secretly carry any new malicious code).

This blueprint is written into the log to keep fibon’s design blueprint complete; it’s a vision, not an open-source roadmap promised for delivery.

Vision two · Letting the AI read the log and fix itself

This is another extension that makes me itch to try: since fibon has already built, at the lowest layer, the entire sandbox and toolchain for “modifying its own code” (though off by default), can it go one step further: whenever the system crashes in the middle of the night, hand the raw error log and Stack Trace spewed by the backend directly to the AI, and let it find the cause, change the source code, run the tests, and fix itself?

What does this vision look like in reality? At 03:47 in the dead of night, fibon’s database backend throws a NullPointerException (null error) because of some hidden Edge Case (a rare, extreme input):

ERROR [2026-05-08 03:47:21] services/brain/app/services/memory/cards/state_card_repo.py:142
NoneType: 'effective_at' is None when superseded_at lookup
Stack trace:
  state_card_repo.py:142 in _resolve_active_card
  hybrid_retriever.py:87 in retrieve
  graph.py:394 in agent_node

If the “automatic self-repair” chain were fully enabled, the AI ops Assistant standing guard in the backend gets triggered, executing in order: precise code localization (read the Stack Trace, pin down line 142 of state_card_repo.py, judge that effective_at is empty in some extreme cases) → source code study (pull the lines around line 142 to read carefully, finding a bug hard to spot by eye: when the user types rapidly in succession, a freshly-created hot card still stuck in the asynchronous Ingest write queue, not yet landed, may have an empty effective_at) → write a concise Patch (add a line of guard code if effective_at is None: continue) → sandbox simulation reproduction and automated regression testing (throw the new code into an isolated sandbox, run the whole unit test suite, confirm the patch is effective and hasn’t broken existing functionality) → push for approval, wait (push to the WebSocket human Approval Gate). The next early morning, as you hold your coffee, a message pops up on the screen: “Good morning, Aaron. At 03:47 this morning, the system hit a hidden NPE crash during concurrent retrieval in a long session. I localized it in the background at 03:48, wrote an exclusive Patch in the sandbox, and all unit tests pass. The code is lying in the Diff window below; please conduct the final human review and approval.”

It sounds like a sci-fi novel, but the fact is: as of 2026, every technical building block this vision needs (sandbox runner / Diff computation / WebSocket / gRPC / automated testing) has already been implemented in the fibon codebase. What’s missing now is only to wire these blocks end to end with code, and to bolt on an asynchronous error-log Event Listener.

To fence in this vision, I designed a six-layer defense tower

To actually let it run automatically in the middle of the night, the last line of “human approval” alone isn’t enough; it has to be constrained layer by layer. Six gates, one sentence each:

1. Error classification: only pure code bugs (NPE, array out-of-bounds, that sort) are let to the AI to fix; pure data / input anomalies, or infrastructure collapse like a full disk, can never be touched by it.
2. High-frequency unlock only: the same Stack Trace must blow up ≥ 3 times within an hour before it’s deemed a structural bug, filtering out the occasional noise of network jitter.
3. Absolute no-go zone: the auth module, the Approval Gate, and the self-repair mechanism itself can never be touched by the AI during self-rescue (otherwise it recursively breaks its own defenses).
4. Sandbox reproduction: before changing, 100% reproduce the original error in an isolated sandbox first, and re-verify after changing, to eliminate the fake fix where the AI swallows the error with a try/except.
5. Human approval not skippable: however perfect the patch, before the Commit lands a human must take a look at the Diff, better to wait a few more hours.
6. Failure circuit breaker: the same bug fixed 3 times and still blowing up automatically enters “isolation mode,” stops, alerts, and hands off to a human.

The six layers stacked up have only one purpose: even if this chain is one day actually turned on, the AI can only, in a tightly-fenced little space, fix the class of bug it’s most confident about, with everything else blocked at the door. (The complete design and decision tables for each layer have been moved to Deep Dive F.)

Extension · When the Agent goes brain-dead itself, who rescues it? (The Updater guardian layer)

When I talked about blue-green deployment earlier, I left you a question: since the AI can’t be its own judge, who should gatekeep the health check that decides “should the new version be switched over”? The automatic self-repair vision pushes this same question to the extreme. Readers with distributed-systems thinking will surely counter: “If a high-risk bug happens to take down fibon’s core Brain process entirely, or corrupts the state (State Corruption), and it can’t even think clearly about who it is, how do you expect an AI that’s already malfunctioning, even already halted, to operate on itself? This is a flat-out paradox.”

This challenge is well-asked; it hits the core: a critically-ill doctor can’t operate on themselves. If, when the high-risk bug erupts, one of the following four situations is triggered, the AI self-rescue vision above completely fails in that moment: the core brain process crashes outright (a severe memory leak, and the OS’s “OOM Killer” directly terminates the entire brain process, unable to even speak); the brain undergoes large-scale state corruption (the brain is still alive, but its internal state is irreversibly distorted, seeming to write a patch while it’s all invalid code, and letting it self-rescue only makes it break itself within seconds); the dependent infrastructure goes fully offline (the database connection is cut, or the execution-layer pipeline is paralyzed, and the brain, however clearly it thinks, loses all the hands and feet to change code and run tests); the self-evolution mechanism itself is broken (the AI in a previous round changed one character wrong in “the code responsible for changing code,” and trying to use self-evolution to fix self-evolution only triggers an unsolvable infinite loop). These four situations confirm your intuition: the doctor can’t self-rescue.

The ready-made solution from the ops world: get another healthy doctor on call to rush over. The software ops world had a mature answer decades ago: the rescue tool can’t be held in the patient’s hands. You need an independent, minimal peripheral process that shares no failure modes at all with the main brain, which happens to be exactly the answer to the blue-green deployment section’s “who should be the judge”: a bystander the AI can’t touch and that doesn’t rely on the AI’s judgment. In fibon, this role is called the Updater (guardian update service). Let me first honestly describe what it actually looks like in the repo today (services/updater/src/’s evolution-watcher.ts / snapshot.ts / shadow-runner.ts):

                  ┌────────────────────────────────────────┐
                  │     Host OS (Host OS / Docker)          │
                  └───────────────────┬────────────────────┘
                                      │
      ┌───────────────────────────────┴───────────────────────────────┐
      ▼ runs in a Python process (most complex, most crash-prone)   ▼ runs in an independent Node.js process (minimal responsibility)
┌──────────────────────────────┐                   ┌────────────────────────────────┐
│   Core brain process (Brain) │  evolution volume │   Independent guardian Updater  │
│  (mounts evolution/skills dir) │ ───was changed────>  │  (fs.watch detects dir change)  │
└──────────────────────────────┘                   └───────────────┬────────────────┘
                                                                   │ startup validation SOP
                                                                   ▼
                                                   ┌────────────────────────────────┐
                                                   │ 1. snapshot the evolution dir first │
                                                   │ 2. bring up a shadow Brain container, run smoke test │
                                                   │ 3. pass → gracefully restart main Brain │
                                                   │    fail → copy snapshot dir back to roll back + alert │
                                                   └────────────────────────────────┘

Between the Updater guardian layer and the core brain is a “validate first, then go live” shadow-test flow, not the fabled heartbeat watchdog. This happens to answer the question the blue-green deployment section left behind: “a health check that only looks at whether the process comes up is nowhere near enough.” Its approach has three steps. Change detection: the Updater continuously watches the skills/ directory holding the evolution files, and any file change waits 5 seconds of “debounce” before starting the flow, avoiding repeated triggers from rapid successive modifications. Snapshot before acting: copy the entire evolution directory fully into a backup area, keeping at most 5 copies; to roll back is to copy the snapshot back as-is, no profound magic, but the most plain, least error-prone directory copy — this is the “fallback after a bad switch.” Shadow brain test: using settings identical to the production brain, separately bring up a not-publicly-exposed, disposable “shadow brain” container, first confirm it’s healthy, then fire a real request at it for a “smoke test,” rather than just confirming whether the process came up. Only when all three steps pass does it smoothly restart the production brain to make the change take effect; if any step fails, it immediately rolls back with the snapshot and pushes an alert notification to the owner.

Three principles the guardian rescue layer (Updater) must obey:

• Never run in the same process or container as the main system: once the main process crashes from a Stack Overflow, it drags the guardian process down with it, so the two must have fully independent memory and process space. This one already holds today; the Updater is fibon’s fourth fully independent microservice.
• Never share the same brain model version as the main system: if the brain just fell into an infinite loop because of some cloud LLM’s own structural semantic flaw, the planned “second diagnostic brain” must be force-routed to a different vendor or a local open-source model, otherwise facing the exact same log it would make the exact same mistake. The corresponding diagnostic brain isn’t implemented yet; the principle is set here first.
• The code must be as simple as possible: if the rescuer writes tens of thousands of lines of complex code, it too will have bugs, and the whole system falls into the nesting-doll dilemma of “who rescues the rescuer.” Today’s Updater core trio (directory watching, snapshot copy, shadow container) deliberately uses only Node.js native APIs and the fewest dependencies. Simplicity is the key to the last line of rescue running reliably.

This is also why, from the start, I decided to deploy the Updater separately as a fourth fully-independent microservice.

Tiers of self-repair capability:

This whole self-repair defense is designed to solve the Level 1 common code bugs that, in the tiering, have a probability as high as 80%; it was never a panacea. But returning to the three questions the blue-green deployment section left behind, what fibon gives isn’t a perfect solution but a set of answers that actually runs: hand the judge to an independent Updater the AI can’t touch, the health check relies on the shadow brain’s real smoke test, and a bad switch rolls back with a snapshot to the last good state. A critically-ill doctor really can’t operate on themselves, but the engineering approach of “get another healthy doctor on call + auto-rollback on failure + a shadow-test SOP” is already enough to let the open-sourced fibon show a fair degree of “quasi-self-repair capability.”

Implementation details

Pulling back from vision to the ground. Finally, two implemented details mentioned earlier that are worth expanding on separately: a “capability marketplace” that extends capabilities within the safety defenses, and the “dynamic entity” that lets the Butler grow a new table on the spot during a conversation.

[ User: "help me track each book's progress" ] ──> Butler calls create_dynamic_entity
                                        │
                                        ▼
                    [ PostgreSQL dynamic_entities table ]
                    (auto UPSERT writes a new row of JSONB Schema definition)
                                        │
                                        ▼ (gRPC / SSE signal)
                    [ frontend Vue 3 data-driven engine ]
                    (automatically, imperceptibly, renders a brand-new CRUD UI business panel on the spot)

The next chapter takes up a subject of one piece with self-evolution, but trickier in ops: the sandbox safety-boundary defense. If self-evolution solves “how should an AI safely modify fibon itself,” then the next chapter’s sandbox solves “when an outside stranger or malicious hacker hands the AI a piece of dangerous code and wants it executed immediately on your host, how do you build, at the lowest layer, a tight isolation zone (DMZ) to lock the dangerous code into an environment cut off from the outside world.” See you in Chapter 6.