How Does an AI "Remember" You?

Quick summary: How fibon turns conversations into searchable long-term memory — three kinds of cards (state / event / knowledge), cardinality, five-channel fusion retrieval, and conflict arbitration.

Skip if: All you want is the conclusion — “it remembers facts across sessions.” The low-level teardown lives in Deep Dive E. (This is the longest chapter on the whole site — the memory system took 2–3 months and still isn’t fully finished, so it runs long; feel free to skip around to the sections you care about.)

Where the Problem Begins: The Pain of a Single Chat Window

Recall the scenario from the opening of Chapter 1: you had a deep conversation with an AI about a complex topic (say, planning a 5-day self-guided trip to Kyoto), and a week later you want to pick it back up: “By the way, do you still remember that hotel shortlist we talked about?” At that point, one of two things usually happens:

• Case A: You open last week’s chat window → the AI can scroll back through the history and continues seamlessly.
• Case B: You casually open a new window → the AI suffers instant amnesia and greets you with first-meeting politeness: “How can I help you today?”

The essential difference between these two, in industry jargon, is the “cross-Session” problem.

What is a Session?

A “Session” is one chat window, or a single conversation thread. Every time you click “New conversation,” you start a brand-new Session. The design philosophy of today’s mainstream AI interfaces (ChatGPT, Claude, Gemini) is the same: within a Session, the AI remembers what was said earlier; the moment you switch Sessions, memory drops to zero.

There is engineering sense in this — confining memory to a single window keeps the AI from treating irrelevant chatter from three months ago as present-day background noise. But if you want the AI to be a personal assistant that stays with you over the long haul, this design is a disaster: you would never re-introduce yourself to your private secretary at every meeting with “Hi, my name is Aaron, I live in Taiwan, and I’m building a project called fibon.” A real assistant must have long-term memory that crosses time.

This is exactly the core pain fibon set out to solve: how do you make an AI remember you firmly, across different chat windows? Smart Chat mode and the entire memory system were born around this goal.

The Container for Memory — fibon’s “Projects”

Before we get to memory cards, I need to introduce a container concept that looks tiny but is actually decisive: fibon’s “Project.”

Why do we need Projects? Because your life is not a linear extension of a single topic. Right now you might be juggling 5 unrelated things: planning a Kyoto trip, learning AI design patterns, writing a cloud-and-on-prem disaster-recovery tech report, making dinner plans with a friend, researching real estate. The memories and context of these 5 things are logically independent. If you lazily dump all memories into one “big memory pool,” disaster strikes fast: you ask “find that hotel shortlist from last time,” and the AI might dredge up “the site-selection shortlist for an off-site data center” from your DR report; you ask “how do I implement the Reflection pattern,” and it might surface the transit plan for your Kyoto trip. The whole memory store descends into chaos, and every retrieval drags in piles of noise.

A Project is a topic folder with “physical isolation.” You create a “Kyoto Trip 2026” Project, and from that moment every conversation, fact, and piece of knowledge about that trip is automatically tagged with the Project; switch to “Learning AI Design Patterns” and the memory pool is fully disconnected. Every card carries a hard project_id in the table, and memory retrieval defaults to searching only the current Project’s cards — memories from other Projects are physically, naturally isolated.

The Three Foundation Bricks of fibon’s Memory

With the container covered, the core question becomes: what shape of data goes inside the Project folder? The answer is three kinds of small cards.

This is also the core thesis of the whole chapter, stated plainly: long-term memory should not store “conversation logs”; it should store “state, events, and knowledge.” Put more fully: long-term memory isn’t about storing conversations — it’s about turning facts into a structure that can age, be superseded, be traced, and be arbitrated; and that structure must be tested longitudinally, or it will look fine in a short demo and quietly rot over long-term use. The rest of this chapter is simply delivering on those four adjectives.

These three card types were not invented out of thin air. The first question I asked myself while designing them was: “What are the actual ingredients of something that gets remembered long-term?” I took the scenario I know best — software development — and deconstructed it. A project travels from requirements analysis through system analysis, development, debugging, deployment, and iteration. What does that road leave behind? Every team communication, every requirements change, is an “event” — once it happens, it never changes. Which features the project currently supports, which version it runs on, is “state” — it can be updated at any moment. And why the project exists, which principles it uses, which problems it has solved — that is distilled “knowledge.” There is one more ingredient hiding underneath everything — “time”: every communication has a point in time, every state has a validity interval, every piece of knowledge has the day it was learned. Events, states, knowledge, plus the time that threads through all three — at least so far, I have not encountered anything that cannot be deconstructed into these four ingredients. That is where this card system comes from.

As for the “card” form itself, the inspiration is borrowed from modern note-taking methodologies:

When humans record and digest knowledge, writing short “small cards” beats writing long prose by a wide margin — cards have high information density, are easy to search, and easy to recombine. fibon applies the same philosophy to AI memory: break long conversations into atomic little cards, instead of stuffing in entire transcripts.

Why are “small cards” better for an LLM? This was my core hypothesis when designing the memory system: “The smaller the input and the more precise the facts, the lower the LLM’s hallucination probability should be.” The reasoning is that LLMs are extremely easy to distract with irrelevant filler — dump in a giant blob of conversation history and the key points get diluted (there is a famous paper, Lost in the Middle: with long contexts, LLMs are most likely to miss critical information buried in the middle). A short, distilled structured card (for example [Current employer: Anthropic]) lets the LLM grab the core fact at a glance far better than a long stretch of dialogue. Put differently: instead of throwing whole chat transcripts into a vector database for crude full-text retrieval (what comes back is still filler-laden paragraphs), fibon’s structured extraction (card-ification) is a purpose-built engineering patch against Lost in the Middle — separating the meat of a fact from the bones in the background, and feeding only the concentrated essence to the LLM.

fibon’s three card types

To mimic how a human brain actually remembers things, cards come in three major types:

One member is missing from the table — time. It is not a fourth card type, but the fourth dimension hiding underneath all three: an event card’s occurrence time, a state card’s effective and superseded interval, a knowledge card’s learning and update trajectory — all pinned to the same timeline. It is so important that two later sections deal with it head-on (the memory half-life in state-card rule three, and the “time anchoring” section). I name it here because as you read every design below, you should carry this awareness: every card lives in time.

State Card vs. Event Card — Deconstructing “Project State”

These two cards look similar at first glance, but they solve completely different cognitive pains. Compare with a real scenario: suppose you opened a “fibon open-source plan” Project, and over the past two months the following happened:

• 2026-03-01: Project kickoff, targeting an open-source release in 2026-07.
• 2026-04-15: Architecture meeting decides to introduce skill compliance verification (Skill Compliance).
• 2026-04-30: The decision on whether to enable self-evolution: Path B chosen, downgraded to a safe reference design.
• 2026-05-05: Aaron starts writing this design log.

Ask the following two questions, and the brain’s memory-retrieval paths are radically different:

Question A: “Where does fibon’s progress stand right now?” This needs a [state card]. The AI does not have to dig through two months of conversation; it only needs the single truth of this moment:

[State card: fibon open-source plan]
Current stage  = pre-release acceptance
Last updated   = 2026-05-05
Next milestone = open-source release in 2026-07

This card keeps updating over time — last month’s “wrapping up the previous round” gets superseded. Whenever you ask, what the system returns is always the latest version that holds true right now.

Question B: “Why did we decide back then not to enable self-evolution?” This needs an [event card]. “Current state” cannot answer it; the AI must wind the clock back and find the event that sits, forever unchanged, on that coordinate axis:

[Event card: 2026-04-30 decision meeting]
What happened  = three-way evaluation of whether to enable self-evolution
Final decision = Path B chosen (downgraded to a safe reference design)
Core rationale = lowest safety-narrative risk, lowest engineering cost

This event card will never be overwritten — history has happened, and it must not be tampered with. Look it up three years later, and the facts of 2026-04-30 still lie right there.

Why must the architecture separate the two? Using only one card type causes cognitive collapse: with only state cards, historical events all vanish, and you can never answer “what was the motive and context of that decision at the time”; with only event cards, historical facts stack endlessly, and when you want “the latest address right now,” the AI has to read through every move you made over the past 10 years and derive it itself.

So each does its own job: state cards can be replaced and answer “now”; event cards can only accumulate and answer “the past.” They can also link across types — for example, the 2026-04-30 [event card] points to and affects the [state card]‘s [self-evolution status = not enabled].

The Four Core Maintenance Rules of State Cards

Because state cards “update, get replaced, and decay,” four hard rules were carved into the foundation to prevent factual chaos.

Rule one: the Supersede-Fork mechanism

The fact that state cards update carries a philosophical hole: from what day is the sentence “fibon’s current stage is pre-release acceptance” true? And from what day does it stop being true? Without recording time, historical auditing is ruined. So every state card carries two timestamps:

[State card: fibon open-source plan]
Current stage                 = pre-release acceptance
Effective at (effective_at)   = 2026-05-05  → true from this day on
Superseded at (superseded_at) = NULL        → still true, not yet superseded

When the stage changes, the old card is never Deleted — it is “marked Superseded”:

[Old state card - marked superseded]
Stage         = wrapping up the previous round
Effective at  = 2026-04-15
Superseded at = 2026-05-05   → from this day on, this fact is no longer true

[New state card - INSERT]
Stage         = pre-release acceptance
Effective at  = 2026-05-05
Superseded at = NULL

Both facts coexist in the same table, just like amending a law — the old clause is not tossed into the incinerator; it is annotated “this article was superseded by the new law as of such-and-such date.” Ask “current progress?” → filter WHERE superseded_at IS NULL and return the latest card; ask “what was the progress in April 2026?” → run temporal-history retrieval and find the card whose time interval matches. This is Fact Auditing: history traceable, the present confirmable.

Rule two: the doppelganger rule — replaced (single_value) vs. accumulated (multi_value)

When processing a new memory, you must distinguish the fact’s cardinality:

• “Gets replaced” type (single_value): home address, job title, browser preference, current employer. New data triggers the Supersede-Fork above and banishes the old fact to the cold palace.
• “Keeps accumulating” type (multi_value): technology choices, interests, skills, topics discussed. You wrote “used PostgreSQL,” and the next day “also used Redis” — the two coexist.

On write, the conflict arbitrator (conflict_resolver) first reads the card’s cardinality attribute to pick the route. A boundary that is easy to confuse needs drawing here: rule one’s Supersede-Fork is the exclusive treatment of single_value cards (gated by the SUPERSEDE_FORK_ENABLED flag); a multi_value card whose content is an entirely new fact gets a parallel row via direct INSERT, while an update colliding with an existing card of the same name does not fork — it goes “overwrite in place” and leaves an audit entry in card_update_log.

Rule three: the half-life decay of memory

Distinguishing replace vs. accumulate is still not enough — time is memory’s greatest enemy. “The project plans to open-source in July,” written 30 days ago, is still highly relevant today; “I recently switched to Firefox,” tossed off 6 months ago, is most likely stale today; “I love Python,” said 6 months ago, most likely still holds today. Different fact types decay at completely different speeds (half-lives). The retrieval scoring formula introduces mathematical exponential decay:

final relevance score = raw vector score × e^(−λ × Δt)
(Δt = card age in days, λ = decay coefficient)

With this patch, the 90-day-old [Firefox] score gets heavily discounted, letting today’s [Chrome] win easily; while the 90-day-old [loves Python] still keeps a high weight of 0.64 and gets retrieved alongside today’s [wants to learn Rust].

Aging Facts warning: To get closer to a real assistant, an aging-warning mechanism was introduced: if a replaced-type state card has sat in the database for more than 180 days without being re-verified or superseded by a new value, the system automatically wraps it in an <aging_facts> block when retrieved, and the AI proactively confirms in its reply: “Aaron, I remember you used to live in Taipei — is that still accurate after half a year?”

Rule four: proactive conflict arbitration

If the same fact is in serious contradiction across two conversations (yesterday you said fibon plans to open-source in July; today you say it moved to August), whom should the system believe? Most implementations go “the new wave crushes the old” — brutally overwrite yesterday with today. But that has a huge hole: what if you misspoke today? What if the LLM confused the context and produced an extraction hallucination?

fibon holds that the system should not decide on the user’s behalf — when there is a real conflict, it should raise its hand and ask. Three layers of conflict filtering:

1. Same-turn self-correction: in the same conversation you say “I live in Taipei” and immediately type “typo, I meant Kaohsiung” → overwrite directly, don’t bother the user.
2. Cross-conversation low-frequency change: fewer than 3 changes within 30 days → judged normal fact evolution, handled automatically without bothering the user — single_value cards (with the SUPERSEDE_FORK_ENABLED flag on) take rule one’s Supersede-Fork to preserve history; multi_value cards overwrite in place and leave an audit entry in card_update_log.
3. High-frequency or wildly spanning contradiction: the change spans ≥ 30 days, or there have been ≥ 3 frequent modifications → immediately pushed into the “arbitration waiting queue.”

When the third case fires, the frontend lights up a “conflict arbitration panel” showing the two fighting facts, with three buttons for your verdict: [keep the old fact] / [accept the new fact] / [rule that both coexist].

The last line of defense — the Contradiction Detector. Within a millisecond of retrieval completing, the backend runs a final sweep. If, among the cards about to go into the prompt, two share the same tag but their contents fight (for example, retrieving both progress “wrapping up” and “acceptance passed”), the foundation injects a <contradiction_alert> block into the system prompt, and the AI proactively confirms: “Aaron, my memory store contains these two contradictory progress entries at the same time — which one should I go with?” This is the final safety net before anything enters the arbitration queue.

Time Anchoring — Where Exactly Should the Word “Today” Be Pinned?

Everything so far was about rules inside the cards, but there is one temporal hard wound that cuts across all of them: when you tell the AI “I moved today,” which day exactly is this “today”?

Never store it as the literal string “today.” The most instinctive beginner mistake is to store the user’s spoken “today” as a string in the database. That breaks spacetime: yesterday’s today is called yesterday; tomorrow’s today is called tomorrow. A week later the AI retrieves the card, reads “moved today,” and mistakenly believes the move happened “on the day the card is being read” — the whole timeline is off by 7 days. So relative times like “yesterday, tomorrow, last week, next month, this weekend” must be interpreted and nailed down to absolute dates (such as 2026-05-07) in the very second the conversation happens — deferred processing is not allowed.

The LLM itself has no “sense of time.” Worse still, an LLM is a blind person without a watch: its training data’s timestamps stop in the past, and at the moment the API is called it simply does not know “what year, month, and day it is now.” Without strapping a watch on it, it cannot convert relative time into absolute time while extracting cards.

fibon’s solution: the time-pinning flow on the write side (Ingest Flow)

Every time the user hits send and the conversation enters background memory internalization (the Ingest flow), the foundation fires a chain reaction:

1. Grab the current clock: obtain the server’s current precise local time in the user’s timezone (e.g., Asia/Taipei).
2. Forcibly strap on the watch: inject a dynamic time tag into the System Prompt assembled for the LLM — <current_time>2026-05-08T16:44:10+08:00 (Asia/Taipei)</current_time> (strictly ISO 8601).
3. Force the LLM to resolve time during extraction: command the LLM, while extracting cards, to output a custom occurrence-time hint field (called occurred_at_hint in the code), reading the watch to convert relative words into absolute dates (output “yesterday” as 2026-05-07).
4. Hard write to the database: the backend parses the hint and converts it into a timestamp written to the card’s occurred_at. If the LLM fails to resolve it, the backend immediately degrades safely and forces the timestamp of the moment the message entered the backend — better a rough one-day error than a lost timeline.

A hard rule is baked into prompt_builder.py’s guard rules: “When the user mentions relative time words such as ‘today / yesterday / last week / next month / the weekend,’ they must be unconditionally interpreted and converted using the dynamically injected <current_time> tag as the sole reference.”

Time-range queries on the read side (Retrieval)

Once all dates on the write side become absolute, queries on the read side become elegant. The user opens a new window and asks “what did I do yesterday?” — at query time <current_time> is 2026-05-08, the Butler reads the watch and translates “yesterday” into 2026-05-07, and the backend fires a SQL Range Query against occurred_at with that absolute date. With the write side and the read side each holding a precise dynamic clock, the spacetime-confusion problem is completely dissolved.

The Third Card Type — Knowledge Cards

The previous few sections all dealt with “fact-level” memory (state cards and event cards). But the human brain holds another, higher-order treasure: abstract concepts and objective knowledge. You figured out [the stateless authentication principle of JWT], mapped out [the perfect transit route to Fushimi Inari Shrine], teased apart [the essential difference between Plan-Execute and ReAct architectures]. These are neither “personal facts true right now” nor “single events of the past” — they are high-density abstract knowledge: citable across sessions, bidirectionally linkable, deepening over time. This is fibon’s third card type: the [knowledge card].

The ultimate goal of knowledge cards: growing your own “personal knowledge base (Personal Obsidian)”

The inspiration comes from the note app Obsidian’s backlinks and graph visualization. In fibon’s vision, every time you converse, research, or argue, fibon’s backend automatically distills the valuable knowledge points into independent concept knowledge cards, and the cards wire up related_to bidirectional reference edges among themselves like a neural network. Over time, the backend grows a “digital-brain knowledge graph” that belongs entirely to you.

Let me be clear about what this is: a vision. The knowledge-card machinery itself is built — and switched on in the open-source launch (see Implementation details 2) — but whether it actually makes the AI answer more accurately and more cheaply hasn’t yet been validated by real measurement. What follows is what it should deliver, not a conclusion already measured.

The part I care about most is knowledge reuse, and there’s an easily-overlooked cost structure behind it: for an LLM, the input tokens it reads are far cheaper than the output tokens it generates (most providers price output several times higher than input). So if a knowledge point is figured out once and distilled into a card, the next time it’s needed you just read that ready-made card back in as input — instead of making the LLM “think” and “write” it all over again. What you save is exactly the most expensive part: the output. Knowledge-card reuse is a bet on that lever: swapping the costly “regenerate from scratch” for the cheap “read back a ready answer.” This ledger, too, only gets signed off against real traffic; for now it’s a direction with a sound rationale, still awaiting measurement. (The full ledger behind “output costs more than input, and reuse saves the output” is in Deep Dive C: Token Economics.)

The hardest part of knowledge cards: keeping them from becoming a garbage dump

State cards have a “single current truth” to converge on; event cards have a “timeline” to anchor to. Knowledge cards have neither — knowledge has no single objective answer, and it evolves over time (JWT best practices, how to write the Reflection pattern, the interpretation of DDD all shift every year or two). Left unattended, knowledge cards quickly grow into a heap of duplicate, stale, mutually contradictory junk — at which point they’re a liability, not an asset. This is the fundamental reason knowledge cards are harder, and were switched on later, than state and event cards. fibon currently holds this entropy down with three mechanisms:

1. Tag canonicalization: semantically close tags merge automatically, so JWT and json web token don’t split into two cards each growing its own disconnected tree.
2. Freshness flags: a card not re-verified for over 90 days gets stamped “this summary may be outdated” when retrieved, and after citing it the AI proactively asks whether you want to re-verify — rather than passing off old knowledge as current.
3. Binding to event cards: every knowledge card links back to “which conversations you actually discussed it in,” so it has provenance and is traceable, not an orphan conjured from nowhere.

But honestly, these three only slow the rot; they don’t guarantee freshness. Whether knowledge cards still degrade into a garbage dump at larger scale and over longer time is an open problem I haven’t validated — and the one I most want to keep working on.

Combining knowledge cards with event cards

The most charming part of this multi-dimensional structure is that knowledge and lived experience cross-reference each other:

[Knowledge card: JWT (JSON Web Token)]
Definition       = a lightweight, stateless, JSON-based identity authentication mechanism.
Related concepts = [OAuth 2.0], [Session-based Auth], [Bearer Token]
↓ [ Events in your life that have discussed and witnessed this knowledge card ]
- Event card #001 (2026-03-15): heated debate with Claude over JWT vs. Session for fibon's auth system.
- Event card #002 (2026-04-02): late-night fix of a sneaky JWT refresh-token expiry bug.
- Event card #003 (2026-04-20): used this card at the biweekly architecture meeting to walk teammates through the auth comparison.

This means your AI assistant doesn’t just remember the objective definition of a technology — it remembers “the real stories that happened between you and this technology in the past.” Next time you ask in a new window, “How exactly should I use JWT?”, its answer will be: “Aaron, you discussed JWT with me 3 times across March and April. Quick refresher on the core definition… which part do you want to dig into today? The auth architecture you built for fibon on 3/15, or that refresh-token expiry bug that gave you a headache on 4/2?” That is the romance of personally growing a knowledge base for every user.

The 3 × 6 Multi-Dimensional Geometric Structure Across All Three Card Types

At this point the memory map is complete: 3 atomic card types, plus the four rules inside state cards. But the real architectural advantage is here: every card, regardless of its type, is hard-threaded through by 6 cross-card geometric coordinate axes (6 Dimensions).

The underlying schemas of mainstream AI memory systems (LangChain memory, Letta, mem0) are almost all flat, one-dimensional “fact rows” — one row is one fact, with no multi-dimensional coordinates. fibon went multi-dimensional from day one: a geometric cube of 3 atomic card types × 6 cross-card coordinate axes. One clarification first: these 6 axes are logical coordinates, not 6 columns sitting side by side on one table — the time, scope, source, and confidence axes live as columns on the cards themselves, the relation axis lives in the separate card_relations table, and the concept axis is held up jointly by flat tags and the knowledge-card link tables. The table below shows each axis’s physical landing spot in PostgreSQL:

How do you play with this “3 × 6 structure”? With a schema like an OLAP cube (online analytical processing cube), you can slice, dice, and aggregate memory along any dimension:

• “Find all knowledge cards created this month.” → lock the [Concept axis] to knowledge cards + slice the [Time axis] to this month’s interval.
• “What share of cards came from the research Assistant vs. the coding Assistant?” → Group By agent_id on the [Source axis].
• “Flag the facts the AI inferred on its own rather than ones I said out loud.” → filter provenance = 'agent_inferred' on the [Confidence axis], then pair with inference_confidence to grade inference confidence.
• “Which cards are most strongly related to ‘JWT’ yet logically contradict each other?” → lock the [Concept axis] to JWT + pull contradicts edges on the [Relation axis].

These multi-dimensional queries simply cannot be written against LangChain’s or Letta’s flat schemas, because their data structures never reserved these axes.

The Invisible Backend Heavy Lifting — “Five-Channel Fusion Retrieval” When Fetching Memory

Everything so far solved “how to store memory precisely.” When the user asks a question, “how to retrieve memory flawlessly” is a separate discipline.

The most pedestrian approach: question → embed into a vector → compute vector similarity (cosine) → fetch Top 5 and feed the LLM. In a complex personal-assistant scenario, pure vector retrieval leaks like a sieve: it is dull to “exact project-code matching,” easily misses on “typos or rare words,” is helpless against time queries like “what did I do yesterday,” and has no ability to “expand relationships along the knowledge graph.” So fibon’s retrieval backend runs a “five-channel parallel recall and fusion architecture”:

                          [ User Query ]
                                │
  ┌────────────┬────────────┬───┴────────┬────────────┬────────────┐
  ▼            ▼            ▼            ▼            ▼
1. Semantic  2. Exact tag 3. Dual-eng. 4. Time-range 5. Graph
   recall       matching     fuzzy        query         expansion
 (pgvector)  (Tags Exact) (bigm+trgm)  (Temporal)    (1-hop graph)
 [ Top 20 ]  [ all hits ] [ Top 20 ]   [ in range ]  [ 5 seeds ≤30 ]
  └────────────┴────────────┴───┬────────┴────────────┴────────────┘
                                ▼
          [ Reciprocal Rank Fusion (RRF, k=60) ]
       (multi-judge voting builds the initial ranking)
                             ▼
     [ Backend core re-ranking and post-filter mesh ]
       - mathematical decay by state-card cardinality and age
       - scope boost for the current conversation context layer
       - Read-Your-Own-Writes (RYOW) for just-written, not-yet-persisted memory
       - hard removal of zombie cards whose superseded_at is set
       - last-line contradiction detection and aging warnings
                             ▼
           [ Distill the purest Top 5 memories ]
                             ▼
      [ Inject into the System Prompt and feed the LLM ]

The five channels each do their job and cover each other’s blind spots:

• Semantic similarity: great for keyword-free, feeling-driven queries like “have I ever grumbled about programming languages before?”
• Exact tag matching (Chinese must be segmented first — worth spelling out): when you issue precise commands or filters, it pulls every #python card without missing one. But Chinese has a prerequisite hurdle — there are no spaces between characters, so the sentence must first be cut into “words” before it can match tags. This step uses jieba for segmentation, loaded with Academia Sinica’s traditional-Chinese dictionary dict.txt.big (jieba’s default dictionary leans simplified and mangles traditional compounds like 資料庫 / 牛肉湯); English is already space-delimited, so jieba passes it straight through. The resulting words then exact-match card tags.
• Dual-engine fuzzy matching (character-level, segmentation deliberately skipped): the tag channel works on “words,” but users often make typos or use rare words and abbreviations, where segmentation just gets in the way. So this channel skips segmentation entirely and matches fuzzily at the character level — and since character granularity differs between Chinese and English, it runs two PostgreSQL extensions in parallel: Chinese uses pg_bigm on “adjacent two-character grams” (bigrams — e.g. 記憶體 → 記憶, 憶體), best at catching Chinese typos, person names, and rare proper nouns; English / Latin uses pg_trgm on “adjacent three-letter grams” (trigrams — e.g. cat → c, ca, cat…). The two predicates are OR-ed in a single SQL statement; PostgreSQL’s bitmap OR scans both GIN indexes at once and takes the higher of the two similarity scores (GREATEST), so neither Chinese nor English queries slip through — also a safety net for when the vector model’s clairvoyance fails.
• Time-range query (PR-5): it parses time expressions in the question (“last week,” “back in April,” “from last month to now”) into an absolute date range and directly fetches cards whose effective_at / occurred_at fall inside it. This channel is exactly where the earlier “Time Anchoring” section pays off — the write side pinned relative time into absolute dates, so a query like “what did I do yesterday” gets a dedicated time lane instead of forcing the semantic vector to guess.
• Graph relation expansion: using the top 5 cards recalled by the other channels as seeds, it steps one hop outward along the relation axis (1-hop graph walk), pulling back at most 30 neighbor nodes per expansion (with a hard 200ms timeout) — dragging out latent knowledge cards that are highly relevant to the current topic but share no surface wording (talking about JWT pulls in [OAuth 2.0]).

The architecture also has high-availability fault tolerance: if any one of the five channels dies (the HNSW vector index is rebuilding, or the CJK bigm module crashes), the other four immediately take over the vote. A built-in adaptive monitor keeps a sliding window of the last 10 latency samples per channel: once the window’s P95 exceeds 150 milliseconds, that channel is demoted and quarantined for 10 minutes; during quarantine, as soon as new samples bring P95 back under the threshold, the flag clears and the channel rejoins instantly — guaranteeing the frontend conversation never stalls.

A fun post-processing detail: Read-Your-Own-Writes (RYOW): high-concurrency systems have a classic bug. This turn you just said “I moved — my new address is Kaohsiung,” and a second later you ask “where do I live now?” Because that new memory is still in the backend’s asynchronous write queue (the Ingest queue) and hasn’t been written to PostgreSQL yet, the read side still fetches the old “Taipei,” and the AI gives the wrong, stale answer. fibon does not adopt heavyweight “cross-process database transaction locks” that tend to blow up across distributed nodes; instead it lightly adds a “same-turn overwrite cache” at the very top: after five-channel fusion retrieval finishes, the code checks whether the current Session has hot memories that just arrived from the frontend and are still stuck in the Ingest queue, not yet persisted; if found, it overwrites the stale retrieved address directly in memory. The lightest possible code, and the “memory time-lag” bug is gone.

Why Does This Massive Memory System Matter So Much to fibon?

Before recapping the goals one by one, here’s everything this chapter pulled apart, collapsed into one picture — the full chain a single turn runs through, from “question” to “answer” to background “distillation into cards,” and where each of the five LLM / embedding calls happens and what it does:

sequenceDiagram
  participant User as User
  participant GW as Gateway
  participant BR as Brain
  participant PG as PostgreSQL (card store)
  participant RD as Redis (Ingest queue)
  participant LLM as Cloud model

  User->>GW: Ask "what did I do yesterday?"
  GW->>BR: gRPC SubmitTask

  Note over BR,LLM: ① Read · embed the question
  BR->>LLM: Embedding(question)
  LLM-->>BR: query vector
  Note over BR,PG: Five-channel parallel recall<br/>(jieba→tags / bigm+trgm / time-range / graph / vector)
  BR->>PG: query cards (5 channels)
  PG-->>BR: candidate cards
  Note over BR: RRF fusion + decay/scope re-rank + RYOW overlay<br/>→ distill Top 5 cards
  Note over BR,LLM: ② Generate · inject Top 5 cards into System Prompt
  BR->>LLM: generate answer (Reasoning model)
  LLM-->>BR: answer
  BR-->>GW: stream answer
  GW-->>User: show answer

  Note over BR,RD: After answering · this turn goes to background Ingest (fire-and-forget)
  BR->>RD: push to ingest_stream
  RD->>BR: consumer pulls
  Note over BR,LLM: ③ Extract · turn→state/event cards, pin "yesterday" to an absolute date
  BR->>LLM: Ingest extraction (Haiku)
  LLM-->>BR: structured cards + occurred_at
  Note over BR,LLM: ④ embed card content
  BR->>LLM: Embedding(card)
  LLM-->>BR: card vector
  Note over BR,PG: conflict arbitration (cardinality → supersede-fork / INSERT)
  BR->>PG: write cards + tags + vectors
  Note over BR,LLM: ⑤ distill knowledge cards (Haiku)
  BR->>LLM: knowledge extraction
  LLM-->>BR: knowledge cards
  BR->>PG: write knowledge cards + related_to edges

In one line, where the tokens go: the read side spends only ① embedding + ② generation (② generation is the expensive output-token call); the write side spends ③ extraction + ④ embedding + ⑤ knowledge cards in the background, all fire-and-forget and never blocking your answer. The five-channel retrieval itself calls no LLM — jieba segmentation, bigm/trgm, time-range, and graph all run locally in PostgreSQL.

Back to the four goals set in Chapter 1:

Goal 2: precisely filter the information given to the LLM (see less, see more accurately). Supersede-Fork over time, cardinality isolation of facts, context-layer weighting, physical Project-scope isolation — all of it builds a filter mesh for the LLM in the backend: keep irrelevant old noise out, and ensure what enters the Context Window is always the 5 core cards that are rigorously computed, most precise, and true right now.

Goal 3: cut token cost (a design goal, still being validated). By construction it always hand-picks only the Top 5 high-density cards rather than stuffing in 50 long passages, and memories decayed to the floor or already superseded don’t even qualify to step into the prompt — the memory fed into a given turn is genuinely lean. But as the honesty note above said, once the background extraction cost and the various injected blocks are added to the full ledger, how much it nets out end to end is still unsettled, and I’m re-counting that ledger; once the knowledge cards’ “check the local store first, use it directly if not stale” path opens up, another chunk is expected to be saved.

The pain point from the opening of Chapter 1 — “the conversation UX feels off” — gets an answer in this chapter that at least looks feasible: cross-Session amnesia → cards persist permanently, precise recall even across ten thousand windows; scrolling forever to find old data → structured 3 × 6 multi-dimensional storage, precise dimensional slicing queries; the waste of “just ask again” → the knowledge-card vision will lean on a fast local cache to end redundant computation of generic knowledge.

I say “at least looks feasible” on purpose — because everything this chapter took apart is the backend foundation, and that foundation is, by this point, actually standing: the data structures, five-channel retrieval, conflict arbitration, and time anchoring are all running code, not boxes on a slide. But the next pit — possibly a bigger one — is the UI/UX: how to “grow” this multi-dimensional memory into an interface a user can actually see and explore. The way I picture presenting it, I haven’t seen anything quite like it on the market (or maybe I just haven’t looked hard enough), so there’s no trail blazed by anyone before to copy. Finishing the foundation only buys the entry ticket; turning it into an experience an ordinary person finds smooth, legible, and even a little bit cool — that’s the real fight of the next stretch.

An Honest List of the Problems Not Yet Solved

Honestly, midway through building this memory system I hit a giant blind spot — the multi-turn chemistry of the chain “retrieve memory → AI responds → user follows up based on the response → retrieve again” in long multi-turn conversations. The initial Smart Chat core main test round ran 71 single-turn tests (ask one question, answer it, score immediately) and produced an exhilarating +23.9pp capability lift (far ahead of the memoryless native control group). But however pretty single-turn tests look, they cannot verify whether memory warps as context stretches across a real 10-turn back-and-forth.

But the one that was counterintuitive enough that I re-read it three times is a different case —

Multi-turn testing is expensive (each run costs 5 to 10 times the tokens), and the test design itself is brutally hard — “what counts as the perfect multi-turn conversational behavior” has no standard answer anywhere in the world; you can only rule as rationally as engineering discipline allows. Separately, both ends of the knowledge-card chain from “The Third Card Type — Knowledge Cards” are finished (ADR-014’s schema, the Gateway API, the frontend UI, and the Brain-side “auto extraction from conversation” and “System Prompt injection” are all in the main branch), and the two flags KNOWLEDGE_EXTRACTION_ENABLED / KNOWLEDGE_INJECT_ENABLED now default to ON in the open-source launch build. The personal knowledge base is live; extraction-quality and cost validation now happens as it runs — observed continuously, ready to tune or roll back.

Implementation details

[ AgingBench longitudinal fix report ]
• Cards marked superseded: from the original "0" ──> up to "15"
• Revision recall:
  - Before fix: d0: 0.65 ──> d2: 0.00 ──> d4: 0.00 (total paralysis)
  - After fix:  d0: 0.65 ──> d2: 0.54 ──> d4: 0.41 (recall back in service)
• Interference aging rescued as a side effect:
  - d2 recall jumped from 0.14 all the way up to 0.64!

Next chapter, we charge head-on at the elephant in the room the AI agent world least wants to face — why, at the foundation, can you not blindly trust a single word the LLM says or a single promise it makes? It will fully reveal how fibon uses a cold-blooded “compliance audit architecture” to forcibly audit and confirm: the AI really did honestly complete every step of the manual (Skill) you gave it.