A field note from the AIdeazz AI Lab — a real incident on a live production system, written up from the logs. November 3, 2026.
A family-language companion had chat history, semantic search over 588 stored conversation turns and a memory reader wired into every reply, and still forgot the people it talked to. The search cut-off had been tuned for a different embedding model, so real matches were thrown away, and nothing ever wrote down a fact about a person.
What it looked like from outside
The owner had asked many times for the assistant to remember families, and it did not. The code looked complete. It kept the last five turns of each conversation in a database through LangChain, it embedded every turn for semantic search (retrieval-augmented generation, RAG), and a reader put "key memories about this user" into every reply. Production said otherwise. The table of facts about users held 13 rows after a year, 7 of them test entries. The RAG table held 588 embedded turns, yet the reply builder received nothing from it. An end-to-end test with a made-up family through the real code on the production database showed the break exactly: after the family said their son starts at a new school on Monday, the later message "today was my son's first day of school" retrieved no earlier turn at all.
What was actually happening
Two separate gaps, both silent. First, the semantic search kept only matches above a cosine similarity of 0.55. That number had been carried over when the embeddings moved from a paid API model to a free local multilingual model, whose scores for "the same topic" sit much lower. Measured on the test family, the related school turn scored 0.397, while unrelated turns scored 0.26 to 0.37. With the cut at 0.55 the search returned an empty string, which the reply builder treats as "nothing relevant", so no error and no log line ever appeared. Short bot replies such as "you're welcome" also scored as high as real content, crowding out what the person had said. Second, the fact memory had a reader and no writer: the only code that stored a fact about a user ran during an onboarding form almost nobody finished. Conversation history and semantic search remember sentences; neither one turns "my son Mateo is 8 and starts school Monday" into a fact the assistant can bring up a week later.
The fix
The cut-off was set from measurement, not memory: 0.38, which keeps the related turn and drops the unrelated ones. Search now recalls what the person said rather than the assistant's own replies, skips turns already present in the five-turn history window so nothing is sent twice, and logs a line each time it recalls something. A missing writer was added inside the existing memory module: after each grown-up message a free model notes up to three lasting facts (family members and ages, city, school, upcoming appointments with their day), in a background thread so replies are not slowed, with duplicates skipped and identification numbers, phone numbers, addresses and bank details never stored. The reader was widened from five facts to ten. Both chat channels share the same memory code, so one change fixed both.
How I know it worked
Production database counts: 13 user facts (7 tests) and 588 local embeddings before the fix. The end-to-end test ran the real memory code against the production database with a made-up family and deleted every one of its rows afterwards. Before the change: the school message recalled nothing at the 0.55 cut, with measured scores of 0.397 for the related turn and 0.26 to 0.37 for unrelated ones. After: the same message recalled the school turn at 40%, "I got my residence card" recalled the immigration appointment at 72%, "how do I say outlet again?" recalled the earlier question about the Spanish word at 41%, and the fact writer stored three facts including the appointment, while a pure lesson question produced none and a phone number in a test message was skipped. Both services were redeployed with file checksums equal to the committed code and no errors after restart. Real families had not yet written since the deploy at the time of writing, so the live path is proven by the end-to-end test, not yet by traffic.
The rule this earned
A threshold belongs to the model that produced the scores. When the model changes, measure the scores again before trusting the cut, and test memory end to end with a known conversation, because an empty result looks exactly like "nothing relevant".
The named concepts behind it
Naming a failure mode is what makes it possible to recognise the same shape somewhere new, before it costs another weekend.
Calibration drift
A number tuned for one model, one vendor or one data source stays in the code after the thing it was tuned for is replaced, and quietly stops meaning what it meant.
Many AI systems decide with a number: keep search results above a similarity of 0.55, alert when a classifier is more than 80% sure, flag a transcript below a confidence of -1.0. Each number was chosen by looking at the scores one particular model produced.
The failure is replacing the model and keeping the number. Different embedding models, speech models and classifiers spread their scores differently: one model's "clearly the same topic" may be another's 0.4. Nothing crashes. The cut simply starts throwing away everything, or letting everything through, and because an empty search result and a "nothing relevant" result look identical, the system reports success while doing nothing.
The fix is to treat a threshold as part of the model, not part of the code: when the model changes, run a small known set (one pair that should match, a few that should not) through the new model, read the actual scores, and set the cut between them. Then keep that set as a test, so the next swap re-measures automatically.
Ordinary-life version: a bathroom scale moved from a soft carpet to a tile floor. The "healthy weight" line you drew on it still sits in the same place, but the readings all shifted, so the line now tells you nothing.
Silent failure
The system did something reasonable, and told nobody.
The most expensive bug class there is, because the clock keeps running while everyone assumes things are fine.
A silent failure is not a crash. A crash is loud and gets fixed. A silent failure is a component making a defensible local decision -- drop this message, skip this record, return an empty string -- that nobody downstream is told about. From the outside, a system that is working perfectly and a system that is completely dead can produce the identical observation: nothing happened.
The defence is not "add more logging". It is to make the healthy state provable, so that "nothing happened" can be distinguished from "nothing was supposed to happen". Two things do that:
- Log the outcome, not the attempt. "sending notification" tells you nothing. "notification DELIVERED (id 4661)" versus "notification REJECTED 400" tells you everything.
- Run a canary. A synthetic transaction pushed through the real path on a schedule, which shouts when it does not come out the far end. Without one, you are relying on a customer to report your outage.
Verify from logs, not config
Configuration tells you what somebody intended. Logs tell you what happened.
A setting, an environment variable or a present API key is a statement of intent. It is evidence that somebody meant for a behaviour to occur. It is not evidence that the behaviour occurs.
The gap between the two is where the longest outages live, because reading the configuration feels like verification. It produces confident, wrong statements: the key is set, so the provider works; the schedule says every fifteen minutes, so it runs every fifteen minutes; the file was deployed, so the new code is running.
Each of those has a cheap, decisive check that costs seconds:
- Probe the dependency, do not read its credential. A key that exists proves nothing about the balance behind it.
- Grep for the action line, not the setup line. A startup banner proves the process started, not that it ever did its work.
- Compare timestamps after a deploy. If the running process is older than the file on disk, it is still executing the previous version from memory.
The rule this earns: never report a system's behaviour from its configuration. Grep the line that proves the behaviour happened, and quote it.
Liveness is not correctness
A dead job announces itself. A job that runs perfectly and emits slightly wrong output never will.
Almost every check you own measures liveness: did it run, did it return, did it exit zero, did it publish. Almost none measures correctness: was the thing it produced the right thing. These are different properties, and the gap between them is where the expensive incidents live.
The asymmetry is what makes this dangerous. A job that stops firing is loud — the output is missing, someone notices within a day. A job that fires on schedule and produces output that is subtly wrong is silent, and it stays silent for as long as nobody reads the output, because every signal you have is reporting the truth. The scheduler really did fire. The API really did return 200. The file really was written. Each check passes honestly while the only thing that matters fails.
Two shapes to watch for:
1. The safety net that adjusts instead of refusing. A guard catches a bad condition, then modifies the input so the operation can proceed — renaming a colliding key, truncating an over-long field, coercing a bad type. The error disappears from the logs and the bad condition ships anyway. A guard that never refuses is not a control; it is a laundering step, converting a real signal into a clean log line. Prefer failing closed: a skipped run is cheap and visible, a wrong run is expensive and invisible.
2. The record that drifts from the reality. Any check that compares against a cache, a state file, or a local ledger is only as good as that memory. When the memory can be truncated by a restart, a fresh machine, or a path that writes to one place and reads from another, the check degrades quietly and keeps returning "fine". Seed the memory from the artifacts themselves wherever you can, and periodically assert that the two still agree.
The practical defence is to add one check that reads the output rather than the exit code, and to make it something a human would actually notice — a count that should be stable, a uniqueness constraint, a spot comparison against what shipped last time. You are not trying to verify everything. You are trying to have at least one signal that fails when the job succeeds incorrectly.
---
This note is one entry in a running wiki of production engineering lessons — every concept linked to the incident that taught it — at aideazz.xyz/ai-ops-wiki.html.
No customer data, credentials, hostnames or internal record identifiers appear in these write-ups.