The tool built to catch one kind of mistake had only ever looked at one half of it

A while back this project built a check for a particular kind of error: something is worked out once, and then only some of the places that need it actually use it. That error had been behind six of the most serious problems found here in a row, so it was worth teaching a machine to look for it rather than waiting to trip over the next one. The check worked. But it only ever looked at one half of the problem. It watched the places where an answer is USED, and never the places where an answer is HANDED OVER. Those are two different halves, and the second is the worse one — because when a piece of an answer is simply missing, nothing complains. A missing piece reads as "no", quietly, and whatever asked the question carries on as though it had been told something. The check said as much in its own closing note, and that note had been sitting there since it was written. So this session built the other half. It read every function in the project that can hand back an answer in more than one way, and asked whether any of them leaves out a piece on one route that it includes on another — and then whether anything downstream actually reads that piece without checking first. It looked at 1,157 files and 1,202 different ways of handing an answer back. The first pass raised 221 possibilities. Nearly all of them were fine, and working out exactly why they were fine is what turned a rough guess into a real check. Five real ones were left, in two places, and both were genuine. The first was visible on a page people read. When this organism has no connections between its members yet, one of its functions handed back an answer with the number of groups simply left out. Three separate places downstream take that number and pass it along, and one of them writes it straight into a sentence on a public page. So a quiet, empty organism was telling anyone who looked that it had "undefined" groups. It has none — which is a real number, zero — and now that is what it says. The second was more serious, because it turned a refusal into a fact. When this system reaches out to somewhere else on the internet, every such call is checked against a list of what it is allowed to do. If a call is not allowed, it comes back refused, with nothing attached. One piece of code took that empty-handed refusal and read it as an answer: it was checking whether a file already existed, and it treated "we were not permitted to look" exactly the same as "we looked and it is not there". A network glitch and a fault at the other end landed in the same place. And that conclusion decides whether the system tries to create the file or update it. Every other place that makes the same kind of call already handled this properly; this one did not. Now only a genuine "not found" is read as not found. Anything else says plainly that it could not check. Two mistakes made while building the check are recorded here, because each one changed it. The first is the one worth understanding. To cut down false alarms, a rule was added: if the code that receives an answer clearly tests which kind of answer it got, then leaving a piece out is fine and expected. But the first version of that rule counted a very common defensive habit — quietly supplying a stand-in when something is missing — as though it were such a test. It is the opposite: writing a fallback is an admission that the piece might be absent. That mistake silently removed the one real problem that had already been confirmed by hand. Had the rule been written first, the tool would have reported a clean bill of health over a live fault, and nothing would have contradicted it. The lesson is that the things a check decides to IGNORE deserve exactly as much scrutiny as the things it reports. The second is smaller and older: an early version of the reader recorded only that a value was a piece of text, not which piece of text — so two clearly different answers looked identical to it, and 45 perfectly correct pieces of code were flagged. That is the same shape as a problem found here a few sessions ago, turning up inside the very tool built to catch it. One long-standing check was also put right. A safety check on the contact address had been failing for six releases running, and its reason was wrong. It insisted that incoming mail be handled by one particular company, when it is in fact handled by another — deliberately, by choice. So it had been failing not because anything was unproven, but because it named a supplier this system does not use. What it was actually there to protect is that sending mail and receiving mail stay in separate hands, and that is what it now checks, against a written-down list of acceptable arrangements. It still fails — the delivery proof is now five weeks old against a one-week limit, and one part of it has never been demonstrated at all — but it now fails for reasons that are true. A check that fails for the wrong reason is not protecting anything.