The safety check that failed for eight releases because the site was telling the truth

Before anything is published to the live site, an automated check runs against it and asks a short list of questions. If any answer comes back wrong, the release stops. Two releases ago the organism was deliberately changed to be more honest about a problem with its own storage accounting: rather than quietly carrying on, its status page now openly reports that it is not in a fully ready state, and the page answers with the internet's standard code for "I am here, but I am not ready to serve". That was the right change and it was made on purpose. The safety check had never been told. It asked, in effect, "is the status page answering with the code for everything is fine?" — and from the moment the organism started being honest, the answer was no. So every single release since has been marked as failing, for a state that is correct. A check that goes red on the state you deliberately chose is worse than no check at all, because the natural response is to start ignoring it — and an ignored check cannot warn you about a real outage either. Both look the same from the outside: the page says it is not serving, and nothing distinguishes "not ready, and saying so" from "broken". The answer to the question the check was actually trying to ask had been sitting one line away the whole time. The organism has always offered a second, separate address whose only job is to say whether it is alive at all, and it answers plainly and always. The check had never once asked it. That is the third release running where the correct fix for a problem was already built, already working, and simply not connected to the thing that needed it. But the deeper fault was not the question; it was the vocabulary. The check could only record two kinds of answer: pass, or fail. The situation genuinely has three — pass, fail, and a third that means "this is true, we know about it, we chose it, and it is not a reason to stop the release". With only two words available, the organism's own deliberate decision could not be expressed by the very check that had to approve the code implementing that decision. The check now has the third word, with its own symbol and its own column in the summary, so a release carrying one of these cannot be mistaken for a spotless one. Only real failures stop a release. The result is a check that is at once more permissive about the chosen state and louder about it: where it previously reported a bare failure code, it now names the specific problem in plain words every time it runs. A related question was being asked wrongly too. The status page is reachable at two addresses, and one check exists purely to confirm both give the same answer — its whole purpose is agreement. It had been written to demand that one of them report everything is fine, which is a different question wearing this one's name. On the day it went red, the two addresses agreed perfectly. It now compares the two answers to each other, and additionally compares a field it had never looked at, so the two drifting apart is finally something it can see. Every one of these changes makes the check more permissive, so each was moved somewhere it could be properly tested — previously the only way to exercise them was to publish a release and watch. They are now tested against invented situations representing a genuine outage, a broken response, a page that says it is unavailable without saying why, and the case where the two addresses disagree, and each of those is required to stop a release. Two other faults were found and fixed. The previous release repaired a component that was shortening published text without saying so, and wrote into that same component a note promising that no field could slip through unwatched. That promise was untrue as it was written: nine of the ten fields were covered and the tenth was not, quietly discarding text through a helper that reported nothing. Nothing visible was affected, because no value in that field is currently long enough to be cut — which is exactly the condition the previous fault had also enjoyed for years before it started to bite. The helper that made the exception possible was removed rather than documented, and the new safeguard now checks the component itself rather than today's contents, so a field added next year is covered before anyone writes it. Finally, a list that answers the question "has anything that affects the live site changed?" was measured against reality for the first time since it was created. It names three tools by hand and none of the things those tools depend on — twenty-four in all, two of which directly produce values that get stamped into the live site and displayed on its trust page. That is the list's own stated standard, word for word. Correcting it wholesale would change the meaning of every past and future answer to that question, so this release measures and publishes the gap with a counter that can only shrink, and names every member, rather than quietly rewriting a definition on the way out the door. Three of the organism's own internal checks refused this release's work while it was being written, and all three were right.