Search for a command to run...
For a long time, Panacea Index had a bug that was worse than being wrong. It was being right in two different ways at once.
You would run a search and see a preview: this supplement and this outcome, fourteen studies, a clean directional bar. You would click through to the full page and the header would say something else — twenty-two studies, maybe, or a different balance of increase and decrease. Same supplement. Same outcome. Same database. Two numbers, and no way for a user to know which one to believe.
The worst part: neither number was a straightforward error. Each was the correct answer to a slightly different question, asked along a slightly different path through the data. This is the story of how two correct paths diverged, why that is the most dangerous kind of bug, and the fix that finally pinned them together.
To understand the disagreement, you have to understand that our data is queried from two directions.
One direction is precomputed. For every supplement-and-outcome pair, we aggregate all the underlying claims ahead of time into a consensus record — how many studies found an increase, how many a decrease, how many no effect, and the net direction. This is fast. It is what the search preview reads when it shows you a count next to a result.
The other direction is live. When you land on the consensus page, we fetch the actual claims for that pair and derive the counts again, right there, from the raw rows. This is what the page header shows.
Both are reasonable things to do. Precompute for speed, recompute for freshness. The bug is that they have to agree, and for months they did not.
The fork happens at a layer most users never see: normalization.
A claim in our database names a supplement and an outcome in free text. "Creatine monohydrate" and "creatine supplementation" are the same supplement. "Grip strength" and "muscular strength" are close enough to belong together. Before you can count anything, you have to fold these variants into common buckets. We do that in two stages. First, every variant string is canonicalized into a normalized entity. Second — and this is the part that matters — those normalized entities are linked into a smaller set of shared concept buckets we call signal keywords, so that a query for one concept pulls in every variant that belongs to it.
Here is the fork. The precomputed consensus record is built by expanding each claim through its signal-keyword links. The live page, in its older form, resolved a supplement name to a normalized entity and queried from there. When the signal-keyword expansion and the name-to-entity resolution landed on slightly different sets of underlying rows — because of an entity that was linked on one path and not the other, or a fallback that behaved differently — the two counts diverged.
Neither path was broken in isolation. The precompute was a correct expansion. The live query was a correct resolution. They were just correct about different things, and nothing enforced that "different things" had to be the same thing.
A bug that crashes is a good bug. You find it immediately. A bug that returns a wrong number is worse, but you usually notice when the number is obviously impossible.
A bug where two correct-looking numbers disagree is the worst. The preview looks right. The page looks right. They are both internally consistent. The only way to notice is to compare them, and the only person who would compare them is the user who clicked through and felt something was off — exactly the user you most want to trust you. Every disagreement was a small, quiet erosion of credibility, and there was no alarm.
It went on for months because each side, examined alone, passed.
We tried, over four iterations, to make the two paths compute the same thing. Each pass closed some of the gap and left a remainder, because we were trying to make two independent pieces of logic agree after the fact. That is fragile. It is asking two functions to stay in sync by hope.
The fix that actually worked changed the contract between the paths rather than the logic within them.
When the search preview shows a result, the link it generates no longer carries just a supplement name and an outcome name. It carries the exact bucket identifiers the preview used — the specific signal-keyword pair that produced the count you saw. When you click through, the page detects those identifiers and takes a pinned path: it asks for exactly that bucket, using a query that expands through the same signal-keyword links, with the same fallback rules, as the precompute. The name resolution path is still there for the case where someone arrives without the identifiers, but the primary flow now guarantees that the number you saw is the number you get.
In other words, we stopped asking the page to re-derive the answer. We made the link remember the answer it already showed you, and made the page honor it.
The precompute and the live page now share an expansion function — literally the same SQL semantics, in the query that builds consensus and the query that reads it. That shared function is the single source of truth. The two paths agree because they are no longer two paths. They are one path, parameterized two ways.
There is a general principle hiding in here that I have tried to internalize.
When two components must agree, do not coordinate them by making each one correct in isolation and hoping they converge. Identify the shared truth they are both describing, put it in one place, and make both components read from that place. Agreement enforced by shared definition is durable. Agreement enforced by parallel correctness is not.
Our counts agree now. They agree because we stopped trusting them to agree, and made them unable not to. That is a much better foundation to build on than hope.
