How we score

Same rules for every capsule.

Every solid, half-there, ghost and hollow on this site is computed from published studies rather than decided by anyone here. This page is how, and just as importantly, where that method runs out.

What do the states mean?+

• Solid: well studied, and it works.
• Half-there: the science is still forming.
• A ghost: almost no real studies.
• Hollow: well studied, and it does not work.
• A cord: the pair itself has been tested together.
• Air: nobody has ever tested this pair.

What the four states mean

Every capsule on this site is drawn from its studies. The drawing is computed, never chosen, and the same rules run for a best seller and for something nobody has heard of.

Solid: Well studied, and it works. Several strong studies point the same way, and the ones that disagree are outweighed by the ones that do not.

Half-there: The science is still forming. Something has been tested, but not enough of it, or the results do not agree yet. Most of this site is half-there, because most of the science is.

A ghost: Almost no real studies. Nobody has properly looked. A ghost is not a verdict against the supplement. It means the question is open and the label is ahead of the evidence.

Hollow: Well studied, and it does not work. Enough good studies exist to answer the question, and the answer is no. A hollow capsule is the most expensive kind to earn, and we only draw one when the evidence can carry it.

Where the studies come from

We search PubMed, the public index of medical research, one supplement and one health outcome at a time, and we keep the search we ran so it can be checked. We prefer combined reviews, which gather many trials into one answer, and fall back to individual trials where no review exists. Today the site rests on 802 findings read from 649 papers, covering 362 claims across 82 supplements.

What we throw out

Most of what a search returns cannot answer the question, and saying so is part of the method. We drop a study when the thing tested was not the supplement people buy: a drug version of it, a different plant that happens to share a chemical, or a blend where the supplement never appears on its own. We drop animal studies. We drop a trial that gave the same supplement to both groups, because it never tested it. And when two reviews cover the same trials, we keep one, so the same evidence is not counted twice.

Never tested together

Pairs get one more state. A cord means the two were actually tested together in the same study. Air means nobody has ever tested the pair, and it is the most useful thing this site can tell you, because it is the thing no label will. We never claim air from a hunch: it requires a search we ran, logged and dated. Of 4,685 pairs screened so far, 4,226 came back as air and 158 hold a cord. Another 301 are held back because their search was too shallow to claim air honestly, and an unfinished search is not a finding.

What this method cannot tell you

Every method has a shape, and its shape decides what it can miss. These are ours. They are here because a reader who knows this field will spot them, and would be right to trust us less if we had left them out.

We weigh how a study was built, not how well it was run.+

A study counts for more when it is randomised, when it pools other trials, and when it is large. Nothing in our scoring asks whether it was run carefully, or whether the same handful of small trials keeps being gathered into new reviews. So a supplement tested many times in small, weak studies can read solid, while one tested twice in large careful studies may not. Where a verdict rests on many small studies rather than a few large ones, read solid as the published evidence points this way, not as the question is closed.

Studies that find nothing are less likely to be published.+

This is the oldest known problem in this field, and it tilts every pool of research towards good news before anyone reads a word of it. We push back on it in two ways: we go looking for studies that found no effect rather than waiting to trip over them, and when two reviews cover the same trials we never drop the one reporting no effect in favour of a newer, cheerier one. Neither of those recovers a study that was never published.

We read summaries, not the full papers.+

Every row on this site is read from a study’s published abstract, which is the summary the authors wrote themselves. It is enough to tell what was measured and what was found. It is not enough to catch a flaw buried in the methods section. Every row links to its source so you can go further than we did.

A number can be right and still be the wrong number.+

When we say a page rests on evidence from a certain number of trials, some of those trials may be counted twice, because two reviews under two different claims can gather some of the same studies. Treat it as the weight of research behind the page, not a headcount.

Checking, and how far it has got

The first 173 findings on this site were re-read from source and compared against what we had written down, by a second reader who could not see the first answer. That check corrected about one row in four. Almost every correction was a count written down wrong. Two were a finding filed under the wrong outcome, and both were changed to say the supplement showed nothing there, which is the cautious direction.

That is 173 of the 802 findings here today. The rest were read under the same rules the check tested, and by the same method, but they have not themselves been re-read. We would rather tell you which half is which than round the number up.

None of this is medical advice, and none of it is a safety assessment. The studies we read were testing whether a supplement works, not whether it is safe for you.