Research Desk
How to read this stuff without getting fooled
Every rating on this site comes from one hierarchy. Levels do not accumulate: a hundred Level 2 studies never add up to Level 4. Understanding that single point will make you harder to sell to than most people in this space.
The evidence hierarchy
Level 5 — Practice-changing
- Replicated large randomised controlled trials, pivotal Phase 3 programmes, or a major regulatory decision.
- This is the level at which it is reasonable to say something 'works' — within the populations studied.
Level 4 — Strong human evidence
- Strong controlled human evidence: well-run randomised trials that are not yet replicated or definitive.
- Enough to take seriously, not enough to treat as settled.
Level 3 — Preliminary human signal
- Preliminary human signal: small, short, open-label or uncontrolled studies.
- Useful for deciding what to test next. Not useful for confident claims.
Level 2 — Preclinical
- Preclinical: animal models, organoids, tissue studies.
- Most peptide enthusiasm lives here. Mice are not tiny humans, and healing a surgically induced rat tendon is not the same problem as your shoulder.
Level 1 — Mechanistic
- Mechanistic: cell culture, receptor binding, computational modelling.
- Explains how something could work. Says nothing about whether it does.
Level 0 — Anecdote
- Anecdote: testimonials, forum reports, influencer experience, 'my mate swears by it'.
- Genuinely interesting as a hypothesis generator. Never evidence of effect.
Research literacy, in eight parts
Sample size
Twelve people is a pilot, not a proof. Small trials produce extreme results in both directions purely by chance, and the extreme ones are the ones that get shared.
Replication
One dramatic finding is a hypothesis. When independent groups repeat it and get the same answer, it becomes knowledge. Ask who else has found this — and whether they were connected to the original authors.
Surrogate endpoints
Raising IGF-1, shifting a blood marker or shrinking something on a scan is a stand-in for the thing you care about. Surrogates sometimes move in the right direction while real outcomes don't move at all.
Control groups and blinding
Pain, fatigue and recovery respond powerfully to expectation and to time. Without a control group, you cannot tell a treatment effect from healing that was going to happen anyway.
Conflicts of interest
Industry funding does not automatically invalidate a study — most drug research is industry funded. But it changes how much benefit of the doubt is reasonable, especially with selective reporting.
Why headlines overstate
'Reduces risk by 50%' can mean a change from 2 in 1000 to 1 in 1000. Relative numbers sound enormous; absolute numbers are what you actually experience.
Preprints and press releases
A press release is marketing with citations. A preprint has not been peer reviewed. Neither is worthless, and neither is a finished result.
Publication bias
Studies showing nothing get published less often. So the visible literature on any hyped compound is systematically rosier than the research actually done.
The question that cuts through almost everything