Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why not? I think it was Jeff Bezos who said “When data and anecdotes disagree, it’s usually the anecdote that’s right. Something is wrong with the way you’re measuring your data.”

If there’s a debate at all then the data are probably ambiguous or inconclusive, so let’s talk real human experience. Data says LSD is the best thing ever, and here’s one guy who has a terrible experience - is he wrong?



Jeff Bezos in that quote is talking about customer complaints and how they relate to SLOs.

When a customer is complaining about e.g. "your site being broken", there's usually some real problem they're complaining about (though whether it's your problem to solve, or their ISP, or their computer, or their lack of knowledge of how web browsers work, etc. is another question entirely.) But the point being, if your data says the customer can't possibly be experiencing an issue — i.e. if your data disagrees with the customer's own lived experience of having a problem at all, with the data saying that e.g. the customer made a successful purchase, when the customer says they couldn't even load the site — then that should suggest that your tools for measuring your data are broken, or that there's something else equally-fishy going on (like a Man-in-the-Middle.)

None of this applies to medicine/psychology, because medicine never has the sort of data that could even theoretically be used to make a claim like "this is 100% working, and anyone who says they have a problem is lying" — the sort of claim where even a single counterexample would be enough to refute that statement, and therefore where a single counterexample would be valuable.

Rather, the sort of claims made in medicine are Bayesian confidence claims. The sort of (evidenced) claim that gets a treatment approved by the FDA, goes something like: "treatment X tends to be well-tolerated in population Y, while producing a positive outcome of power Z with benefits outweighing the measured side-effects."

No single anecdote (= clinical data) refutes that kind of statement. Instead, you need to compile and quantify a bunch of them (= clinical data meta-analysis) to actually make an argument for or against that claim.

Knowing this, any attempt someone might make to wield a single anecdotal claim to influence the credence you give a statistically-derived Bayesian-confidence statement of the safety and efficacy of a medical treatment — especially where you don't have an intuitive sense for how much data went into the statistics that led to the original statistical claim — should be regarded as an attempt to manipulate you with rhetoric, rather than honest debate praxis.

Which is not to say that the anecdote is false! You can totally believe that the person's lived experience is real, and empathize with them, and try to come up with solutions for their problem; while also taking as hokum any attempt by them to convince you that their anecdote generalizes.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: