There's a big difference between the (apt) metaphor of a model you describe and this case.
To extend your model, you're conflating direct data collection/experimentation with literature assessment, and essentially saying that's acceptable. It's not.
A journalist interviewing an expert (academic, or other specialist) is analogous to the literature review in a research context: you do not have direct observation, but you are assuming that peer review and other mechanisms have ensured the literature is grounded in direct observation.
A journalist recounting first-hand experiences or recounting those of people he/she has interviewed is really doing direct data collection or experimentation: the reason you try to get multiple sources for a significant claim is precisely why scientists run experiments multiple times.
A journalist claiming second-hand experiences as first-hand is equivalent to claiming to have observational data when in actuality you are relaying the conclusion of another experiment.
Take this neuro-toxin example: it happened to N people, but by claiming to meet them personally, he's implying it happens more often but is covered up, which would mean the real prevalence is higher (when in reality, we have no evidence that they are).
Or take the suicide rates. By claiming concentration via personal experience, the claim is contrary to what we know through more rigorous data collection, which is that the rates firm-wide are lower than for the Chinese population in general.
Also, the problem of your metaphor is you presume that the model f(observed facts, gut instinct, emotion) yields more valid inference than the sum of multiple journalists' f(observed facts). This is crap. If f(gut instinct, emotion) skew systematically -- a more reasonable conclusion than the counterfactual -- then no, your approach is less valid.
Nassim Taleb had a nice bit about how he doesn't read the news; instead he reads textbooks or other long-production-cycle reports. This is because, he says, the frontier of knowledge doesn't change very often. So what you get in the news is a fundamentally biased sample of only the outliers (which sells papers) and never the consensus that is generally 99% of the body of knowledge.
Oh, sure, I wasn't referrimg to this case but to tptacek's remark about narrative journalism and the article on Truman Capote he linked to.
And just to make sure: I'm not claiming that a longer, more careful and more rigorous observation does not yield a more accurate approximation. I'm just responding to a common complaint against journalists that if only they would stick to what they've actually observed and can prove that would necessarily produce better journalism; this is wrong.
And as for Nassim Taleb not reading the news, only textbooks - well, I don't have a problem with the first part, but if the obsessive, anal Mr. Taleb insists on only the most rigorous sources for learning about this world, than he has a very limited soul, indeed.
Hmmm :), it's a long story, but I can tell you the gist of it. My bachelor's degree is in math and computer science, but after 9 years in professional software development, I came to realize that it wasn't fulfilling enough. I went back to university and studied medieval history (which proved to be a tremendous help in my job as a journalist) and started working for a small web magazine. After a year of doing that, I began pestering a lot of people, walking the streets of Tel-Aviv with hard copies of my stories that I gave out to any big-name journalist that I happened to bounce into, and eventually I was lucky to get an interview with a magazine editor who hired me as a fact-checker; things started rolling quickly from there. Alongside that I worked on a big personal software project, my reporting sustaining me just barely, and not taxing the math circuits of my brains too much - I needed those for my software. After four years at the magazine, with my software project nearly finished, I realized it was time to either turn it into a business or let it remain an interesting algorithms research. I chose the former, and since January have been putting the finishing touches. I've applied to YC, I'll be launching my startup later this week with an announcement here on HN, and will be releasing a big, and I think very interesting, open-source product (an important component of the main software) later in April. I hope to go back to writing eventually, if only part-time.
Actually, I've learned a lot of interesting things about the startup culture (specifically that of Silicon Valley) from reading HN comments (seriously, it's fascinating). There's probably a story there, one that hasn't been told yet.
To extend your model, you're conflating direct data collection/experimentation with literature assessment, and essentially saying that's acceptable. It's not.
A journalist interviewing an expert (academic, or other specialist) is analogous to the literature review in a research context: you do not have direct observation, but you are assuming that peer review and other mechanisms have ensured the literature is grounded in direct observation.
A journalist recounting first-hand experiences or recounting those of people he/she has interviewed is really doing direct data collection or experimentation: the reason you try to get multiple sources for a significant claim is precisely why scientists run experiments multiple times.
A journalist claiming second-hand experiences as first-hand is equivalent to claiming to have observational data when in actuality you are relaying the conclusion of another experiment.
Take this neuro-toxin example: it happened to N people, but by claiming to meet them personally, he's implying it happens more often but is covered up, which would mean the real prevalence is higher (when in reality, we have no evidence that they are).
Or take the suicide rates. By claiming concentration via personal experience, the claim is contrary to what we know through more rigorous data collection, which is that the rates firm-wide are lower than for the Chinese population in general.
Also, the problem of your metaphor is you presume that the model f(observed facts, gut instinct, emotion) yields more valid inference than the sum of multiple journalists' f(observed facts). This is crap. If f(gut instinct, emotion) skew systematically -- a more reasonable conclusion than the counterfactual -- then no, your approach is less valid.
Nassim Taleb had a nice bit about how he doesn't read the news; instead he reads textbooks or other long-production-cycle reports. This is because, he says, the frontier of knowledge doesn't change very often. So what you get in the news is a fundamentally biased sample of only the outliers (which sells papers) and never the consensus that is generally 99% of the body of knowledge.
TL;DR: http://www.youtube.com/watch?v=VyFpOp8Ft0Q