I worked for about a year in medical research after college in a lab run by a very notable medical researcher. Once some (foreign) post-docs spent several weeks re-running a cell-culture experiment to get it to agree with a finding in Nature that our lab had published. Each time they came back to group meeting with a p > .05 result, the PI would go on for about a minute about how this was "problematic". After several tries, they got the result the PI was looking for and everyone breathed a sigh of relief. This didn't seem to trouble anybody.
Maybe I don't want to go to graduate school after all.
Ted Miguel *is* 1... 2... 3... 4... 5... 6... 7... 8... 9... 10... 11... 12... 13... offices down the hall. So be warned!
**BUT NEVERTHELESS:**
My take on this is the same as the very sharp and very hard working Chris Blattman's: http://chrisblattman.com/2015/07/23/dear-journalists-and-policymakers-what-you-need-to-know-about-the-worm-wars/ http://chrisblattman.com/2015/07/24/the-10-things-i-learned-in-the-trenches-of-the-worm-wars/
>One of the things I love about the Internet is that it brought a lot of very smart people to a key intellectual problem, the discussion brought out the central assumptions and claims, and they were answered within about a day or two. See Berk Ozler, for example. My conclusion is that the Kenya deworming results are relatively robust. Yay hive mind.
>On the other hand, the hive mind has a tendency to be grumpy, rude, shrill and angry. I found the debate more dignified than some, but vicious at times.
>I am guilty as well. I was too quick to suspect and insinuate bad faith on the part of the replicators. I can hold suspicions but I shouldn't publicly insinuate or accuse without grounds. I am sorry for that.
>I do find any big, coordinated media push of a scientific finding to be problematic, to say the least. This drove and drives my suspicion of bias, even if accidental.
>To me the big failure in this entire business was by the editor of the academic journal. The competing claims on whether or not the results are fragile or not, and why, should never have been allowed to remain ambiguous.
>To me, a glitzy media push undermined the credibility and intentions of the journal further. This is a general problem in medicine and hard science that I do not see as much in social science (where the journals could care less about news coverage).
>On the journalist side, I can't blame any of the writers for not following the finer statistical points. I had trouble myself. But almost none of the journalists read the reply by Miguel and Kremer (published by the same journal) and maybe none called Miguel or Kremer on the phone. I am told I was the first. Tell me if I'm wrong, but isn't this the definition of sloppy journalism?
>I think the deworm the world movement has also tended to exaggerate or selectively quote the evidence in order to justify the cause. GiveWell has a much more balanced account: the evidence is not that great, but it's good enough. Sort of. I thought GiveWell had one of the best posts. Do read it. http://blog.givewell.org/2015/07/24/new-deworming-reanalyses-and-cochrane-review/
>To me, the real tragedy is that, 18 years after the Kenya deworming experiment (which was not even a real experiment) we do not have large-scale, randomized, multi-country, long-term evidence on the health, education, and labor market impacts of deworming medicine. This is not some schmuck cause. This is touted as one of the most promising development interventions in human history.
>I also fear for the reputation of replications in development economics. I imagine some of the problems could be addressed by getting more clarity into pre-analysis plans for replications. But the incentives to make mountains out of molehills is huge. Maybe everyone should sign a "no glitzy media push" pledge.
The whole thing made me cynical about research in every way possible. I was getting paid less than $30k a year to prop up a feckless post-doc's resume. Person I worked for was on some *really* important boards and actually had his entire salary paid by a pharmaceutical company. This kind of thing was pretty common. I got to sit in on a meeting where he and other med school bigwigs were trying to decide whether or not to advise a large company to move into the next phase of a clinical trial, though, so that was pretty cool. At some point (probably when I asked for a raise and was stonewalled by administration) I had the realization that I was effectively being paid a pauper's wages to do R&D for the pharmaceutical industry and that I was supposed to do this all with a smile for the love of science or something. I can't speak for the rest of academia, but I'm having a hard time convincing myself that the whole thing--grad students, post-docs, technicians and all--isn't just a ploy to keep R&D costs for the pharmaceutical industry as low as possible. Everything the lab did was geared towards the creation of new drugs. A lot of it was funded by pharmaceutical companies, but I'm not sure that support wasn't just a way for them to write their costs off when they did their taxes.
A decade later, in 2013, these two economists did something that very few researchers have ever done. They handed over their entire dataset to independent researchers on the other side of the world, so that their analyses could be checked in public.
What? The handing over the code and all is pretty rare, but putting up the data (not right away always) is the norm.
I found the second link in 3 to be an effective rebuttal of many of the points.
I had the realization that I was effectively being paid a pauper's wages to do R&D for the pharmaceutical industry...
If you go to graduate school, you can sometimes do it for middle class or better wages.
I guess it's progress of a sort that replication studies now seem to be considered high-status enough to involve overhyped results and coordinated media campaigns.
7: I'd always assumed Hick was a pseud. (Kidding.)
Aiken et al.'s supplementary material and Hicks, Miguel, and Kremer's response to the 3ie replication working paper clarifies this explanation.
4: It gets worse. In all honesty, though, healthy cynicism is probably way better than wide-eyed naïveté for your mental health if you did go to grad school/stay in pharma.
What fields aren't better if you have healthy cynicism?
Anyway, if you get a result saying that having worms improves your education, I bet you check the code more.
Clueless question - the studies are all about the future impact of large deworming programs but I don't see any mention of how effective the deworming was for individuals. Were the worms eliminated from the treated people? If so, isn't that a sufficient good in itself?
15: I think the question people are trying to answer is whether it's worth pouring money in as a specific cause to be elevated above all the other needed health care services.
Unless someone like you cares a whole awful lot, the worms will all die.
15 Hell yes. When I lived in Morocco I used to deworm every 4-6 months and twice after returning stateside. This after hearing of a friend who pulled a 2-footer out of her toddlers butt. Her maid on seeing this occurrence agreed to take some vermicide herself and subsequently evacuated what sounded like several lbs of body weight in worms. And even without that just reading about this horror was enough.
I'm not going to click on that link. The comment was enough.
Right, it's lunch hour some places in the world, dude.
And now that I think of it isn't Arrakis known for its worms?
In Soviet Arrakis, worm shits you.
I'm always leery of science journalism that doesn't mention interminable conference calls.