Re: Guest Post - Made Up Data Edition

1

Goddamn but social psychology needs to fix its shit.


Posted by: Sifu Tweety | Link to this comment | 04-29-13 6:57 AM
horizontal rule
2

Asocial psychology is worse.


Posted by: Moby Hick | Link to this comment | 04-29-13 6:59 AM
horizontal rule
3

This guy isn't even American so it probably doesn't matter.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:05 AM
horizontal rule
4

1: No, clearly this is a sign that all social sciences are fraudulent and unscientific. </my string theory colleague who has never worked with real-world data ever>


Posted by: essear | Link to this comment | 04-29-13 7:05 AM
horizontal rule
5

Leaving aside my nationalism, how do you have a Ph.D. dissertation with fake data and have the candidate not know anything? Getting to know that data better than anyone is supposed to be part of the dissertation process.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:07 AM
horizontal rule
6

4: If you don't have any data, you can't fake it.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:12 AM
horizontal rule
7

From the article:

"What about gender differences?" he asked Stapel, requesting to see the data. Stapel told him the data hadn't been entered into a computer yet. Vingerhoets was stumped. Stapel had shown him means and standard deviations and even a statistical index attesting to the reliability of the questionnaire, which would have seemed to require a computer to produce.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:16 AM
horizontal rule
8

Then, in the spring of 2010, a graduate student noticed anomalies in three experiments Stapel had run for him. When asked for the raw data, Stapel initially said he no longer had it.

This is like a different planet for two reasons.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:18 AM
horizontal rule
9

5: that doesn't usually mean checking to make sure it follows Benford's law and so on.

I'm kind of disappointed that Stapel didn't have a more sophisticated method for faking the data, though. Couldn't he have done it automatically?

Anyhow, related.


Posted by: Sifu Tweety | Link to this comment | 04-29-13 7:18 AM
horizontal rule
10

7,8: yeah, those examples are entirely insane.


Posted by: Sifu Tweety | Link to this comment | 04-29-13 7:20 AM
horizontal rule
11

Did he use regular M&M's or peanut ones? And did he remove all of the green M&M's so the subject's horniness didn't skew the results?


Posted by: Barry Freed | Link to this comment | 04-29-13 7:20 AM
horizontal rule
12

9: Right, but see 8. It's very strange to me that the graduate student gets the data all cooked from the professor.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:21 AM
horizontal rule
13

9.last "Dirk Smeesters" is such a great name.


Posted by: Barry Freed | Link to this comment | 04-29-13 7:23 AM
horizontal rule
14

12: it is indeed very strange.


Posted by: Sifu Tweety | Link to this comment | 04-29-13 7:24 AM
horizontal rule
15

I'm kind of disappointed that Stapel didn't have a more sophisticated method for faking the data, though.

I'd think you could do it using the procedures people use for bootstrapping or testing statistical procedures.


Posted by: Moby Hick | Link to this comment | 04-29-13 7:32 AM
horizontal rule
16

Best line: "I miss him, but there are equal amounts of instances when I want to punch him in the face."


Posted by: E. Messily | Link to this comment | 04-29-13 7:41 AM
horizontal rule
17

I would just take other public datasets and scramble them as necessary to produce the desired result, that way digit distributions and such should be natural.
Unless someone trying to fake data unwittingly discovers that someone else was publishing fake data.


Posted by: SP | Link to this comment | 04-29-13 7:42 AM
horizontal rule
18

I have some code that takes nosie and fits it to specified statistical parameters. That could totally work with a little tweaking!


Posted by: Sifu Tweety | Link to this comment | 04-29-13 7:45 AM
horizontal rule
19

I have some code that takes noise and fits it to specified statistical parameters
I believe this describes a significant fraction of our research.


Posted by: SP | Link to this comment | 04-29-13 7:48 AM
horizontal rule
20

If you take specified statistical parameters and fit them to noise, you should call it 'data mining.'


Posted by: Moby Hick | Link to this comment | 04-29-13 7:54 AM
horizontal rule
21

I have some code that takes noise and fits it to specified statistical parameters

"I call it 'Econometrics'."


Posted by: Ginger Yellow | Link to this comment | 04-29-13 7:54 AM
horizontal rule
22

And 20 just described the remaining fraction of our research.


Posted by: SP | Link to this comment | 04-29-13 7:57 AM
horizontal rule
23

Relevant.


Posted by: Sifu Tweety | Link to this comment | 04-29-13 8:46 AM
horizontal rule
24

Why do the kids today feel the need to bootstrap everything?


Posted by: Moby Hick | Link to this comment | 04-29-13 9:02 AM
horizontal rule
25

24 to the first comment of the article in 23.


Posted by: Moby Hick | Link to this comment | 04-29-13 9:03 AM
horizontal rule
26

... Seriously, this experiment sounds ridiculous to me, even before he threw the questionnaires away and ate all the M&Ms himself ...

Thing is lots of similar experiments have been published. The whole "stereotype threat" thing seems ridiculous to me which I suspect you all will not agree is sufficient reason to discount it.


Posted by: James B. Shearer | Link to this comment | 04-29-13 10:09 AM
horizontal rule
27

Most of those experiments sound pretty easy to run (given the ability to get human-subjects permission). There's an entertaining special issue in there: Catchy-Sounding Experiments Actually Performed, with a summary for each one that *didn't* turn out of what later published works are now on shaky foundations.


Posted by: clew | Link to this comment | 04-29-13 3:00 PM
horizontal rule
28

Best line: "I miss him, but there are equal amounts of instances when I want to punch him in the face."

See, this is why the Dutch will never be a great country & western nation.


Posted by: ajay | Link to this comment | 04-30-13 3:27 AM
horizontal rule
29

What happens when you play a Dutch country song backwards? Your tulips stop wilting, your windmill begins to work, and the hole in your dyke spontaneously closes.


Posted by: LizardBreath | Link to this comment | 04-30-13 3:33 AM
horizontal rule