## Re: Guest Post - Puzzle

1

I'm one of the outlaw 9%!

Posted by: Natilo Paennim | Link to this comment | 07- 5-15 8:51 AM
2

Remember, no finite selection determines a rule.

Posted by: nosflow | Link to this comment | 07- 5-15 8:56 AM
3

I got it.

Posted by: Moby Hick | Link to this comment | 07- 5-15 9:01 AM
4

I got it. And remembered 2. And checked the (spoiler?) boundary case.

Posted by: Awl | Link to this comment | 07- 5-15 9:08 AM
5

Are we doing spoiler warnings? The below assumes you've gone through the puzzle and know the anser.

Is there a unique boundary case? I came within a finger-twitch of "failing" the quiz and then my eye caught the "be sure to check your answers" warning and decided to try some counterexamples. I was partly pulled in the wrong direction by feeling like I needed to figure it out in as few guesses as possible, though nothing in the setup indicated you got extra points or anything like that.

It reminds me of the old "Step 20 is ignore steps 1 through 19" exercise.

Posted by: Yawnoc | Link to this comment | 07- 5-15 9:17 AM
6

I tried lots of what I thought would be counterexamples but they all worked. That's how I worked out the answer. Only got one No.

Posted by: ajay | Link to this comment | 07- 5-15 9:25 AM
7

I didn't get any no's and didn't get the answer. As with so many of these puzzles, framing is everything. Or maybe I'm just not very smart.

Posted by: ogged | Link to this comment | 07- 5-15 9:33 AM
8

Or both.

Posted by: Moby Hick | Link to this comment | 07- 5-15 9:35 AM
9

Me and ogged.

Posted by: chris y | Link to this comment | 07- 5-15 9:37 AM
10

4 yes, 4 no, right answer.

Posted by: Unfoggetarian: "Pause endlessly, then go in." (9) | Link to this comment | 07- 5-15 9:38 AM
11

SPOILER

See, their basic discussion of confirmation bias rings completely true to me. I've been guilty of it plenty of times (e.g. recently on this blog, when I reposted the fake Goffman Twitter link).

But there seems to me to be an ENORMOUS difference between being more likely to believe a news story that fits your preexisting biases, and failing to ask questions about your own idea or theory. That's where their claims just seem absurd to me. E.g.:

This puzzle exposes a particular kind of confirmation bias that bedevils companies, governments and people every day: the internal yes-man (and yes-woman) tendency. We're much more likely to think about positive situations than negative ones, about why something might go right than wrong and about questions to which the answer is yes, not no.
Sometimes, the reluctance to think negatively has nothing to do with political views or with a conscious fear of being told no. Often, people never even think about asking questions that would produce a negative answer when trying to solve a problem -- like this one. They instead restrict the universe of possible questions to those that might potentially yield a "yes."

This just doesn't remotely resemble many workplace situations I've been in. Sure, I can think of some cases where the above occurred -- but it was very rarely a widespread phenomenon. Far, far more common is the "We have a newborn idea, everybody poke holes in it" phenomenon.

I dunno. I just feel as if the NYT is wildly caricaturing in order to make a (doubtful) point.

Or maybe I really AM that much of a critical grouch/outlier.

Posted by: Witt | Link to this comment | 07- 5-15 9:43 AM
12

It doesn't quite ring true to me, either. People love to be naysayers about other people's ideas. The idea-haver is rarely as critical of their own idea, or they do agree and drop the idea, but they're certainly not sheltered from criticisms.

Posted by: heebie-geebie | Link to this comment | 07- 5-15 9:48 AM
13

That analysis has an explains too much from too little feel to it. I wonder how much could be explained just by people not understanding how sequences work. I got almost as many no as yes results since I put in a few tests to see if order mattered.

Posted by: fake accent | Link to this comment | 07- 5-15 9:51 AM
14

Also, the Bush administration did test the theory of whether the US would be greeted as liberators. It would have been better left as a conjecture.

Posted by: fake accent | Link to this comment | 07- 5-15 9:56 AM
15

I tested n^1, n^2, n^3 first off and was so pleased with myself I submitted in one. Easily fooled!

Posted by: k-sky | Link to this comment | 07- 5-15 9:59 AM
16

Got bored after a few yeses and clicked "just tell me the answer". I probably would have prematurely answered and gotten it wrong, because I'm generally bad at these kinds of things.

Posted by: AcademicLurker | Link to this comment | 07- 5-15 10:00 AM
17

I came up with the answer they're leading you to get, and then got moderately lucky and tested it against 0-0-0.

Posted by: Nathan Williams | Link to this comment | 07- 5-15 10:29 AM
18

I think I know what they were trying to show, Witt; let's see if I can say it better... Starting with the guess "P makes A into B", many people are much better at constructing complicated cases of A and checking whether they get B than at taking P out of the system and seeing if they get B anyway. And often we even forget to try a lot of not-A to see if *that* turns into B, which isn't relevant for strict modus ponens testing but is often important to the application. These must have formal names but I've forgotten them.

There are famous embarassing cases in geomorphology and other earth sciences, and it's a basis of the software testing that I'm told CEE doesn't teach, so if your office always covers all the parts, go you!

Posted by: clew | Link to this comment | 07- 5-15 10:34 AM
19

clew is right, it's a known bug in human cognition. (except, interestingly, in scenarios involving cheating, where we are much better at verifying or disproving theories. )

Posted by: ajay | Link to this comment | 07- 5-15 10:39 AM
20

I think the phenomenon tested here is less, "Hearing no is not fun," and more, "Guessing right on flimsy evidence feels good." Like if you're playing 20 Questions, you might jump straight from, "Is it an animal?" to, "Is it a panda?" just because it's fun and sometimes you're right.

Posted by: Yawnoc | Link to this comment | 07- 5-15 10:46 AM
21

It's interesting that the purported cognitive bias in play is not just that we're primed to perceive patterns everywhere but that we're primed to perceive complex patterns when a simpler pattern is more parsimonious.

Posted by: Yawnoc | Link to this comment | 07- 5-15 10:55 AM
22

I do think this would be good for testing for candidates in my line of work, where there often aren't enough eyes to look over your analyses critically and you need to be able to self-correct.

Posted by: Minivet | Link to this comment | 07- 5-15 10:59 AM
23

I got the answer right, but I think this "cognitive bias" is actually a reasonable heuristic for our brains to use. If I tell you "I'm thinking of a trait shared by these people: Andrew Jackson, Grover Cleveland, and George H.W. Bush", you would reasonably think I was fucking with you if I told you the common feature I had in mind is that they're all male. So the underlying heuristic is that the probability, given the incredibly common trait I have in mind, of my randomly choosing an exemplar of a much less common trait is very low. In the number sequence case, if you're grabbing at random a sequence from the very general class "increasing sequence" it's going to be rare for it to look like a familiar example of a much less common sequence like "powers of 2".

I think the problem is not in using this heuristic, but in failing to test to see that it's working. But I don't think "perceive complex patterns when a simpler pattern is more parsimonious" is necessarily a bias.

Posted by: essear | Link to this comment | 07- 5-15 11:20 AM
24

Andrew Jackson, Grover Cleveland, and George H.W. Bush

"Who are three people who have never been in my kitchen?"

Posted by: Todd | Link to this comment | 07- 5-15 11:37 AM
25

I think essear has it basically right: it's a socialization thing. I remember seeing many, many problems like that in highschool and the answer was invariably something at least reasonably mathematically complicated, even if the same sequence could have been part of one of an infinite number of different sequences. I always found it annoying that "guess which sequence the person writing the test had in mind" was presented as having an answer that wasn't mostly psychological.

Posted by: MHPH | Link to this comment | 07- 5-15 11:42 AM
26

"They were all male" *is* the parsimonious explanation for much of their lives and actions.

Posted by: opinionated radical feminist | Link to this comment | 07- 5-15 11:47 AM
27

Far, far more common is the "We have a newborn idea, everybody poke holes in it" phenomenon.

I'm glad the Ninja Librarian Association is so careful. Just think of the destruction they would wreak if they ran off willy-nilly every time someone had an idea.

Posted by: essear | Link to this comment | 07- 5-15 12:08 PM
28

I agree with 23 but not 25. The socialization only gets you so far, as the kind of questions you're talking about don't allow you to test your pattern. The ability to test should make this problem feel very different.

At any rate, yes your first guess should be the rare pattern, but to confirm that you have a rare pattern you should be first checking that most things give "no."

Posted by: Unfoggetarian: "Pause endlessly, then go in." (9) | Link to this comment | 07- 5-15 12:18 PM
29

I found order mattered after two guesses, thought about the uncountable infinite number of possible rules, and gave up. In hindsight, I should've used the "it's in the New York Times so it can't be too complicated" heuristic.

Posted by: Eggplant | Link to this comment | 07- 5-15 12:24 PM
30

Bush, Bush, Harrison: yes
anyone but Obama, anyone but Obama and the other guy listed before this one, Obama: no

Posted by: fake accent | Link to this comment | 07- 5-15 12:40 PM
31

28 - The possible answers get trickier the more pieces you add, sure. But there aren't really any fewer of them. And the trick to the question, after all, is that it's a lot simpler than you'd immediately guess given how we're taught to look at problems like this.

Posted by: MHPH | Link to this comment | 07- 5-15 12:52 PM
32

Tried a sequence with odds. Tried the sequence backward. Put in one, two, three. Got it.

Posted by: Cala | Link to this comment | 07- 5-15 1:05 PM
33

32 is what I did, except that I tried exponential first.

Posted by: Moby Hick | Link to this comment | 07- 5-15 1:07 PM
34

I got it, because I thought "let's eliminate the simplest answer first." The more times I couldn't elimanated that as the answer, the more I felt my anxiety grow because my default is to assume that someone is playing a trick on me. I kept trying everything I could to get that answer eliminated or its opposite confirmed. Almost every preventable disaster in my life has been because of not wanting to test for "no". I hope this means I'm getting better at that, finally.

Posted by: Penny | Link to this comment | 07- 5-15 1:57 PM
35

23 is good.

Posted by: Alex | Link to this comment | 07- 5-15 3:06 PM
36

Got it, but did 4 or 5 "no's" to make sure.

If clew is right in 18, it's a kind of reasoning problem I see kids making all the time: jumping to conclusions on limited evidence and not testing counter examples

Posted by: Calypso | Link to this comment | 07- 5-15 3:37 PM
37

Kids undergo peer review, but not of the kind that requires them to defend their conclusions.

Posted by: Moby Hick | Link to this comment | 07- 5-15 3:48 PM
38

U. S. A., U.S.A, U.S.A.

Posted by: Moby Hick | Link to this comment | 07- 5-15 5:51 PM
39

That game was really something.

Posted by: MHPH | Link to this comment | 07- 5-15 5:59 PM
40

Got bored after a few yeses and clicked "just tell me the answer".

Substitute "faintly irritated" for "bored" and yep, same story.

Posted by: Mister Smearcase | Link to this comment | 07- 5-15 6:04 PM
41

I didn't actually watch the game.

Posted by: Moby Hick | Link to this comment | 07- 5-15 6:05 PM
42

Let them count themselves if they're so crazy mad after mathematics, I said to myself and LizardBreath.

Posted by: Mister Smearcase | Link to this comment | 07- 5-15 6:06 PM
43

Carli Lloyd scored three goals within about fifteen minutes, right at the beginning. One was from around the 50 yard line.

Posted by: MHPH | Link to this comment | 07- 5-15 6:16 PM
44

23: Whether this actually maps onto how people make decisions or whatever, I have no idea. But the puzzle didn't ding you for using the heuristic "these kinds of things are always mathematical formulae", which is where everyone who's run into these things before starts, but for not knowing how to check.

Posted by: Cala | Link to this comment | 07- 5-15 6:16 PM
45

I was watching Inspector Morse, which is basically the same thing because they're both British in origin.

Posted by: Moby Hick | Link to this comment | 07- 5-15 6:18 PM
46

I used negative numbers and zero to be sure (and to see if it would break).

What I like is the way it allowed for a plain text answer. Nice work there. Even so I trued to write it so the parser could evaluate it. That I did not try to destructively test.

Posted by: md 20/400 | Link to this comment | 07- 5-15 6:34 PM
47

They also appear to have exactly the same number of rules about not trying to date witnesses, suspects, and extremely recent widows.

Posted by: Moby Hick | Link to this comment | 07- 5-15 6:35 PM
48

43- 54.4 yards to be exact. It's a Canadian football field after all.

Posted by: SP | Link to this comment | 07- 5-15 6:53 PM
49

I watched an Inspector Morse yesterday in which the murder victim is Japanese, but it turns out someone had replaced him, ante mortem, with a different Japanese but nobody noticed because Asians. It was from 1989.

Posted by: foolishmortal | Link to this comment | 07- 5-15 8:39 PM
50

In their defense, I think they all just met him that day.

Posted by: Moby Hick | Link to this comment | 07- 5-15 8:45 PM
51

Morse solves it while listening to the Mikado, I fear.

Posted by: conflated | Link to this comment | 07- 5-15 11:06 PM
52

Almost every preventable disaster in my life has been because of not wanting to test for "no".

"Not love, quoth he, but vanity/ Sets love a task like that"?

Posted by: ajay | Link to this comment | 07- 6-15 2:26 AM
53

I tried the usual things. Odd numbers, numbers in reverse order, numbers where they descended, where one number was repeated, where all of the numbers repeated, where they were 2 and then 4 different from each other, and several other possible options. I think I had about 12 attempts before I felt confident to guess.

Posted by: nattarGcM ttaM | Link to this comment | 07- 6-15 4:43 AM
54

My greatest fear in doing policy work is that I will publish something that will cause people to laugh and point out an obvious explanation that I have overlooked.

I know what you mean. Imagine the egg on my face when I realized that I had completely ignored slavery.

Posted by: Opinionated "States Rights" Historian | Link to this comment | 07- 6-15 7:19 AM
55

54: No sweat, I'm sure that will be just as easy to ignore.

Posted by: peep | Link to this comment | 07- 6-15 7:28 AM
56

On the veldt, people who ignored eggs were less likely to reproduce because they didn't have as much protein.

Posted by: Moby Hick | Link to this comment | 07- 6-15 7:30 AM
57

52: Ha! Yes. My own vanity revolves around avoiding the glove-in-the-face.

Posted by: Penny | Link to this comment | 07- 6-15 7:58 AM
58

I was one of the 91% of cognitive plebs, but I also think the visual design of the puzzle reinforces this (as well as the context of newspaper puzzles).

If there had been one more checkbox to say "I think this is right / wrong", and the widget had showed green whenever you matched the prediction, the visual feedback would have been inline with the intent of the question. It would be "successful prediction" instead of "wrong". That's how unit tests tend to work.

Posted by: conflated | Link to this comment | 07- 6-15 5:59 PM