Chapter 18: When Data Doesn’t Convince You

options("scipen" = 9999, "digits" = 4)

Q1

When two hypothesis explain the data equally well, one way to change our minds is to see if we can attack the prior probability. What are some factors that might increase your prior belief in your friend’s psychic powers?

1. We reality shift to a world where psychic powers exist, because they don’t in this world.
2. I go to the gas station with my friend and they buy a single lottery ticket for the exact number that wins the jackpot later that night.
3. My friend bends a spoon or lifts an object with their mind.
4. My friend’s psychic powers transfer to other dice or cards instead of just guessing the result of a six sided die roll.

Q2

An experiment claims that when people hear the word Florida they think of the elderly and this has an impact on their walking speed. To test this, we have two groups of 15 students walk across a room: one group hears the word Florida and done does not. Assume $H_1 = $ the groups don’t move at different speeds and $H_2 = $ the Florida group is slower because of hearing the word Florida.

The experiment shows that the Bayes factor for \(H_2\) over \(H_1\) is 19. Suppose someone is unconvinced by this epxeriment because \(H_2\) had a lower prior odds? What prior odds would explain someone being unconvinced by this experiment and what would the BF need to be to bring the posterior odds to 50 for this unconvinced person?

Now suppose the prior odds do not change the skeptic’s mind. Think of an alternate \(H_3\) that explains the observation that the Florida group is slower. Remember if \(H_2\) and \(H_3\) both explain the data equally well, only prior odds in facor of \(H_3\) would lead someone to claim \(H_3\) is true over \(H_2\), so we need to rethink the experiment so that these odds are decreased. Come up with an experiment that could change the prior odds in \(H_3\) over \(H_2\).

If someone were unconvinced by this experiment, we can assume that their posterior odds are, say, three or less. Then their prior odds would need to be \(3/19\) or less. If they are totally unconvinced, say their posterior odds are \(1\) or less, then the prior odds would need to be \(1/19\) or lower to cancel out the Bayes factor entirely. If someone’s prior odds were \(1/19\), in order for their posterior odds to be \(50\), the bayes factor would need to be \(19 \times 50 = 950\). If their prior odds were \(3/19\), the bayes factor would only need to be \(\lceil 950 / 3 \rceil = 317.\)

One alternative hypothesis, which we’ll call \(H_3\), could be that the two groups of students were measured at different times: the slower group was measured in the morning. We could fix the experiment by ensuring that both groups of students were measured at the same time, or by getting multiple groups of students to repeat the experiments at the same time (adding replicates to our experiment). It could also be possible (say \(H_4\)) that the slower group just contained all slower students. We could fix this by randomizing students into groups and by replicating the experiment multiple times. It seems like replicating the experiment might help no matter what.