Shermer, drawing upon ideas from Carl Sagen, provides five guiding questions in the examination of any claim. The first question is: does the source of the claim make many similar claims? One who claims to have seen a strange phenomena is rendered less believable if he or she is constantly claiming to have observed a strange phenomena.
Second, has the source been verified by other people? An extraordinary claim is believable once it has been corroborated by other people (who, presumably, are not the kind of people who would not be surprised by such a claim or not people who have observed many incidences of strange phenomena).
Third, has there been a sustained and vigorous effort to disprove the claim? In other words, a cross checking routine is vital to overcoming the skeptic. And, of course, the more unusual the claim the more cross-checking is required.
Fourth, is the claim purely negative? Claims that merely provide argument against an idea, but never articulate an alternative, should be treated with suspicion.
Finally, does the claim fail to account for bias on the part of the source? If one approaches evidence with some prior conviction that might determine the interpretation of the evidence, then the claim is suspicious.
All the above tests rest on certain assumptions. First, there is a scale of ordinary to extraordinary. What counts as ordinary is not defined, but we are sure that Shermer has, at least in his own mind, a scale of ordinary to extraordinary that counts for him. Of course, the scale is different according to different groups of people. For example, an extraordinary claim, for some, might be one that Shermer counts as ordinary (evolution might be a good example).
Second, Shermer assumes certain kinds of people are better than others at verifying claims. Those who themselves make extraordinary claims are not able to verify other extraordinary claims, this would be considered biased. Of course, Shermer’s own bias is, for Shermer, no bias at all. But how would he know this? How would he know if his view point was not biased? It would be particularly difficult to notice his own bias if he is surrounded by people who share it.
The interesting thing about each claim is the assumption that skepticism is prompted the more extraordinary a claim is. But that is not a statement about the claim; it is a statement about the belief in a claim. And the belief is held by people. One can see that Shermer’s actual test for what is believable is not whether or not it stands the scrutiny of reason or evidence, but whether or not it is approved by certain people with certain beliefs.
