Anecdotal evidence: A closer look - Commentary

by Ronald Sones

Have you ever tried to make a point with some social science professional, only to be told something like, “Well, that’s just based on anecdotal evidence; it’s not scientific; we can’t rely on that.” Usually, that kind of reply was intended to make you feel like a fool and get you to shut up.  And if you didn’t understand this dynamic, it probably worked.

Well, I’m here to tell you that you don’t have to be a victim of this kind of mind-abuse trap. First, you need to be aware of just what “anecdotal evidence” is.

These are simply statements that purport to be true, which are based on little or no evidence, reported by sources that may or may not have some political or economic agenda intended to influence you to believe the object of their agenda. Based on this, you could well believe that the social science professional had a good point after all. Indeed, we would tend to believe scientifically derived information, i.e., results of randomized double-blind studies in which particular hypothesized ideas are applied to one group but not to another control group, and any differences are analyzed. And clearly, this kind of information should be considered as more reliable than what I’ve described as anecdotal information.

That’s the ideal.  But what happens in real life? In real life, you are continually faced with having to make decisions, some of which will have the profoundest effects on your life or the life of others. For example:

• What kind of work should you do to sustain your very existence?
• Where should you live?
• Whom should you marry?
• If in a trial of an individual, what verdict should you find?
• How should you plan for unexpected difficulties?

For any of these, or a multitude of others, were you able to set up some scientific experiment to evaluate all of the possible alternatives and select the best one?  Of course not! But upon what DID you actually base your decisions? You based them on whatever facts you could gather relevant to each situation at hand. You analyzed these facts as best as you could and then reached your decisions.

These facts were all anecdotal evidence. As I described above, they came from various sources of all kinds, and you had to evaluate how reliable any of them were. This is the crux of the matter: what is the basis for believing anecdotal facts that are presented to us? After all, based on this, it’s clear that anecdotal evidence can be crucial in affecting one’s life.

There is an entire branch of philosophy called epistemology which deals precisely with this issue of why things should or should not be believed. I don’t intend to present a treatise on epistemology here. I’d just like to present some ideas I’ve gleaned over the years on practical steps that can help in this regard.

First, you need to be aware that the vast majority of things you believe are things that someone told you. Usually those “someones” would be parents and teachers, but also friends and other acquaintances. Also, of course, there would be all kinds of sources in your culture, e.g., books, news media, music, entertainment, on and on.

Then, you get to believe things you find from your own investigations and explorations.  So, what do you believe, and what do you reject, and why?

• Your first sources of beliefs are your parents and teachers. While you are very young, you trust them because they provide you with nurture and protection. Later, you evaluate what you are told by comparing what you are told by one source against what you are told by other sources and by what you observe for yourself. But the point here is, you tend to believe information that comes from sources in whom you’ve developed trust, either because they’ve consistently provided you with benefit, or because they have a good track record of giving you information that has held up in the light of other information.
   
• Next comes corroboration. If you receive anecdotal information from multiple independent sources, ones that you believe are not in collaboration with each other, you would tend to accept that the information is true. Otherwise, you would have to believe that all of these independent sources are wrong together, which would normally be very unlikely.
   
• Next would be your own investigations. Naturally, you will tend to trust yourself and believe your own observations. But even here, you need to be aware of your own biases and the possibility that observational tricks, such as optical illusions, could be at play.
   
• Finally comes analysis of the new information in the light of things already known and believed, and in the light of logic. Here is where experience and expertise come in. If the new information squares with things you already know and/or can derive logically, you will tend to believe it.

That’s pretty much it. We do the best we can to make sense of the world based on all of the anecdotal evidence that we are receiving all the time. Unfortunately, sometimes we reach the wrong conclusions: we pick the wrong employment, we live in the wrong place, we marry the wrong person. “It ain’t what we don’t know that gets us into trouble; it’s what we do know that just ain’t so.” (attributed to Artemus Ward) I suppose no one is immune to this, but the point is, this is the world in which we live, and the world in which we need to be prepared to apply grains of salt.

And, just to make things a little worse, there can be problems too even with scientific evidence. For one thing, one needs to be aware of who is sponsoring any particular scientific study.  It’s usually not surprising if a study just happens to confirm the agenda of the organization that sponsors it.  Also, many years ago, I saw an article that said a scientific study showed that some 25% of scientific studies were based on faked or doctored data. If this were true, that would mean that that study itself had a 25% chance of being fake.  That brings us to the old conundrum “All generalizations are false, including this one.”

And this has been recognized on a global scale: “History is the set of lies upon which people agree.” (attributed to Napoleon).  So it would seem that one should always be careful about what one chooses to believe.

En español aquí