Many years back, while working with a client, I was asked to take a written personality test that was required for all executives in the company. Eight senior leaders took this test, costing the company more than $10,000, not including the time invested by the employees and the travel costs to the off-site debrief meeting. The test had more than 300 questions and took nearly an hour to complete. By the end, I was exhausted. Afterward, I received a 40-page assessment detailing every aspect of my personality. Although some of the aspects of the assessment seemed accurate, a large number of points were way off.
Others on the team focused on the hits and forgave (or didn't notice) the misses. By doing so, they ensured that the assessments would seem more accurate than they actually were. Being a bit of a skeptic, I wasn't willing to dismiss the inaccurate bits so quickly.
What occurred among these executives is the phenomenon psychologists call the "Barnum effect": our tendency to focus on the most accurate aspects of characterizations of ourselves. The phenomenon is named after the legendary showman P. T. Barnum, who believed that a good circus had "a little something for everybody." And with many personality profiles, a little bit of everyone is in every profile. Horoscopes and palm readings seem accurate mainly because they are cast in such a way that they could apply to almost anyone, especially if you're willing to get a bit imaginative in your interpretation.
A great example of the Barnum effect comes from Professor Michael Wiederman from the School of Medicine Greenville and an expert on psychological tests. He shared with me an experiment he tried in one of his classes. He had each of his students take a widely used personality test. But beforehand, he secretly took the test four times, answering the questions randomly each time. He then printed out the personality profile descriptions of the four randomly taken tests and made copies. In the class, instead of handing the students their actual results, he passed out the randomly generated profiles. Each student was asked to read it to themselves.
Then he asked for a show of hands from those who thought what they just read described their personality pretty well. The majority of the class raised their hands. He then had them pass their write-up to the left and read a different randomly generated profile. Same vote, same results. He finally passed out their actual profile. The accuracy level was not significantly greater than the randomly generated ones. This is the Barnum effect in action.
Just because an assessment seems complex and thorough, don't mean it is scientifically valid or even accurate.
On this topic, Professor Wiederman said to me, "Don't confuse complexity with validity. And don't confuse validity with usefulness." Sometimes the best solutions are the simplest ones, and the simplest ones may ultimately prove to be the most useful.
He then went on to recall a study published several years ago in which a battery of widely used depression tests and methods were administered to a group of people, along with some simple questions. Although the tests administered were complex and supposedly scientifically validated, the most accurate predictor of depression was the single question: "Are you depressed?" So much for complexity.
So next time you take an assessment of any kind, be skeptical. Although a sucker may be born every minute (according to P.T. Barnum), it doesn't mean you need to be one.