The Illusion of Confidence


Consider the following scenario of you visiting a doctor. What tells you that the doctor is trustworthy in his/her assessment of the health condition? You may, thanks to your past experiences and intuition, look for signs that indicate the preparedness as well as the accuracy of diagnosis on the part of the doctor. In other words, you are looking for how confident the doctor himself/herself. What if the doctor refers some literature to research or verifies what the initial diagnosis has revealed? Will you continue to be confident of the abilities of the doctor? Does this mean that the doctor is in any way unsure of his/her competence?


It is in such situations that we often place a huge leap of faith on confidence, that it shows how skilled a person is, more so about the latter’s knowledge and the chances of being a subject matter expert. However, this can be an illusion. We depend on our intuition to get around much of our daily lives. However, our confidence in the intuitive outlook of our surroundings can play a bigger role in determining the extent to which we trust our judgments.


Consider the World Open Chess Tournament, one the largest public tournaments, held annually in Philadelphia. Anyone can register for the competition and the point system employed here is one of the most accurate in the world of sports. Since it’s analytical in nature, not to forget that the rankings and the underlying methodology are public knowledge, the ratings are a near-factual representation of the players’ skills. The ratings are further updated after every match.


Christopher Chabris and Danial Simons, who co-authored the book ‘The Invisible Gorilla’, devised a questionnaire to understand what the players accessed their performance and how do they place themselves relative to the rest of the competitors. Although the ratings are up-to-date and offer a definite outlook on the players’ performance, many players themselves seemed to rank themselves higher than they actual numbers indicated. They even expected the ratings to go higher as per their perceived ‘true strengths’.


The researchers observed another tendency among the competitors. Those who were ranked lower on the card came across as being more confident about their abilities. Their estimation of the real ratings was almost twice as much as of those who were in the leading. The top half of the competition estimated their rankings to be around 50 points higher. When the researchers followed it up in the next year, surprisingly the ratings appeared to be in the same range as the year before, thus showing that much of the players hadn’t reached their foreseen ratings. What if the players are never able to reach the true potential that they believe they are capable of?


Note that the players in question weren’t rookies. Most of them were attending the event for a few years. Yet, there seemed to be such a staggering overestimation of their abilities, which could be explained by only one thing. The illusion of confidence. Now, this can happen in two ways. One, the exaggerated view we have of on what could be accomplished by us, compared to others. Secondly, the way we seem to translate the degree of confidence that we carry at any given instance.


Charles Darwin marked that “ignorance more frequently begets confidence than does knowledge.” This enigma of overconfidence hijacking an objective view of the world takes over many of those who operate on the wrong side of the law. Those involved in illegal activities often are caught in the most expected manner (remember the bank robber who applied lime juice and thought that he’ll not be caught by the CCTV?). Psychology suggests that such behavior is a result in part due to a two-fold challenge. A person might be below average in his ability to complete a task and fail to realize that same. The latter prevents him from working on becoming better at the craft.


Some blame may also be placed squarely on the constant emphasis laid on self-confidence which is best described by the attitude of “a life lived without confidence is not a worthy life!” Picture this, you are told by every self-help book, every career advisor to be confident in your abilities. Well, the only drawback this over-reliance on “How confident are you feeling?” masks the need of a credible feedback loop that you can use to improve your skills.


Confidence and Group Thinking


In an ideal world, when a few people get together to solve a problem, you can expect the following: Each member of the group works on the problem individually. Everyone later compares each other’s answers, followed by a discussion. This will lead to finding the correct solution. One of them eventually rises above the rest as a dependable person to work through every case and end up with the right answer in most cases. And such, this particular person goes on to become a natural leader of the group. This is a classic case of group dynamics.


Behind most group decisions, however, certainty and confidence in a decision take precedence over an objective view on the issue at hand. In an organic team building exercise and further deliberation on overcoming challenges, oftentimes those with a towering personality hold the reins while some quit folk may choose to work in the shadows. Not just that, the aim to mitigate conflicts may push the team towards collective agreement over an estimated solution. While one may suffice that there is “strength in numbers” and display great confidence in a group’s abilities to judge correctly, the same may lead to appalling decisions that could be prevented had individual analysis and an unbiased judgment was allowed.


Case in point, dominant team members might cause others to perceive them as being more talented than they are in reality and handing over greater decision making power to the former. All because they appeared confident! In many cases such ‘confident sounding’ people need not be bullying others into agreement, team dynamics allows them to claim the top spot by merely speaking out first.


Confidence as a personal trait


In psychology, there are five traits that are believed to characterize personal behavior. Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness. Confidence is mostly related to dominance, which in turn doesn’t feature as a principal trait. However, the way confidence is expressed determines to a good extent how many of us make decisions and influence other people.


Speaking of traits, confidence is the root word for the word ‘con’ in say, con man, con artist! William Thompson was a scammer back in the 1840's, who, in the streets of Manhattan, he would approach strangers and ask them to hand over their watches. To do this, the man had to express supreme confidence in his words and actions for people to trust him. Would you trust someone with your belonging in such a situation? Surprisingly many not only part with their stuff but answered in affirmative when asked whether they were confident in Thompson to hold on to their watches until the next day.


In the galaxy of scam artists, who can forget Frank Abagnale of the Catch Me If You Can fame (or infamy?). Not satisfied after cheating his father off $3,400, he went on to have many high profile ‘ventures’ including being a Pan Am pilot, cashing in forged checks amounting to millions of dollars and once escaping by pretending to be an undercover investigator. But can this amount of confidence be a trait? Here, we have to acknowledge the fact Frank’s father himself had a history of fraud. Also, studies have shown that identical twins have similar levels of confidence while fraternal twins don’t, indicating that there might exist a crucial element in our genes that may dictate how confident a person grows up to internalize.


Confidence and disasters


Overconfidence in one’s judgments can be misguided and lead to unfortunate outcomes, which we have seen. When this extends to the way society in itself functions and more importantly is affected, then it’s a slippery slope. Case in point, the various military adventures countries, big and small, have undertaken over the years are proof that relying on the factor of confidence may not be the preferable step when dealing with large-scale issues.


Now, this can play in both the ways this article discusses, individuals and group thinking. Related studies did show that those who might otherwise be wary of expressing their views alone feel more empowered in groups. The end result nonetheless will lead them to have increased confidence in uncertain and riskier choices.


The legal system may sometimes play into the hands of a confident individual by wrongly accusing or acquitting individuals. In some trials, the juries “rely on confidence as a way to distinguish an accurate witness from an inaccurate one.” Siegfried Sporer of the University of Giessen, Germany reviewed all the studies that were conducted regarding the accuracy of testimonies provided by witnesses and almost 70% of those who were confident backed it up with the correct information. This review casts a couple of issues. Firstly, the confidence of a witness depends on how confident they themselves are regarding their deposition. Secondly, even with 70% accuracy, there is plenty of room for things to go awry, mainly wrongful convictions. A system that can prove the testimonies with hard data can help in reducing such erroneous judgments.


Dr. Confident


Remember the example of visiting a doctor that was discussed earlier, Dr. Jim Keating of the St. Louis Children’s Hospital provides some insight into what makes a doctor confident. Decades of experience has given this surgeon an ability to treat a wide range of problems. He says “When people go to the doctor, they often believe that the doctor has an ability to make the right decisions for them. That goes beyond the scientific reality. They trust the doctor’s decision-making more than their own. That’s a problem because it encourages doctors to not be honest about what they know and what they don’t know. It builds your ego to have people think that you know.”


Budding doctors are trained to present themselves with confidence. Patients, when they meet these confident-appearing doctors, may mistake the same for competitive abilities. The almost unquestioned faith in his/her diagnosis by the patient will in-turn reinforce the doctors’ views on their work. As such, we see that confidence is a self-perpetuating cycle. A 1986 study at the University of Rochester shows the influence of misguided assumption. Patients who were waiting for their appointment were shown a pre-taped video of a simulated meeting of a doctor and a patient. In one case, the doctor shows no doubt in his diagnosis. In another version, the doctor seems uncertain about his analysis of the ailment and looks into a book to clarify. As you might be expecting, people voted the former as being more satisfactory in dispensing his duties, compared to the one who referred a book. As the researchers later put it “Doctors who express doubt are probably more self- aware than those who don’t, but people rarely notice that sign of actual competence in an expert”.


But what about those times when a doctor’s analysis of a health condition is wrong. What if the doctor chooses to go that way, although it might turn out to be a bad decision? What if the patient fails to notice that the doctor’s confidence is obscuring reality? Some researchers believe that when faced with a callous display of confidence in the doctor’s self-analysis, maybe it is nice to take a second opinion. Those who welcome views from better-learned professionals are likely to provide a better service in the longer run.


Confidence is quite over


Thomas Jefferson said, “He who knows best, best knows how little he knows”. This hypothesis that people with limited knowledge may exhibit over-expectation of their abilities along with a miscalculated view of themselves was put to test by social psychologists Justin Kruger and David Dunning of Cornell University. In their work titled “Unskilled and Unaware of It”, the researchers prove the aforementioned hypothesis through a series of tests in different scenarios. Various studies carried out prior to this had stated a common perspective regarding incompetent people, that they ‘lack the metacognitive skills necessary for accurate self-assessment.’ In general, novices appear to harbor poor metacognitive skills compared to professionals in any given field of interest.

In one study the researchers found that as opposed to professional comedians, some of the participants weren’t able to get the underlying humor in some jokes that were presented to them. In spite of this, they ranked themselves in the above average group of those who have a good sense of humor. In another, the study subjects were provided a logical reasoning test while another group was presented with a grammar-based one. In both the cases, it was found that those with below par performance rated themselves much higher than others. They lacked the ability to gauge the proficiency of others as well.

An interesting outcome of these studies was that, when the erring participants were provided with a feedback and trained in the subject of the concerned test, not only did their performance improves, the over-confidence that they carried also diminished. In the course of time, they were better off in their estimation of their capabilities. However, this is not what one observes in some cases since those who are comparatively worse-off than their competition rarely receive a negative feedback. Even in the event, such a feedback is provided, people may find it difficult to point out the exact reason to have failed. On the other hand, for anything or anyone to be successful, all of the constitutive factors must go the right way. For failure, one or more of these variables might come up short, in turn dragging the end result with them. This also, in some cases cause confusion in understanding the specific cause for failure, either in one’s own ability to perform or in evaluating the competition.


If you are thinking that this whole misinterpretation of self-capacity is something solely related to incompetence, then you are wrong. The same studies also showed that in many cases, those at the top quartile of the participants tend to under-estimate their cognitive abilities. These candidates were behaving so due to “false-consensus effect”, wherein they assume that their performance turned out well and thus everyone else must have performed equally likely. The researchers extend the words of Thomas Grey when they declare “Ignorance is bliss-at least when it comes to assessments of one’s own ability.”


The deception of Self-knowledge


Will you trust any person with a white lab coat as a credible voice regarding your health? Maybe you wouldn’t, but there’s another way people can be swayed by opinions. Think of a friend who is really good at cooking. When this friend offers some advice regarding an ingredient, you take it without thinking much. That’s because you know that friend has a lot of experience preparing mouth-watering dishes. Suffice to say, you trust when this friend of yours provides suggestions on cooking and is confident about it.


Research shows that with increased experience, people become more confident in their trade while the level of over-confidence steadily decreases. However, with your experience of your confident friend, it is natural to expect similar confidence levels in others when taking their views and opinions seriously. Right? How else will you measure their subject knowledge and experience? In such cases, confidence becomes a personality trait, with people expressing the same to various degrees.


Evolution has ensured that confidence is a strong indicator of the trustworthiness of an individual. Trusting a family member off of his/her confidence is a natural reaction of the human mind. When this trait is extrapolated to a stranger and draw inferences related to one’s confidence level can be detrimental at times. As a postulate of this view, some of us assume our scholarship in a particular subject transforms us to be similarly good in other areas.


Notes:


  • This is one of the six everyday illusions that we experience as explained in the book ‘The Invisible Gorilla’ where the other five illusions are those of attention, memory, knowledge, cause, and potential.

  • Kruger, J. and Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), pp.1121-1134.

 

©2018 by Mind Brunch