Many people see it as the key ingredient of success, and assume that boosting confidence is the solution to all their relationship and career problems. Most people would rather swallow a confidence pill than a pill designed to boost their knowledge, empathy, or competence
Yet, there is little evidence for the positive effects of high confidence, and a great deal of evidence for its detrimental effects. Furthermore, although much of Western society regards insecurity as a sort of character disability, there are many psychological advantages to low confidence, and several reasons for embracing our inner insecurities and self-doubts. Consider the following facts:
Moreover, even when people are told about the better-than-average bias, and asked whether it applies to them, 80% of respondents say "No." In addition, neuroscientific research shows that optimism is a pervasive judgment bias, which causes most people to underestimate the feasibility of negative events happening to them. The bottom line is that most people have more confidence than they need: We are just not as great as we think we are.
2. There is no evidence that overconfidence breeds success: Henry Ford famously noted that "whether you think you can do it or not, you are usually right." It turns out he was wrong. Try thinking that you are going to be the next Steve Jobs, marry Brad Pitt, or have more Twitter followers than Lady Gaga.
And if you think that the problem is that those things are hard to envisage, let me assure you that there is no shortage of people in the world who aspire to these and other unattainable goals. The psychological term for them is "delusional" and while it may be tempting to encourage them to keep dreaming, they would actually benefit more from having more realistic and achievable goals.
3. There is a high price for overconfidence: Our evolutionary ancestors developed confidence as an internal alarm system that alerted them about environmental threats. That is still the adaptive function of confidence, namely to inhibit behavior in the hope of preventing failure or avoiding danger. When you don't feel confident about crossing a busy road, that self-estimate of your ability will stop you from being killed crossing the road.
And if you don't feel confident about giving a talk to a big audience, you should probably prepare more and improve. Although there is clear evidence that low confidence is linked to realistic competence deficits, most people are overconfident, so they have a natural tendency to misjudge danger. The 2008 financial meltdown, the abundance of sociopathic managers and politicians, and the frequency of road accidents -- especially in male drivers, who account for over 80% of serious car accidents despite being much more confident than female drivers about their driving skills -- are just some examples.
So, what does this imply for individuals' career success and the effectiveness of organizations?
The first implication is that the world needs more humility -- and low confidence is not a bad starting place to achieve that. Unless we are aware of our limitations, and dissatisfied about our perceived levels of ability, we will have no incentive to improve, and no motivation to work hard to get better. Conversely, when we feel annoyed or discontent because we experience a gap between the person we think we are and the person we want to be, our insecurity can be a catalyst of success.
The second implication is that an individual's short-term gains from overconfidence come at the expense of long-term collective gains. In other words, if we keep rewarding those who think highly of themselves, simply because they think highly of themselves, then we will always end up with incompetent charlatans in positions of power and influence.
This is an important, yet neglected, point that most people neglect when they highlight the apparent benefits of overconfidence. For example, would you rather have a pilot, surgeon, or teacher who is great at faking competence, or someone who is actually competent? The same logic should apply in all professions and areas of life.
The third and final implication is that we need to get better at distinguishing between confidence and competence. Clearly, we are not very good at this and that's why some individuals successfully exploit overconfidence as a presentational strategy. When bad singers appear on a talent show, their confidence only makes them look sillier -- because it highlights a huge gap between their self-perceived or claimed abilities and how competent they actually are.
It is time to apply the same discerning powers other situations: Job interviews, first dates, and networking events. When we hear people making claims about their talents, let's not assume that they are true, even if they are being honest (as a consequence of being self-deceived). Most talented people don't brag about themselves, and most of the self-promoters in the world are simply impostors. Britain understands this more than America, but less than most Asian countries.
In the end, the societies that value humility, hard work, and self-knowledge will eclipse those that celebrate hubris, entitlement and self-esteem -- or has this already happened?