If you were a better-than-average driver, how would you know?
Are you good at maths? Are you good at drawing?
Your answer to these questions is probably reasonably accurate. Unless you're a mathematician or an artist, you probably don't have much pride or identity tied up in your skill in these things, so that won't cloud your judgment. (And if you are a mathematician or artist, then - relative to average people - you're almost certainly good at maths or drawing.)
Are you good at driving? Are you a good judge of character? Are you (or would you be) a good parent? Are you a good voter?
Everyone thinks they are a better than average driver. The appropriate conclusion to draw from that is that, if you need to know whether you're a good driver, the way to discover this information is not to simply introspect on the question for a few seconds. I suspect the same holds for whether you're a good parent, voter, or judge of character.
That's a serious problem! It probably doesn't matter whether you're a good driver. Even if you're worse than average, you're probably not so bad that you shouldn't be driving at all. But bad parents and aggregated bad voters do a lot of harm. If you're likely to be a bad parent, then that is actionable information: don't have children! If you're likely to be a bad voter, don't vote! If you're a poor judge of character, get a prenuptial agreement.
If you really wanted to know whether you're a good driver, you could probably find some reasonable metric and compare yourself to the average. But there are no controlled experiments in parenting or politics, which makes it very difficult to be sure you have reliable knowledge on these questions.