How do you distinguish good experts from bad experts?
How do you distinguish good experts from bad experts? It would be an extraordinarily useful skill - a meta-skill that gives you access to so many others.
Priests and biologists both claim to know the origin of human life. How do you choose between the two? It seems obvious to me, but people disagree on this so there's something non-obvious about it.
How do you distinguish between a good mutual fund manager and a bad one? It's really hard. So you might choose to get an index fund instead, which is probably not a bad idea. In one sense, that would be judging the market to be your good expert. Patri Friedman didn't do that in 2008; he chose a bear strategy instead. This is evidence that Patri Friedman is good at choosing good investing experts or is a good investing expert himself (but not sufficient evidence - watch him for another ten years).
How do you distinguish between bad science and good science? Patri Friedman thinks he can. squid314 challenged him:
You need to prove that you can "beat the house"; that your own judgment is likely to be less fallible than the nutritional scientists' for some reason. Top nutritional scientists should know everything you do about the problems with certain kinds of studies and take that into account. So all of the pitfalls that apply to nutritional scientists equally apply to you, unless you have some specific reason to think they don't. And then the experts have the extra advantage of much greater familiarity with the subject matter.
Patri claims "that by using [rigorous scientific procedures], digging into a few papers, correcting for funding biases, one can occasionally decide to take a contrarian view, and be correct."
I suspect that that approach would yield a strong conclusion on the priest/biologist question.
How do you distinguish between good economics and bad economics? Nobel Prize-winning economists Paul Krugman and Paul Samuelson have a different view of macro-economics than Nobel Prize-winning economists Milton Friedman and Friedrich Hayek (of course, all four of them disagree, but if you're going to divide them into two groups, that's a sensible categorization). Who are the better economists? How do you know whether you are a good enough economist, or a good enough judge of economists, to decide?