Kevin Simler at Melting Asphalt recently published a very nice summary post – Crony Beliefs. Building on the evo-psych work of Trivers, Haidt and Kurzban he distinguishes between two types of beliefs (“employees”):
- Meritocratic Beliefs
- beliefs that are entertained because of their epistemic value, i.e. beliefs that pay their rent in accurate predictions about the world
- Crony Beliefs
- beliefs that are not accurate representations of reality, but are kept around because of social value and signaling purposes
The important thing to understand is that Crony Beliefs pay rent too – but not in accurate predictions, but rather as entry and maintenance costs of social bonds.
This observation fits very well with the “bleeding-heart” approach to cognitive biases: while our reasoning is deeply flawed, most heuristics are understandable from the evolutionary point of view, actually work quite often (especially in the simpler ancestral environments) and, as Kevin adds, can even provide a lot of utility, albeit non-epistemic.
There are only few (minor) things I’d like to add to this picture.
Break-downs of meritocracy still exists
I, for one, typically explain my own misbeliefs (as well as those I see in others) as rationality errors, breakdowns of the meritocracy. But what I’m arguing here is that most of these misbeliefs are features, not bugs. What looks like a market failure is actually crony capitalism. What looks like irrationality is actually streamlined epistemic corruption.
My feeling is that despite this, true meritocracy breakdowns are still more common than true crony beliefs. While there are a LOT of social biases (authority bias, bias from social desirability, conformity bias, groupthink etc.), I think still most of the biases are simply heuristics extrapolating too far & breaking down, i.e. providing neither epistemic nor social value.
Bounded-rationality, compartmenalization and reflective equilibrium
AND ON TOP of that we have a bunch of beliefs that don’t have any causal links leading outside of our skulls, and beliefs that we have, but kind-of-sort-of don’t really know if they are useful in any way, because the slow combine-harvester of System 2 (or some other possibly non-conscious process) didn’t yet mowed over the belief and “decided” to integrate it, throw it away, or leave around unattached.
I’d really like to learn more about how this bounded belief examination and integration works.
Mindsets vs. beliefs
Finally a point that Julia Galef raised: beliefs are possibly not quite stable and the same metaphor could be applied to mindsets instead.
Instead of crony/meritocratic employees (beliefs), you might have a crony/meritocratic HR & hiring process.
Julia’s metaphor of scout vs. soldier mindset roughly maps to meritocratic vs. crony beliefs too.
Kevin’s article is definitely worth reading in its entirety, because it gives a very vivid metaphor to understand the interplay of the two types of beliefs.