Goldman Conjecture: How do I know who to believe?
I’ve just finished reading the book (well, technically, listening to it in my car), and it’s a reasonable, Skeptical discussion of philosophy of science and epistemology. Massimo Pigliucci has three doctorates (genetics, botany and philosophy of science), and writes clearly about the issues, ranging from the demarcation problem (what is science and what isn’t?) to how science works.
However, it was his penultimate chapter on expertise which promoted me to write this post. In it, he advocates five principles which are adapted from Alvin Goldman’s Experts: Which Ones Should You Trust?. Here they are (paraphrased):
- What is quality of the arguments presented by the experts (eg. are they citing evidence or using fallacious arguments)?
- Do the arguments agree with the arguments and evidence presented by other experts in the field (eg. do they agree with the consensus)?
- What recognition of expertise does the person have (eg. do they have a relevant academic qualification or other recognition)?
- What are the biases affecting of the experts, and how do they related to the positions they are espousing (eg. if they’re researching drugs, are they funded by pharmaceutic companies)?
- What is the experts success rate like (eg. do they have regular/recent/relevant peer-reviewed research published)?
In the book, Pigliucci compares an advocate of creationism with an advocate of evolution, and unsurprisingly his conclusion is that the evolutionist is more worthy of trust than the creationist. He also uses the criteria against a single person – Deepak Chopra – and concludes that he falls at the first hurdle: his arguments aren’t really arguments at all.
My problem with the first this is the context in which such assessment is supposed to occur. Outside of a few professionals, who is expert on assessing and weighing evidence? The fact is that most debates in the public sphere are decided on the basis more of effective rhetoric than expert application and assessment of inductive and deductive logical patterns.
The second assumes that people have access to the information necessary to assess the quality of the evidence, and assumes that it is necessarily possible to identify consensus. To Pigliucci’s credit, he does point out that consensus can be wrong, but I’d question whether developing – or even developed – science is quite so amenable to objective analysis, not least because information may be hidden behind paywalls, technical language or mere obscurity. It also assumes a desire to check this information.
[edit: added this paragraph at 2115] It also occurs to me that this requires you to know who is an expert in order to be able to identify who is an expect. Seem like this could result in a recursive loop in which every expert can only be an expert when the comparator expert has also assessed!
The third assumes that most people know what is, or is not, relevant to the issues at hand. For example, which type of biologist should I ask about evolution? Why is a biologist not the right person to talk about abiogenesis?
The fourth, it seems to me, can be problematic on both sides of the equation and requires a subjective decision as to which biases should be included and which should not (for example, is a persons religious affiliation relevant to a discussion on the development of life?).
Finally, the fifth has the same weaknesses as the second, and in addition assumes that people have the ability to assess the relative success of different papers and publications in the knowledge ecosystem.
Although Pigliucci’s writing is a good introduction to epistemology and sciphi, it very much preaches to the choir. His clear disdain of post-modern thinking is amusing (but doesn’t seem to have evolved much since Sokal), but I would hope – as Pigliucci himself encourages – that readers treat his conclusions with some, dare I say it, skepticism. After all, the people who most need to think critically are the people least likely to read it…