Sep 24, 2017

A conundrum of esoterica

Harry Cutter: Every great magic trick consists of three parts or acts. The first part is called "The Pledge". The magician shows you something ordinary: a deck of cards, a bird or a man. He shows you this object. Perhaps he asks you to inspect it to see if it is indeed real, unaltered, normal. But of course... it probably isn't. The second act is called "The Turn". The magician takes the ordinary something and makes it do something extraordinary. Now you're looking for the secret... but you won't find it, because of course you're not really looking. You don't really want to know. You want to be fooled. But you wouldn't clap yet. Because making something disappear isn't enough; you have to bring it back. That's why every magic trick has a third act, the hardest part, the part we call "The Prestige"."

While some may view exposure of illusions and magic maneuvers as educational opportunities others may think it ruins the experience. This is a practice that brings lots of criticism and may even result in ostracizing of the professional from the community. These consequences are not isolated to the magic profession, it also happens in medicine. A physician who has received lots of criticism for revealing problems with medical practice and scientific research is John Ioannidis. His articles which include Why most published research findings are false (2005, PMID: 16060722) and Why Most Clinical Research Is Not Useful (2016, PMID: 27328301) challenge current approaches to scientific research and medical practice. In 2016 article titled Evidence-Based medicine has been hijacked: a report to David Sackett (PMID: 26934549) he describes some of his current concerns. He writes,

When I published a story in the Christmas BMJ on how physicians are treated by the pharmaceutical industry with free lunch vacation with full entertainment in the Arabian peninsula[9], a powerful politically-connected syndicalist doctor in Athens wrote to the medical society asking for my exemplary punishment and revocation of my medical license. He also attacked me personally at the board of directors of the national disease control center where I was vice president. He entered one day the board room and said that he cannot co-exist with a person of such exceptionally low moral standards. No one defended me, but eventually he did not have his way. I feel sorry that he had to co-exist with such a horrible person like myself.

However, things got far worse when EBM became more successful and recognized in many places beyond Canada. The same people who were previously spitting when mentioning “EBM”, started using the very same term to buttress their eminence-based medicine claims to prestige. Several senior people started to ask me to work with them, hoping that they would publish papers in major journals. Saying “no” and trying to stick to high standards for my work bought me even more enemies, including leaders of academia, politics (of the entire corrupted range of left-to-right spectrum) and academic politics. Even the syndicalist who had once tried to annihilate me re-approached me: “John, we all know that you are the best scientist in the country. Why don’t we work together? You know how successful I am.” He presented a long list of his power attributes and connections. The catalogue was stunningly impressive. Then he added: “the only thing that I lack is major publications in top impact journals. So, here is what we will do: I will give you power and you will put my name in major evidence-based publications.”

I hate having power, so obviously I declined. I have always preferred to work with the young and the powerless. But this made even more powerful people even angrier with me. A senior professor of cardiology told a friend of mine that I should not be too outspoken, otherwise Albanian hit men may strangle me in my office. I replied that they should make sure to get correct instructions to my office – turn left when they come up the stairs. I would feel remorse, if the assassins entered the wrong office and strangled the wrong person.

This type of aggressive behavior is not rare in the history of science where people have been ostracized, prosecuted, and burned at the stake for standing for truth and against dogmatic teachings. In the history of science knowledge acquisition and transparency have mostly been tightly controlled by a few as philosopher of science Janet D. Stemwedel explains,

One of the big ideas behind science is that careful observation of our world can bring us to knowledge about that world. This may seem really obvious, but it wasn't always so. Prior to the Renaissance, recognized routes to knowledge were few and far between: what was in sacred texts, or revealed by the deity (to the select few to whom the deity was revealing truths), or what was part of the stock of practical knowledge passed on by guilds (but only to other members of these guilds). If you couldn't get your hands on the sacred texts (and read them yourself), or have a revelation, or become a part of a guild, you had to depend on others for your knowledge.

The recognition that anyone with a reasonably well-functioning set of sense organs and with the capacity to reason could discover truths about the world -- cutting out the knowledge middleman, as it were -- was a radical, democratizing move.

Ioannidis' concerns remind us that some things about the social practice of knowledge acquisition and transparency have not changed. He also states that EBM as currently practiced is not what it was originally intended to be. The original goal of Evidence-Based medicine was to practice medicine under a scientific and skeptical attitude and de-emphasize the unsystematic approach to practice in order to improve care. Furthermore, Ioannidis adds that even re-training these days would have negative effects,

David, I was astonished by your sense of humility and self-knowledge when I heard that you decided to undergo residency training again to refresh your clinical skills when you were already a full professor. Several years ago, I decided not to practice medicine any longer. I might have caused more harm than good. I could not even think of remedying this by repeating training. Re-training on how medicine is practiced today might make me worse. In some settings, we are close or past the tipping point where medicine diminishes rather than improves well-being in our society. Some truly excellent and committed physicians certainly continue to make positive contributions to health, improve lives, and save lives. However, with 20% of GDP being spent on health and health care so inefficiently, with such limited evidence or with conflicted evidence, medicine and health care can become a major threat to health and well-being.

In a new article, How to survive the medical misinformation mess (PMID: 28881000), Ioannidis et al address some problems under the current paradigm,

The Medical Misinformation Mess comprises four key problems:

  1. Much published medical research is not reliable or is of uncertain reliability, offers no benefit to patients, or is not useful to decision makers.
  2. Most healthcare professionals are not aware of this problem.
  3. Even if they are aware of this problem, most healthcare professionals lack the skills necessary to evaluate the reliability and usefulness of medical evidence.
  4. Patients and families frequently lack relevant, accurate medical evidence and skilled guidance at the time of medical decision-making.

If an unsystematic training and educational assessments do not address these problems and physicians are practicing medicine without this knowledge, the best conclusion we can deduce from Ioannidis et al.'s findings may be found in Shtulman new book,

This tendency to overrate our understanding of natural phenomena has been labeled the illusion of explanatory depth. It is the illusion that our explanatory knowledge, as grounded in intuitive theories, runs much deeper than it actually does. This illusion has been documented in people of varying ages, from four-year-olds to forty-year-olds, and with varying levels of education, from minimal exposure to science to graduate-level training in science. It has even been documented in people who have significant firsthand experience with the domain under consideration (e.g., cycling experts asked to explain the mechanics of a bicycle). As someone who holds a PhD in psychology, I qualify as an expert in the field, yet I fall prey to the illusion of explanatory depth every time I prepare a new lecture on a psychological topic. I start my preparation convinced I know enough about the topic to occupy a full hour’s worth of class but soon discover I know only enough to occupy five minutes’ worth. My first year of teaching was one long foray into the illusion of explanatory depth.

Researchers who have studied this illusion have determined that it’s not just a matter of general overconfidence, on par with overconfidence in our driving ability or our financial investments. It’s specific to complex causal systems—systems with multiple causal pathways, multiple levels of analysis, nonvisible mechanisms, and indeterminate end states. Consequently, the illusion does not pertain to forms of knowledge that lack such properties, like knowledge of procedures (e.g., how to bake chocolate chip cookies) or knowledge of narratives (e.g., the plot of Star Wars). If you think you can bake chocolate chip cookies or recount the plot of Star Wars, you probably can.

Our knowledge of natural phenomena thus suffers on two fronts: from our limited ability to explain these phenomena (in accurate terms) and from our limited recognition of this limited ability. We are blind to our own blindness.

Shtulman, A., Scienceblind: Why our intuitive theories about the world are so often wrong, 2017

Answering assessment questions and giving explanations without having the necessary skills to assess the evidence and, furthermore, not being aware of problems with the evidence itself is making up stories that suffer from the illusion of explanatory depth. Explanations are evidence of our knowledge, Daniel Kahneman was asked about the concept of knowing,

Kahneman has now retired from academia but remains in demand as a consultant, advising large firms on decision-making. He is currently fascinated by this concept of “knowing”. “What does it mean to know something?” he asks, his eyes sparkling behind his glasses. “It has very little to do with actual evidence. It is usually when you have no alternative, when it is the only thing that comes to mind. This is especially true if it is anchored psychologically by the fact that other people you trust also believe in this thing. And it is only then that you invent reasons for it.”

Overconfidence and story cohesion do not necessarily correlate with validity of evidence, Kahneman adds,

The confidence people have in their beliefs is not a measure of the quality of evidence [but] of the coherence of the story that the mind has managed to construct. Quite often you can construct very good stories out of very little evidence. . . . People tend to have great belief, great faith in the stories that are based on very little evidence.

Life for the physician is a conundrum as the evidence is imperfect, but so is storytelling, especially with no evidence.

Uncle Monty: I have a feeling there are many things that you're going to see that you've never heard of before. Life is a conundrum of esoterica.

No comments:

Post a Comment

1. You should attempt to re-express your target’s position so clearly, vividly, and fairly that your target says, “Thanks, I wish I’d thought of putting it that way.
2. You should list any points of agreement (especially if they are not matters of general or widespread agreement).
3. You should mention anything you have learned from your target.
4. Only then are you permitted to say so much as a word of rebuttal or criticism.
Daniel Dennett, Intuition pumps and other tools for thinking.

Valid criticism is doing you a favor. - Carl Sagan