The logic of causal inference

Kenneth Rothman gives a great overview of causal inference in the introduction of this 1988 book, here's an excerpt:

Inferring Causal Connections—Habit, Faith or Logic?

An understanding of the process of causal inference is often muddled by differing concepts of the term "inference" in empirical science, a problem that may reflect the uncertainty of the process itself. What passes for causal inference by scientists is often just decision-making perched upon weak criteria that lack a logical base. Many of the commonly used modes of causal inference are fallacious, their popularity notwithstanding. [...] The difficulty with this method is that consensus is no guarantee of correctness, as has been historically demonstrated in many instances. For one thing, consensus is temporary, and can change. Were consensus a correct basis for inference, then a once flat earth must have become spherical, and the thymus gland must have been physiologically inert until a couple of decades ago when it suddenly developed a useful role in the immune system. For another thing, consensus itself requires no further justification, and may be based on shared beliefs that are irrational. When One Hundred Authors Against Einstein, a collection of essays by 100 physicists attempting to discredit relativity theory, was published in 1930, Einstein reputedly responded to a reporter’s query about the book with the remark: "Were my theory wrong, it would have taken but one person to show it.” Similarly, it would require but one mathematician to point out a flaw in a deductive proof, despite consensus of other mathematicians that the proof is correct.

Another faulty approach to inference is to infer by "appeal to authority." While it would hardly seem to need saying that the seal of approval from a noted authority does not provide any logical basis for inference, in everyday practice many inferences are premised on little but the credentials of the theory's leading exponent. The flip side of an appeal to authority is the ad hominem criticism, which disallows an inferential argument simply because of the imagined lack of necessary credentials, or presumed biases, of the person who presents it. Of course, on reflection it seems obvious that an inference should stand or fall by virtue of its logic and the relevant data, and not by virtue of the imputed viewpoint, philosophy, personal biases, heritage or other interests of the person who puts forth the argument. As straightforward as this principle may seem, a trend in the biomedical literature is to encourage ad hominem evaluation by judging a work according to the affiliation or funding source of the researchers, rather than judging it by their findings or arguments. [...]

Another fallacy in causal inference is the fallacy of post hoc ergo propter hoc, which is Latin for "after this therefore on-account-of this.” It is a fallacy to infer that the rooster causes the sun to rise each morning by crowing at dawn, despite multiple repetitions of the same sequence of events. Although this fallacy is well recognized, the extension of a series of events observed repeatedly into a law of nature appears to be the backbone of the inductive method. Induction is still touted because it often seems to work. The difficulty is that in principle there is nothing behind the so-called logic of it but for the post hoc fallacy. Russell summarized the fallacy in this way:

We have . . . the ordinary procedure of induction: ’If p, then q; now q is true; therefore p is true.’ E. g., ’If pigs have wings, then some winged animals are good to eat; now some winged animals are good to eat; therefore pigs have wings.’ This form of inference is called 'scientific method.'⁴

How can we differentiate between instances in which an inference is correct from the instances where it is not? The positivists, or verificationists, some of whose views are represented in this volume, are philosophers who, notwithstanding Hume's criticism of induction, propose specific criteria or conditions intended to strengthen the validity of an inductive inference. The fallibilists, or falsificationists, consider inductive inference impossible, that is, they accept Hume's criticism. Popper is the most notable falsificationist. Popper’s answer to the problem of distinguishing between possibly correct and incorrect scientific statements is that real progress can be made only by attempting to disprove the statements. If you wring the rooster’s neck and yet the sun rises, it follows that the rooster's crowing was not a necessary cause for the sun to rise. If a hypothesis is not disproved despite attempts to do so, it is neither proved nor disproved, but remains in conjectural limbo, subject to being disproved or modified at some later date. This view of science is one of proceeding by trial and error, or, as Popper described it, conjecture and refutation. Under this scheme there is no need for induction, which Popper dismissed as an illusion. A process of trial and error could account for my dog’s behavior in learning to ring my doorbell, just as it could account for the evolution of scientific theories. The most important implication of the conjecture-and-refutation philosophy for scientific practice and thought is its focus on disproof. [...] The point of this argument is that the fallibilists propose that science can proceed more swiftly to discard erroneous theories and replace them with better ones by actively seeking to refute theories, rather than by fruitless accumulations of "supporting observations."

Naturally, Popper’s claim that induction does not exist has tended to rile those who believe that we all use induction every day to negotiate through every aspect of our environment. The antinomy of ideas here may put them beyond synthesis, although some readers will undoubtedly wish to reconcile these opposing schools of thought enough to avoid paralysis of all inferential processes. To do so fully appears impossible, but there may be at least enough room for agreement to console the anxious reader. Certainly everyone seems to agree that empirical science does not provide irrefutable statements about nature. Inductivists believe that, post hoc ergo propter hoc notwithstanding, useful scientific statements can be drawn from repeated observations. Fallibilists propose that the logical role of observations is only to refute theories. For fallibilists, however, there are no restrictions on the origin of conjectural knowledge. Conjectures are not to be judged by their mode of production but only by how well they stand up to the test of experience. One way to begin reconciling the views of inductivists with those of fallibilists is to consider induction, even if nothing but a psychological illusion, to be a process that generates conjectures. Since it is legitimate for theories to derive even from dreams (and some successful ones reportedly arose from dreams), why not from induction, which even if only an illusion is at least predicated on observations? It is sometimes claimed that induction is necessary even to apply yesterday’s accepted theories to today's experiences, but the process is nevertheless inescapably uncertain. For some theories, no matter how cherished, the day arrives when they no longer can be accepted. When the day comes, we need not consider the overthrow of the theory to be a logical failure of induction if we consider induction to be a road to theory formation, rather than evaluation. Of course, this view of induction sidesteps the question of whether induction can justifiably provide empirical support for a theory.

Some twentieth century philosophers of science, most notably Thomas Kuhn, have emphasized the role of the scientific community in determining the validity of scientific theories.⁶ Critics of the fallibilist doctrine have suggested that the refutation of a theory amounts to a choice between the validity of the theory and the validity of the scientific infrastructure of theories on which the refuting observation is based. Kuhn and others have argued that the scientific community determines what is to be considered accepted and what is to be considered refuted. Kuhn’s critics in turn have characterized this description of science as one of an irrational process, "a matter for mob psychology.”⁷ Those who cling to a belief in a rational structure for science consider Kuhn’s vision to be a regrettably real description of much of what passes for scientific activity, but not prescriptive for any good science. Nevertheless, the idea that causal inference is a sociological process, whether considered descriptive or normative, is an interesting thesis that has fostered sharp debate between subjectivists and objectivists. The view that the accumulation of scientific knowledge is inherently subjective is also a tenet of the Bayesian outlook that is increasingly popular among statisticians. Subjectivism in scientific philosophy is predicated on the doctrine that the concept of truth is not merely elusive but also relative, at least in the sense that scientific truths either correspond to the beliefs of scientists or are built upon beliefs. Beliefs, of course, need not be rational and change with time, thus puncturing the concept of any objective truth. Critics of subjectivism maintain that objective knowledge exists independently of beliefs, which play no part in determining the correctness of an inference. Indeed, inference by consensus or by ad hominem judgment is logically fallacious because it involves belief rather than logic.

Rothman, K., Causal inference, 1988

Be careful how you draw your conclusion.

Tweet to @jvrbntz