Natural Kinds, Evidence, and Randomness in Health Outcomes Research

Abstract

The body of medical knowledge upon which contemporary clinical practices are based is surprisingly (and disturbingly) frail. Very few systematic, ongoing attempts have been (or are being) made to collect information concerning patients' experiences following medical and surgical treatments. At present, medical knowledge within a given field derives mostly from relatively small numbers of disconnected clinical trials. One major reason for the near absence of efforts to collect systematic, ongoing outcome data is the persisting impression, fostered by most medical statisticians and epidemiologists, that such efforts cannot provide valid medical causal knowledge. Only randomised controlled trials, they claim, can provide valid knowledge. Clinicians' observations and experience do not even count as medical evidence on this view, but instead are seen as "mere opinion". This thesis examines the key concepts of natural kinds, medical evidence and randomness to determine whether and to what extent these traditional views are justified. The principal goal of health outcomes research is taken to be the identification (or specification) of natural kinds of patients, i.e., sets of patients who are disposed to respond in defined ways to specified treatments (e.g., the kind of patient who tends to benefit from radiation therapy for breast cancer). Natural kinds are based on the operation of natural laws, as manifested through the dispositional properties of individual members of each kind. Regarding evidence, all observations made in a critical and systematic manner are viewed as constituting evidence - irrespective of whether or not randomisation was employed in generating those observations. Evidence is of varying weight, however, and rules of evidence are proposed to take into account the philosophical notion of weight of evidence. Randomisation is determined to be a (merely) axiomatic, relational notion, with little or no epistemological value in the real world. Randomness applies, for example, to Statements like "if repeated random samples with replacement are made from a collection of black and white balls, in the long run the proportion of white (or black) balls sampled will closely approximate the proportion of white (or black) balls in that collection." But allocating patients to receive treatment or placebo according to the toss of a coin does not convert the (merely) axiomatic force of randomness into a warrant for believing the results of clinical trials. This central conclusion is based in part on an extensive analogy constructed between (1) the problem of intrinsic fertility slopes in agricultural experiments (in which some parts of a field are more intrinsically fertile than others) and (2) the differing propensities of patients to improve spontaneously without treatment. In both cases, the validity of causal inferences derived from study observations is threatened, and in both cases this threat is better controlled using balanced, rather than randomised, allocation designs. Certain statistical methods (i.e., regression analyses and recursive partitioning) are potentially useful for inferring taxonomies of patient kinds from observational data. These are discussed along with a host of related theoretical and practical issues that must be addressed in developing more robust empirical bases for clinical practice.