User:AS48/Popper's principles of demarcation

This article is concerned with the three principles that make up the criterion of demarcation between science and pseudoscience developed by philosopher Karl Popper. This article concentrates on the practical aspects of the problem of demarcation, as opposed to its philosophical aspects, which are discussed elsewhere (Karl Popper, Demarcation problem, Falsifiability).

It must be stressed from the start that, while associated largely with scientific theories, the principles of demarcation are useful in any domain: business, politics, social issues, and even personal affairs. These principles provide a simple and reliable method for determining whether a given activity is rational or whether it is a worthless pursuit. Thus, the term "theory" must be interpreted here as any idea, assertion, method, or claim. Popper himself recognized the value of his principles outside the domain of science. The principles, however, have become even more important in the last few decades. As our world is becoming more and more complex, we are increasingly facing situations – in matters of health, education, work, technology, politics, investments, purchases – where a decision must be made even while knowing little in a particular field. In these situations we are liable to fall victim to irrational thinking and fallacious arguments; for example, we are liable to assess an idea by its popularity or by its promotion, rather than by its usefulness. Popper's principles show us how to assess an idea logically, and help us to recognize mistaken ideas in any field.

Background
The criterion of demarcation is needed only for theories in the empirical sciences, which must be accepted or rejected through observation and experiment. These theories deal with real-world phenomena, and we accept a given theory if it explains and predicts events accurately.

The theories of mathematics and logic, on the other hand, do not have to reflect the real world, and can be proved analytically. These theories are part of artificial systems, which we deliberately design to be perfect (through deductive reasoning). Thus, once proved, we know that the theories will work in any future situation. They do not have to be tested against real-world phenomena, and do not require a criterion of demarcation.

The criterion of demarcation is needed in the empirical sciences in order to combat the deleterious effects of induction: if we decide that verifying a theory means applying it in real-world situations, we must face the fact that, no matter how many situations we tested, there will always remain an infinity of future, untested ones. When proposing a theory, then, we rely on induction, which is illogical: we draw conclusions about an infinite number of future events when all we have as evidence is a limited number of past events. So, to have theories at all, we must trust the concept of induction even as we know that it is illogical.

Pseudosciences and superstitions are also based on induction: if we noticed the position of the planets while a happy event took place, we may be tempted to plan our activities based on the future position of the planets; and if we noticed a black cat while a tragic event took place, we may decide to avoid black cats in the future. Because we rely on inductive reasoning for scientific theories as much as for pseudosciences and superstitions, we need a way to differentiate between the two types of theories; namely, a way to separate useful theories from worthless ones. This is what the principles of demarcation aim to do.

According to Popper, we must accept the fact that in the empirical sciences (and hence for any idea involving the real world) it is impossible to prove a theory in an absolute sense, as in mathematics or logic. The only way to advance, therefore, is through a process of trial and error, of conjectures and refutations. Since we can never be sure that our current knowledge is correct or complete, we must treat our theories as mere conjectures: we must doubt them, do our best to refute them, and continue to use them as long as we fail to refute them. We accept theories, therefore, not because we proved their validity (which is impossible), but because we failed to prove their falsity. Thus, scientists doubt and attack their theory; pseudoscientists think their task is to defend their theory.

The three principles of demarcation are as follows:


 * A theory must be tested by looking for falsifications of its claims; it is unscientific to attempt to verify a theory by looking for confirmations of its claims. Thus, we must accept a theory, not when we find situations that confirm it, but when we fail to find situations that falsify it.


 * A theory must be falsifiable. Someone who proposes a new theory must specify at the same time certain situations which, if encountered, would be seen as falsifications of the theory. A theory that is unfalsifiable in any circumstances is unscientific.


 * It is unscientific to modify a theory if the modification aims to annul a falsification of its claims. In other words, if we encounter a situation that falsifies the theory, we must not relax the theory's claims so as to make that situation acceptable. Instead, we must consider the theory refuted.

The three principles are closely related and support each other. Thus, we could not apply the first one (looking for falsifications) without the second one (there must exist falsifying situations). And the second one would be meaningless without the third one (the theory must not be relaxed so as to annul a falsification); for, by modifying it again and again to cope with each new falsification, a theory that is obviously invalid could be made to appear successful. The three principles are discussed in greater detail in the following sections.

Analysis
Popper's interest in the idea of demarcation started in his youth, when he became suspicious of various theories that claimed the status of empirical sciences; in particular, Freud's and Adler's psychological theories and Marx's political theory. Not only were these doubtful theories seen as scientific, but they appeared to be extremely successful. Thus, a psychoanalyst could explain through either Freud's or Adler's theory any act performed by any person; and a Marxist could explain through Marx's theory any social or political event.

While believers were impressed by the fact that these theories always seemed to work, Popper recognized that this was in fact their weakness: "But were these theories testable?... What conceivable event would falsify them in the eyes of their adherents? Was not every conceivable event a 'verification'?... It began to dawn on me that this apparent strength was in fact a weakness.... The method of looking for verifications seemed to me unsound – indeed, it seemed to me to be the typical method of a pseudoscience. I realized the need for distinguishing this method as clearly as possible from that other method – the method of testing a theory as severely as we can – that is, the method of criticism, the method of looking for falsifying instances."

Popper also recognized that the problem of demarcation is related to the problem of induction, previously mentioned: "If, as I have suggested, the problem of induction is only an instance or facet of the problem of demarcation, then the solution to the problem of demarcation must provide us with a solution to the problem of induction." In other words, the same method that would allow us to distinguish between scientific and pseudoscientific theories should also allow us to combat the risks of inductive reasoning.

Popper's solution is based on combining the methods of induction with those of deduction. Unlike induction, deduction can prove the validity of a statement – by showing that it can be derived logically from statements already known to be valid. So we can continue to use induction in order to discover a theory, but rely on deduction in our attempts to refute it. The inductive part is uncertain, because we cannot test the theory in an infinite number of situations; but the deductive part is certain, and we only need it in a single situation. Like induction, deduction cannot prove that an empirical theory is valid; but it can prove that it is invalid.

Here is how Popper explains this concept: "My proposal is based upon an asymmetry between verifiability and falsifiability; an asymmetry which results from the logical form of universal statements. For these are never derivable from singular statements, but can be contradicted by singular statements. Consequently it is possible by means of purely deductive inferences (with the help of the modus tollens of classical logic) to argue from the truth of singular statements to the falsity of universal statements." (Modus tollens is based on deduction.) In other words, if P stands for any one of the assertions that make up a theory, and Q stands for any one of the conclusions derived from this theory, then just one instance of Q being false suffices to refute the theory.

One of Popper's favourite examples is the universal statement "all swans are white." No matter how many white swans we observe, these confirmations do not verify the statement, for we can never be sure that we saw all the swans in the world; but observing just one black swan suffices to refute the statement. In general, it is pointless to design tests that attempt to confirm a theory; for, no matter how scrupulous the tests are, each confirmation does not increase significantly the likelihood that the theory is valid. Instead, we must design tests that attempt to falsify the theory. Logically, we can learn little or nothing from any number of instances that confirm the theory, but we can learn a great deal from just one instance that falsifies it.

Thus, Popper's solution has rescued the principle of empiricism – the requirement that theories be accepted or rejected on the basis of observations and experiments – from the deleterious effects of induction. The price we pay is having to give up the popular ideas that we must look for confirmations and that we must accept a theory on the basis of confirming evidence. Instead, we must adopt a less intuitive method: we must look for falsifications, and reject the theory on the basis of refuting evidence.

If theories must be judged by subjecting them to tests that try to falsify them, it follows that a theory is scientific only if there exist tests which, if successful, would falsify it. Pseudoscientific theories, on the other hand, can never be falsified by tests; so they are, in effect, untestable, and hence undependable. This is how Popper puts it: "Not the verifiability but the falsifiability of a system is to be taken as a criterion of demarcation.... I shall require that its logical form shall be such that it can be singled out, by means of empirical tests, in a negative sense; it must be possible for an empirical scientific system to be refuted by experience." Moreover, "criteria of refutation have to be laid down beforehand; it must be agreed which observable situations, if actually observed, mean that the theory is refuted." This will prevent us later from deliberately avoiding tests known to falsify the theory.

Pseudoscientific theories
If we bear in mind the first two principles of demarcation (a theory must be falsifiable, and it must be tested by looking for falsifications, not for confirmations), it is not difficult to recognize pseudoscientific theories. For, their defenders cover up their failure by disregarding the two principles.

The simplest way to defend a pseudoscientific theory is by looking for confirmations: "It is easy to obtain confirmations, or verifications, for nearly every theory – if we look for confirmations." Since we intuitively interpret confirmations as evidence of a theory's validity, we are fooled by the depiction of successes, failing to see that its promoters deliberately restrict themselves to those situations where the theory works. Thus, although crude, noting the few confirming situations and ignoring the many falsifying ones is an effective way to make an invalid theory appear successful.

A slightly more sophisticated stratagem is to make the theory entirely unfalsifiable, from the start. This is done by keeping its predictions so vague and ambiguous that any event appears to confirm it. Such a theory is, in effect, untestable. The fact that it cannot be tested, and therefore is never falsified, makes the theory look successful; but this success is an illusion, because it is achieved by avoiding tests, not by passing tests. Thus, "a theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice."

Popper uses astrology and the psychoanalytic theories of Freud and Adler as examples of theories that were unfalsifiable from the start: "Astrologers were greatly impressed, and misled, by what they believed to be confirming evidence.... Moreover, by making their interpretations and prophesies sufficiently vague they were able to explain away anything that might have been a refutation of the theory had the theory and the prophesies been more precise. In order to escape falsification they destroyed the testability of their theory."

As for the two psychoanalytic theories, "they were simply non-testable, irrefutable. There was no conceivable human behaviour which could contradict them.... Those 'clinical observations' which analysts naively believe confirm their theory cannot do this any more than the daily confirmations which astrologers find in their practice.... These theories describe some facts, but in the manner of myths. They contain most interesting psychological suggestions, but not in a testable form."

The most sophisticated stratagem is to start with a proper, falsifiable theory, and then make it unfalsifiable by repeatedly modifying it. Whenever faced with a falsifying situation, instead of admitting that the theory was refuted, its defenders expand it by making that situation look like a legitimate part of it. In other words, they cover up the falsifications of the theory by turning them into new features of the theory. Since they deal with one falsification at a time, and since the changes look like improvements, the general degradation of the theory may go unnoticed. But modifications that relax a theory's claims are not improvements; on the contrary, these modifications make the theory less and less rigorous, and hence less and less useful.

Thus, while the theory remains falsifiable in principle, it is rendered unfalsifiable in fact. To a casual observer the theory looks just like the serious, scientific theories. It differs from them only briefly, when a falsification is discovered; at that point it expands so as to permit that situation, thus eliminating the threat. Then it appears again to be a serious theory – until threatened by another falsifying situation, when the same trick is repeated. Clearly, a theory that is repeatedly expanded so as to permit every falsifying situation becomes eventually worthless, no different from those theories which are unfalsifiable from the start.

Popper uses Marxism as an example of theories that start by being falsifiable but are later modified by their defenders in order to escape refutation. Some of Marx's ideas were serious studies of social history, but they were falsified by subsequent events. The theory, thus, was refuted. "Yet instead of accepting the refutations the followers of Marx re-interpreted both the theory and the evidence in order to make them agree. In this way they rescued the theory from refutation; but they did so at the price of adopting a device which made it irrefutable ... and by this stratagem they destroyed its much advertised claim to scientific status."

To prevent this method of covering up falsifications, Popper added the third principle to his criterion of demarcation: a theory, once formulated, cannot be modified if the modification aims to make it correspond to the reality that would have otherwise refuted it. We must admit that the theory was refuted, and treat the modified version as a new theory: "We decide that if our system is threatened we will never save it by any kind of [stratagem].... We should agree that, whenever we find that a system has been rescued by a [stratagem], we shall test it afresh, and reject it, as circumstances may require."

We do not usually need to abandon the theory, of course; we merely declare the original version to be refuted and the modified one to be a new theory. While this appears similar to simply modifying the theory, in practice there is a big difference between the two concepts. By proposing a new theory for each modification, we state in effect that we take each falsification seriously. After doing this several times, it is difficult to delude ourselves that our project is sound. On the other hand, if we perceive each modification as an enhancement of the original theory, we may discover falsifications forever and continue to claim that the theory is valid, that all we are doing is to improve it.

To summarize, here are the differences – according to Popper's principles of demarcation – between scientists and pseudoscientists; or, in general, between individuals who sincerely attempt to determine the value of an idea, and those who simply adhere to an idea, whether or not it is valid.


 * Scientists doubt their theory, and use all available knowledge in attempts to refute it; they search for falsifications of the theory. Pseudoscientists defend their theory, and their work consists in attempts to show that it is valid; they search for confirmations of the theory.


 * Scientists formulate their theory so as to allow only a narrow range of situations that count as confirmations; the theory, therefore, has great practical value. ("Every 'good' scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is." ) Pseudoscientists formulate their theory so as to allow a very broad and poorly defined range of situations that count as confirmations; thus, since there are hardly any situations that count as falsifications, the theory has little practical value.


 * More commonly, the pseudoscientists start, just like the scientists, with a narrow range of situations that count as confirmations. But in order to save their theory later, they expand that range by adding, one by one, those situations that originally counted as falsifications. They keep reducing, therefore, the theory's practical value.

Examples

 * The simplest example of pseudoscientific methods is found in promotional work like advertising and public relations. Typically, we are asked to assess the merits of a product by reading a few testimonials, or success stories, or case studies; in other words, by studying confirmations of its usefulness. A product is like an empirical theory, in that certain claims are being made about its usefulness – claims which must be verified through tests. According to Popper's principles of demarcation, therefore, the promoters ought to show us tests that attempt to falsify those claims, not tests that confirm them. Intuitively, we tend to search for success stories; but we must resist this, and remember that we can learn a lot more about a product's usefulness by studying the falsifications of those claims. The theory about the product's usefulness is falsifiable, and hence scientific; it is its promoters, when ignoring the falsifications, that render it pseudoscientific.


 * A similar fallacy occurs when books, articles, or documentaries try to help us by describing a few successful applications of an idea. We are shown, for example, how certain individuals were successful in real estate, or in finding a job, or in losing weight; or how certain companies were successful following one management concept or another, or using one software system or another. Again, if what we need to know is how useful those ideas really are, we ought to be shown falsifications, not confirmations. It is difficult to convince people who seek advice, as much as those eager to give advice, that they should ignore the successes and study instead the failures. But according to Popper's principles of demarcation, this is the only logical way to determine whether a certain idea is useful or useless. Specifically, we must adopt the idea, not because we find confirmations, but if we fail to find falsifications. In practice, the successes are irrelevant because they may be due to some unusual or subtle conditions. In most cases, no one, not even the individuals involved, is aware of all the conditions that contributed to a success.


 * The popular economic theories provide another example of pseudoscientific thinking. Every day we see in the media predictions about the future of such variables as the stock market, inflation, or a particular commodity or currency. An expert, for instance, may list a number of reasons why the stock market is likely to go up in the following year. Then, we are told that if certain events will occur, the stock market may actually go down. While appearing important and informative, such statements are in fact meaningless, because there are no conceivable events that could falsify them. This is an example of a theory that is pseudoscientific because it is unfalsifiable from the start: no matter what happens in the future, the expert will appear to have predicted it. Because the range of situations that count as confirmations is so broad, the theory has no practical value.


 * It has been noted that the mechanistic theories of mind and society are pseudoscientific according to Popper's principles of demarcation. These theories start with the assumption that human phenomena can be explained with precision, just like the phenomena studied by physics. Then, in the face of falsifying evidence, instead of doubting the theory its supporters keep defending it and modifying it, contrary to the principles of demarcation.


 * The theory of structuralism claimed that all aspects of culture can be explained by performing various transformations on some basic binary concepts (good/bad, left/right, male/female, day/night, etc.). However, both the binary concepts and the transformations were vague, expandable notions, which could easily be employed to describe any tradition, piece of folklore, work of art, social institution, etc. The theory was, therefore, unfalsifiable: no aspect of culture exists that it could not explain.


 * The theory of behaviourism claimed that all human acts can be precisely explained and predicted by reducing them to combinations of some elementary stimulus-response units. This idea was falsified even in simple experiments with animals, but instead of abandoning it, the behaviourists modified it again and again by relaxing its original claims, until the idea became so vague that it appeared to explain almost any human act.


 * The theory of universal grammar claimed that the human capacity for language is an innate faculty, which can be precisely explained, along with all natural languages, using mathematical models. In reality, the use of language cannot be separated from the other knowledge present in the mind, so no exact model can exist for the phenomenon of language. But instead of admitting that the theory was refuted, its supporters kept modifying it in order to deal with the falsifications. In the end, the idea of an exact model for the phenomenon of language became unfalsifiable.


 * It has also been noted that the theories of software engineering are pseudoscientific according to Popper's principles of demarcation. These theories start with the assumption that software applications can be developed by emulating the methods of manufacturing; that is, by designing a software system as a perfect hierarchical structure of constructs and modules, just as an appliance is designed as a hierarchical structures of parts and subassemblies. But this idea is fallacious, because the parts of a software application are related through many structures, not one. So the theories were refuted: applications could not be developed by strictly following the idea of software engineering. But instead of recognizing this failure, the theorists responded by adding features whose purpose was to bypass the restriction to a single structure. In other words, contrary to the principles of demarcation, they attempted to rescue the theories by expanding them so as to permit those situations that had originally been defined as falsifications. The benefits promised by these theories could be achieved only if applications were restricted to a single structure, just as originally claimed. Thus, by relaxing the claims and permitting multiple, interacting structures, their defenders increased the range of confirmations to the point where the theories, while appearing to work, had lost in fact the promised benefits.


 * The theory of structured programming claimed that applications can be represented as a hierarchical structure of three standard flow-control constructs, using various transformations to turn all other constructs into standard ones. This idea was falsified even by simple applications. But instead of abandoning it, its promoters expanded the theory by permitting any non-standard constructs that were useful. The theory was saved by replacing its formal, exact principles with some vague guidelines, at which point many programming styles that were in fact falsifications appeared to conform to the idea of structured programming.


 * The theory of object-oriented programming claimed that applications can be represented as software "objects" that are linked strictly through hierarchical relations. When this idea was falsified, the theory was saved by expanding it so as to allow additional relationships between objects (links between different hierarchies, the use of traditional programming concepts, etc.). Thus, what had been falsifications of the theory became important features of the theory.


 * The theory behind the relational database model claimed that if the database is restricted to "normalized" files, we will be able to represent the database structures and operations with a formal, mathematical system (which is the same as saying that they can be represented as a strict hierarchy). This idea was falsified in practice, and the relational model was saved by inventing new features; for example, by annulling the restriction to normalized files and by introducing programming languages that allow us to override the formal database operations. Every feature added to the model was needed only in order to cover up a falsification of the model.