This has been impressed on me particularly by my service on the Council on Pharmacy and Chemistry. In the course of its work of passing on the claims advanced for commercial remedies, this Council is forced to inquire critically into the basis of the claims of manufacturers.
It is interesting to note the qualitative differences in the evidence for the various kinds of claims: The chemical data are usually presented in such a form that it is possible to tell at a glance whether or not they are based on demonstrated facts, which could usually be verified or refuted without special difficulty. The deductions are usually such as can be legitimately drawn from the data, or else they are obviously absurd. All this agrees with the relatively exact status of chemical science.
In passing to data and deductions from animal experiments, a distinct change is noticeable: Not only are the data less reliable, and less worthy of confidence, but they are more often stated in a less straightforward manner. The presentation of the data often shows evidence of manipulation of the results, so as to make them most favorable to a preconceived conclusion that would recommend the drug. This is not always intentional, but is partly due to the less exact nature of animal experimentation, which leaves a wider play to the arbitrary interpretation of the reporter. A certain amount of this is unavoidable. No serious objection can be raised, provided the experimenter presents all the essential data, and discusses fairly all of the interpretations that would apply to them.
On the whole, it is usually possible to form a fairly definite estimate of the value of experimental data.
When one comes to the clinical evidence, an entirely different atmosphere obtains. When the Council demands evidence of the usefulness of a remedy, the manufacturers generally respond with every sign of enthusiasm. They may have ready a series of articles already published, or they instruct their agents to bring in letters from physicians. The last method seems to meet the most cordial response, judging from the deluge of letters and opinions that floods the Council.
The quality of the published papers is a fair reflection of the deficiencies of what is still the common type of clinical evidence. A little thought suffices to show that the greater part cannot be taken as serious evidence at all. Some of the data are merely impressions—usually the latest impressions of an impressionable enthusiast—the type of man who does not consider it necessary to present evidence for his own opinions; the type of man who does not even realize that scientific conclusions must be based on objective phenomena.
Some of the papers masquerade as “clinical reports,” sometimes with a splendid disregard for all details that could enable one to judge of their value and bearing, sometimes with the most tedious presentation of all sorts of routine observations that have no relation to the problem.
The majority of reports obtained by the agents belong to these classes, notwithstanding the fact that they are often written for the special use of the Council, and therefore with the realization that they are likely to be subjected to a thorough examination, and therefore presumably representing the best type of work of which the reporter is capable. So, at least, one would suppose.
It is also possible, however, that some of these reports are written merely out of thoughtlessness, or perhaps often to get rid of an importunate agent. This is illustrated by the following correspondence, taken literally from the files of the Council.
A letter from a prominent physician “A,” endorsing a certain preparation “D,” having been submitted to the Council, the secretary was directed to write to Dr. A as follows: