Is Cosmetic Science Really "Bad"? Part II: Detecting Baloney Science

By: Johann W. Wiechers, PhD, JW Solutions

Posted: August 25, 2009

page 2 of 5

Shermer has published his ten characteristics more than once. This column employs the original questions he used to characterize the types of science in his November and December 2001 columns in Scientific American. Here, the questions are listed as originally proposed, along with Shermer's reasons for asking them and an explanation of their relation to cosmetic science. Some will be more applicable than others but after having addressed these questions it should become clear whether Goldacre was right or wrong when he stated that cosmetic science was bad science. The ultimate answer will not be revealed in this column and the cosmetics industry is in store for a couple of surprises. For those interested in Shermer's methodology, a video of Shermer's kit can be viewed online. 

Detecting Baloney Science
1. The first question to ask to detect baloney science is, “How reliable is the source of the claim?” This question translates into, “Would you believe a snake oil supplier, or a reputed scientist?” According to Shermer, “pseudo-scientists often appear quite reliable but when examined closely, the facts and figures they cite are distorted, taken out of context or occasionally even fabricated.”

Almost everybody will recognize this when (s)he is contemplating buying a new car. Unrealistic arguments such as, “People will stare at you in envy for driving that make and model!” are used to sell the goods. One statement like this might not be too bad, but a string of such silly arguments forces the consumer to not take the claims seriously, and they become uninterested. In science, scientists often use little tricks to portray their data as better than it really is and unfortunately, it happens more frequently in cosmetics than this author would like.

For example, a scientist may express the variance in data as the standard error of the mean, rather than the commonly agreed standard deviation. As the former is the latter divided by the square root of the number of observations, the former is always smaller and looks better. It may be scientifically correct but it is not in line with common rules in science. As a real example, this author reviewed a particular paper where something was deemed to be significant at a p = 0.5 level, which is as significant as flipping a coin. While this is scientifically correct, it is even more scientifically inappropriate. Another example is when a scientist expresses improvement as a percentage of the baseline--it always looks more impressive but it loses any meaning. Size does matter, you know!

In many TV advertisements, the consumer will witness a slightly graying medical doctor in a white lab coat taking off his (the actors are most often male) glasses and donating some of his precious time to inform about the benefits of the product being sold. Also note that the background color in many cosmetic ads is white, a color that portrays a crisp, clean scientific image. Cosmetic scientists and formulators act in similar ways to portray a scientific image--notice that papers typically cited in research papers give the impression that the researchers have read every issue of journals like Nature, Science and the Journal of Investigative Dermatology cover-to-cover. These are all attempts to increase the scientific credibility of what is written. This is not wrong, but the industry knows that the cosmetic relevance of papers published in Science and Nature is limited.