Is Cosmetic Science Really "Bad"? Part II: Detecting Baloney Science

Reactions from the first discussion in this series regarding whether cosmetic science is really bad indicate that this subject must be a tender point for many in the cosmetics industry. The first column, however, only discussed this author's personal frustrations (and apparently those of other industry professionals, too, based on some reactions) of cosmetic scientists being negatively portrayed. In the end, the question whether or not cosmetic science is really bad was never answered.

As readers may recall, in the first column of this series, the popular science writer Ben Goldacre dedicated an entire chapter in his book Bad Science to cosmetics, suggesting that cosmetic formulators and the industry as a whole are spreading lies because products do not deliver on their label claims. Although he has a valid point since such products do indeed exist, Goldacre goes so far as to claim that cosmetic science is "bad science.” It may make one wonder: How close is Goldacre to the truth if even this author admits he has a valid point? To examine this, it is important to know what "good science" is and how to differentiate it from bad science.

Enter Michael Shermer, the monthly columnist of Scientific American and founder of the Skeptics Society in the United States. Shermer has identified ten characteristics that can be used to differentiate between real science, borderline science and nonsense in his “Baloney Detection Kit.” The next three columns, starting with this one, will apply this kit to cosmetic science.

In his book, The Borderlands of Science–Where Sense Meets Nonsense, Shermer differentiates between real science, borderline science and just plain nonsense. Interestingly, while one would think the boundaries between the types of science are well-defined, they in fact are not. For example, humans initially thought the world was flat and the center of the universe. When Copernicus discovered this could not be the case, he got into great trouble with the church for telling such nonsense. But gradually, his concept moved from nonsense, to borderline science, to real science.

Public opinion and religion often influence the perception of what good science is, and for a scientist like Shermer, this is unacceptable; this led him to identify ten characteristics that will now be applied to cosmetic science. Cosmetic science can therefore be identified as one of the following: real science like chemistry or physics; borderline science like the search for extraterrestrial intelligence (SETI); or pseudo-science, i.e., not science at all--like ufology or the science of investigating unidentified flying objects.

Shermer has published his ten characteristics more than once. This column employs the original questions he used to characterize the types of science in his November and December 2001 columns in Scientific American. Here, the questions are listed as originally proposed, along with Shermer's reasons for asking them and an explanation of their relation to cosmetic science. Some will be more applicable than others but after having addressed these questions it should become clear whether Goldacre was right or wrong when he stated that cosmetic science was bad science. The ultimate answer will not be revealed in this column and the cosmetics industry is in store for a couple of surprises. For those interested in Shermer's methodology, a video of Shermer's kit can be viewed online. 

Detecting Baloney Science
1. The first question to ask to detect baloney science is, “How reliable is the source of the claim?” This question translates into, “Would you believe a snake oil supplier, or a reputed scientist?” According to Shermer, “pseudo-scientists often appear quite reliable but when examined closely, the facts and figures they cite are distorted, taken out of context or occasionally even fabricated.”

Almost everybody will recognize this when (s)he is contemplating buying a new car. Unrealistic arguments such as, “People will stare at you in envy for driving that make and model!” are used to sell the goods. One statement like this might not be too bad, but a string of such silly arguments forces the consumer to not take the claims seriously, and they become uninterested. In science, scientists often use little tricks to portray their data as better than it really is and unfortunately, it happens more frequently in cosmetics than this author would like.

For example, a scientist may express the variance in data as the standard error of the mean, rather than the commonly agreed standard deviation. As the former is the latter divided by the square root of the number of observations, the former is always smaller and looks better. It may be scientifically correct but it is not in line with common rules in science. As a real example, this author reviewed a particular paper where something was deemed to be significant at a p = 0.5 level, which is as significant as flipping a coin. While this is scientifically correct, it is even more scientifically inappropriate. Another example is when a scientist expresses improvement as a percentage of the baseline--it always looks more impressive but it loses any meaning. Size does matter, you know!

In many TV advertisements, the consumer will witness a slightly graying medical doctor in a white lab coat taking off his (the actors are most often male) glasses and donating some of his precious time to inform about the benefits of the product being sold. Also note that the background color in many cosmetic ads is white, a color that portrays a crisp, clean scientific image. Cosmetic scientists and formulators act in similar ways to portray a scientific image--notice that papers typically cited in research papers give the impression that the researchers have read every issue of journals like Nature, Science and the Journal of Investigative Dermatology cover-to-cover. These are all attempts to increase the scientific credibility of what is written. This is not wrong, but the industry knows that the cosmetic relevance of papers published in Science and Nature is limited.

In addition, this author has observed a direct correlation between the number of citations from such journals and the unscientific nature of what follows in the paper describing yet another cosmetic wonder. Are these authors masking something? In a more practical example, at many International Federation of the Societies of Cosmetic Chemists (IFSCC) congresses and conferences, attendees may have noticed that cosmetic companies often prefer their marketing managers to present at such events rather than their technical experts. When probed as to why, they claim that their technical staff is unable to relay the message properly. Is this true? In some cases it may be, but this author believes that in the majority of cases, the technical experts most likely tell a different story (read: more scientific story), whereas the marketing manager plugs the tradename of the product.

Taken at large, however, these minor points should not worry the cosmetics industry. Since the industry does not have many dedicated university courses that study cosmetic science, the scientific credibility--a.k.a. the reliability of the source--is not always the highest possible in the industry but the majority is all right and behaves all right. This is of course different on the Internet where one can find an abundance of self-declared cosmetic experts. Unfortunately, the consumer does not make this distinction and thus groups the self-declared and experienced cosmetic experts together and as a result, the industry fails in the eyes of the consumer.

2. The second question is, “Does the source often make similar claims?” Fishermen are renowned for exaggerating the size of their catch and as a consequence, nobody believes them any longer. Likewise, pseudo-scientists have a habit of going well beyond the facts. 

This author once found himself at a cosmetic conference where two presentations stated the following:

a) It is close to impossible, if not impossible, to create hair relaxers that are non-irritating to the scalp because of the need for a
     high pH to straighten the hair.

b) The described delivery system was said to be so good that it could provide targeted delivery of every molecule to every
     imaginable target site within the skin and hair.

I therefore asked the second speaker why he did not crack the problem of the previous speaker, since hair relaxers are a big market opportunity. Perhaps my concluding question, “If you cannot solve this issue, which issues can you solve? And why should I believe you?” was a bit harsh, but it beautifully underlined the importance of the frequency of making similar claims. Exaggeration leads to disbelief.

Consumers think that cosmetic firms make outrageous claims because some indeed do, and the regulation of claims is relatively recent. The logical consequence is that the general public does not believe what the cosmetic industry claims, even if the formulation and the product truly deliver. The quick buck-makers spoil it for the rest of the industry.

3. The third question is, “Have the claims been verified by another source?” Shermer writes, “Pseudo-scientists typically make statements that are unverified or verified only by a source within their own belief circle.” The lack of verification of cold fusion is a famous example but cosmetics is not that different. Almost everybody working in the cosmetics industry knows that a manufacturing company will not automatically believe everything that a supplier representative says. They want to verify the claims being made for themselves and test the supplier's data under their own conditions to compare it to their own standards. This either may be because they have been disappointed in the past, or because they are good scientists. Likewise, cosmetic claims are verified by regulatory bodies; maybe not immediately but definitely when challenged, and that is a good thing.

One belief related to this subject of verification, and with which this author sincerely disagrees, that is often expressed by critics of the industry is that cosmetic scientists do not publish in peer-reviewed journals and therefore, the industry's research is not up-to-scratch. In fact, there are sometimes good commercial (and competitive) reasons for not publishing scientific work--cosmetic or not, such as engineers developing the latest vehicle engines or IT software.

However, the cosmetic industry does in fact publish in peer-reviewed journals such as the Journal of Cosmetic Science, the International Journal of Cosmetic Science and the IFSCC Magazine, in addition to peer-reviewed trade magazines such as Cosmetics & Toiletries; in fact, this author has reviewed many papers that were rejected for print in all of these publications. Nevertheless, a peer-review process is not objective but rather a personal judgment and it is not uncommon for two reviewers to hold two completely opposing views on a particular paper.

Sometimes, reviewers have personal reasons for rejecting a paper; for example, when they work on the same subject as the author. Also, many papers are rejected simply because they do not offer novel information--the majory of cosmetic science papers are more descriptive than explanatory, and while it may not always be novel, that does not make it bad science. Within the limited pages of a printed journal, however, it may not offer interesting new reading material and therefore it is not published.

Regarding Shermer's third question, this author can only conclude that the cosmetics industry is not any better or any worse than other industries involving commercial science. Three of the ten characteristics differentiating good from bad science have been reviewed here. In this author's view, the cosmetics industry scores neutral on the first question, below average on the second question, and neutral again on the third question.

Again, Goldacre has a point but there are seven more characteristics to be discussed before it can be determined whether cosmetic scientists are as bad as they are portrayed. Read C&T Today in September for the next installment of this series.

Prof. Johann W. Wiechers, PhD 
Independent consultant for cosmetic science, JW Solutions 
Scientific advisor,
Cosmetics & Toiletries magazine
Gasthuispolderweg 30, 2807 LL Gouda
[email protected]

More in Literature/Data