Current Issue cover

Good As Gold: Validating Alternative SPF Test Methods

Contact Author Uli Osterwalder, Sun Protection Facilitator (SPF) GmbH, Basel, Switzerland; Steffen Uhlig, Ph.D., and Bertrand Colson, QuoData Quality and Statistics GmbH, Berlin and Dresden, Germany; Jürgen Vollhardt, Ph.D., DSM Nutritional Products Ltd., Switzerland
Fill out my online form.

To read this article in its entirety, click through to your April 2020 digital magazine. . .

Two years ago, the current authors published the article, “SPF Assessment Revisited—Status and Outlook.”1 The present article provides an update on progress toward alternative SPF methods. It also proposes a validation process for such methods with the goal of equivalating them to and revising the gold standard ISO 24444:2019.2, 3 First, however, consider the challenge of finding alternative SPF methods.

The SPF In vivo test as described in the norm ISO 24444 is a relatively complex method and requires significant equipment and time to be executed. However, because human skin serves as the template and performance indicator, it directly connects with the application of sunscreens and potential biological damage to skin caused by UV.

Human skin and the processes that occur during sunscreen application are also complex4, 5 and difficult to match by alternative templates or procedures. Therefore, it has taken quite some time to develop methods that could, for the vast variety of sunscreens made globally, consistently deliver comparable results.

Furthermore, the In vivo SPF procedure utilizes UV radiation and represents a burden to the test subjects. Therefore, alternative procedures should avoid this burden. Also, the very nature of human subjects intrinsically causes some variation in results. The desire is therefore to identify alternative methods with less variation that are more robust and agree well with in vivo SPF results.

Finding an alternative method that demonstrates high precision is relatively easy, but the challenge has been to find a method that can also predict the true SPF value correctly. This puzzle is depicted in Figure 1. Alternative methods must also demonstrate sufficiently good agreement with the current In vivo SPF method. This raises several questions: How can the agreement of two methods be tested? How many test institutes should be involved? How many different products and types must be tested? And how does the method validation differ from the routine procedure? To gain perspective, it would help to revisit ISO 24444:2010 and its recent revision, ISO 24444:2019, before turning to alternative methods.

SPF In vivo Test ISO 24444:2019

SPF determination began more than 50 years ago as an outdoor method, using erythema as the biological endpoint. UV-induced damage triggers an inflammation reaction that creates visible erythema via a multistep biological pathway. The measured SPF values were very low at that time, often just SPF 2, 3, 4, etc. Today, in vivo SPF determination is more complex, with protocols using standardized solar-simulated light sources2, 3 and much higher SPF values: SPF 30 and 50 are the most popular categories. ISO 24444 and the very similar US-FDA 20116 are currently considered the gold standards in sunscreen assessment, and Garzarella, et al., showed that both methods—ISO’s and the U.S. Food and Drug Administration’s (FDA’s), yield similar SPF values.7

For both protocols, the standard reference samples P2 and P3 are examples of known true SPF values. Data recently collected by Alejandria, et al., for reference sample P2 shows,8 impressively, how the random/overall error of the SPF determination can be reduced by increasing the number of repetitions (see Table 1). The standard error (precision) is reduced by the square root of the number of repetitions. On the other hand, the mean (true) value does not change much beyond a reasonable minimum number of repetitions.

In the case of the SPF measurement, it is assumed that the average value from 10 volunteers is sufficient to determine the SPF in a particular laboratory setting and routine operation. In fact, the differences between the standard SPF values and the average of approximately one thousand measurements of P2 are only 3% (ISO) and 6% (FDA), respectively. So, both can be considered as close to the true value.

On the other hand, a single SPF determination in just one laboratory can, of course, be much further away from the true value. Miksa, et al., showed it is advisable to take the average of at least three or four such SPF values from different test institutes, with five volunteers each, to generate a more reliable SPF value, closer to the true value,9 as can be expected from statistical considerations.

Revisions3 to ISO 24444:2019

The revision of ISO 24444:2019 was prepared by Technical Committee ISO/TC 217, Cosmetics, over the last 3-4 years. This second edition cancels out and replaces the first edition.2 Following are three key changes of the nine made:

  • Three new reference standard sunscreens have been validated and added (see Table 2);
  • Sunscreen application procedures have been described in greater detail; and
  • The reporting tables and requirements have been modified to provide more complete information on the results of the testing.

These changes will further improve the accuracy of SPF determination. The new reference samples, SPF 30 (P5), SPF 43 (P6) and SPF 63 (P8), will especially allow testing labs to check their equipment much closer to the relevant market requirements. Most countries have a cap of SPF values in place, e.g., at SPF 50+, which means SPF > 60 in Europe. The new standards P5, P6 and P8 also will be used in future norms of alternative SPF methods.

Currently Available Alternative SPF Methods

The development of alternative SPF and UVA-PF methods has been under way for several decades. The fact that none of these has replaced the gold standard so far shows this is not an easy task. Figure 2 gives an overview of SPF assessment methods, which can be classified according to several criteria. Classical divisions are along the type of method, i.e., in vivo, in vitro and in silico; and the burden of UV radiation to which skin is exposed. As can be seen, and as is explained below, these classifications overlap.

The ideal assessment method would be accurate and in agreement with in vivo results; show improved reproducibility; and be easy to perform—i.e., would be cost-effective. Below is a brief summary of the alternative SPF methods currently in discussion or under development; some are already commercially available. They are divided into three categories: 1) in vitro transmission methods, 2) in silico methods and 3) hybrid diffuse reflectance spectroscopy. The details of these methods can be found in the indicated literature.

1. In vitro Transmission Methods

The in vitro transmission approach was the first attempt to find an alternative to the in vivo method. The first comparisons in vitro and in vivo date back to over 40 years ago.10 While many substrates have been attempted, the two major requirements are: 1) transparency for UV, and 2) a certain degree of roughness to mimic the skin surface. Over the decades, polymethyl methacrylate (PMMA) has been found to be a useful substrate, although other materials have also recently been suggested.11 Sunscreen is applied to roughened PMMA plates that attempt to mimic the texture of skin. Several plates had been developed with different levels of roughness.

Transmission is measured by a spectrophotometer with an integrating sphere to catch scattered UV light. Together with the erythema action spectrum and the (simulated) solar spectrum, the measured transmission data are then used to calculate the SPF. After several decades of development, a working group of the German Cosmetics Society concluded the in vitro sun protection factor is still a challenge with no final answer,12 although a recent breakthrough has been reported.13, 14 Three in vitro methods include: the Cosmetics Europe (CE) Method;15 Fused Method;16 and other methods, e.g., JCIA method with “skin-mimicking substrate.”17

CE Method

Under the leadership of Cosmetics Europe (CE), a comparison of in vivo versus in vitro SPF test methods was conducted using 24 commercial sunscreens, SPF 6-50+, in three laboratories each. The CE method,15 developed by Miksa et al.,14 uses a robotic arm to control the spread of test products onto two different PMMA plates, molded and sandblasted. The CE method is currently a norm per the ISO technical committee TC 217 (Cosmetics), under the approved work item number and title: “ISO/AWI 23675 Cosmetics—Sun Protection Test Methods—In vitro Determination of Sun Protection Factor.”

Fused Method

The “fused method” is the unofficial name of a combination of different in vitro transmission methods. A major new element is a calibration step and the prediction of the “dispersal rate,” after Batzer, et al.’s, “The 'Dispersal Rate'—A Product Dependent Characteristic to Predict the Reliability of the Calibrated in Vitro SPF on WW5 Plates.”

In their work, the authors conclude:16 We suggest implementing an individual calibration of the in vitro SPF to improve the reproducibility of in vitro SPF measurements between different laboratories. Considering the Dispersal Rate helps to estimate the reliability of the in vitro SPF measured on WW5 plates. In order to evaluate whether those products with a high Dispersal Rate can also be calibrated with special standards, further measurements need to be done. We demonstrate that, besides the known parameter, also the composition of the products should be considered for the interpretation of the in vitro SPF. Our findings could explain some multiple reported problems in correlation between in vitro and in vivo SPF, especially for higher SPFs.

2. In silico Methods

Attempts to calculate the SPF and other performance parameters of sunscreen started more than 20 years ago. The state of the art is summarized by Herzog and Osterwalder.18 In silico is very similar to the in vitro transmission method except the transmission measurement through the roughened PMMA plate is replaced by a calculation with a model of the “non-uniform sunscreen” film on the skin.19 Then, it follows the same calculation as the in vitro method. Commercial examples include BASF’s “Sunscreen Simulator”20 and DSM’s “Sunscreen Optimizer.”21

For market monitoring or surveys, the simulator tool can be combined with UV filter analysis, e.g., according to EN 17156.22 This analysis then provides the UV filter concentrations that can be plugged into the simulator. Simulation tools are now freely available on the internet.20, 21

In silico SPF determination methods are becoming increasingly popular; they often provide realistic, usually rather conservative results. In silico methods using the UV filter spectra of sunscreens supplement the possibilities offered by in vivo and in vitro methods. In silico calculation has, per se, no random error component when repeating the “experiment.” This means any deviation from the true SPF is systematic. In silico SPF calculation also is being used by authorities to monitor the market, e.g., CVUA Karlsruhe, in Germany.23, 24 In the United States, the in silico assessment of market products is relatively easy since declaring the concentration of UV filters on the packaging is mandatory.

Sun Protection Simulator

During the development of new sun protection formulations, quick and inexpensive methods for estimating UV screening performance are highly desirable. The most convenient approach toward this goal is given by computational simulations, such as BASF’s commercial “Sunscreen Simulator.” Models for the calculation of SPF employ the same algorithm used with in vitro SPF measurements but replace the transmittance measurement by calculating the overall absorbance of the UV filters in an irregular sunscreen film. The simulations require a database with quantitative UV extinction spectra of the relevant UV filters, as well as a mathematical description of the film irregularity.

The simulation algorithm also implies the consideration of photodegradation properties of the UV filters in the sunscreen composition. Besides using such simulations for designing new sunscreen formulations, the calculations can also support the understanding of sunscreen performance in general.20

Sunscreen Formulation Tool

Another commercial sunscreen simulator based on comparable mathematics has was developed by DSM in 2017, the “Sunscreen Optimizer.”21, 25 This in silico tool is also useful in the development of sunscreens and UV protective day care formulas. It functions like an in silico laboratory and allows for the comparison of different formulation concepts in many aspects of performance, as well as formulation parameters such as oil load and oil UV filter content.

3. Hybrid Diffuse Reflectance Spectroscopy

Hybrid diffuse reflectance spectroscopy (HDRS or H-DRS) is based on non-invasive diffuse reflectance spectroscopy (DRS). Here, UVA protection (320-400 nm) is directly assessed in vivo, and since the returning signal is not sufficient in the UVB range (290-320 nm), the SPF is calculated by extrapolating the UVA curve into UVB by using in vitro data—which makes it a hybrid method. Figure 3 provides a schematic of the HDRS principle. Currently there are three sub-types of HDRS: monochromatic,26 which is commercially available; polychromatic,27 whose rollout is anticipated; and multichromatic, LEDs,28 still under development.

The HDRS methods are currently developed into a norm in the ISO technical committee TC 217 (Cosmetics), under the approved work item number and title: ISO/AWI 23698, “Cosmetics Sun Protection Test Methods—Measurement of the Sunscreen Efficacy by Diffuse Reflectance Spectroscopy.”

HDRS, Monochromatic

The HDRS concept has been developed in the U.S. by Ruvolo, et al.—the first DRS in 2009, for the determination of UVA-PF;29 then in 2014, for SPF determination.30 Its further development to a commercial method was a multinational effort lead by Rohr, et al.,26 in Germany, with equipment from Bentham, UK. So far, hundreds of samples have been measured by HDRS. The UVA-PF and SPF values appear to be in good agreement with in vitro and in vivo UVA values after ISO 24443 and ISO 24442, and the SPF gold standard ISO 24444, respectively.

HDRS, Polychromatic

The Solar Light Company has built a polychromatic HDRS version27 that measures an integral signal of the UVA part, combined with a full spectrum in vitro scan, and thus has a simpler optical and mechanical design. Tests can be conducted in a shorter time frame versus a monochromatic device that must scan the spectrum one wavelength at a time.

HDRS, Multichromatic, LEDs

Another approach using the HDRS principle was developed at the Charité in Berlin, together with Courage-Khazaka, Cologne.28 Here, to further reduce the complexity of the equipment, LEDs are used as a light source. The first prototype used an LED at 310 nm, which is surprising since UVB is practically not reflected. The latest version uses eight LEDs spread over the whole UVB and UVA range. This technology is still in the research and development stage but deserves close attention.

. . .Read more in our April 2020 digital edition. . .


  1. Osterwalder U., Schütz, R. and Vollhardt J. (2018, Apr 13). SPF assessment revisited–status and outlook. SOFW, 144.
  2. ISO. (2010). Sun protection test methods: In vivo determination of the sun protection factor (SPF). ISO 24444:2010, Cosmetics. https://www.iso.org/standard/46523.html
  3. ISO. (2019, Dec). Sun protection test methods: In vivo determination of the sun protection factor (SPF). ISO 24444:2019(E), Cosmetics, 2nd edn. https://www.iso.org/standard/72250.html.
  4. Vollhardt, J., Baltussen, M., Oosterlinck, F. and Mendrok-Edinger, C. (2015). Revisiting human skin, sunscreen films and protection performance. Can we create the ideal high performance sunscreen? Proceedings: 13th International Sun Protection Conference/London.
  5. Adlhart, C., Edelmann, M. ... Vollhardt, J., et al. (2017). Distribution differences of individual UV filters related to application procedure and substrate probed by Raman confocal microscopy. Proceedings: 14th International Sun Protection Conference/London.
  6. U.S. Food and Drug Administration (FDA) (2012, May 11). Labeling and effectiveness testing; Sunscreen drug products for over-the-counter human use. Federal Register, 76(117), 35620-35665.
  7. Garzarella, K. and Caswell, M. (2013). Disparate SPF testing methodologies generate similar SPFs. J Cosmet Sci, 64, 297–307 (2013).
  8. Alejandria, M., Marra, A., Roberts, G. and Caswell, M. (2019, Jul/Aug). Disparate SPF testing methodologies generate similar SPFs. II. Analysis of P2 standard control SPF data. J Cosmet Sci, 70(4), 181-196.
  9. Miksa, S., Lutz, D., Guy, C. and Delamour, E. (2016, Dec). Sunscreen sun protection factor claim based on in vivo interlaboratory variability. Int J Cosmet Sci, 38(6), 541-549.
  10. Sayre, R.M., Agin, P.P., LeVee, G.J. and Marlowe, E. (1979, Mar). A comparison of in vivo and in vitro testing of sunscreening formulas. Photochem Photobiol, 29(3), 559-66.
  11. Sohn, M., Malburet, C., Baptiste, L. and Prigl, Y. (2017, May 24). Development of a synthetic substrate for the in vitro performance testing of sunscreens. Skin Pharmacol Physiol, 30(3), 159-170. doi: 10.1159/000464471.
  12. Rohr, M., Klette, E., ... Zastrow, L., et al. (2010, Mar 9). In vitro sun protection factor: Still a challenge with no final answer. Skin Pharmacol Physiol, 23(4), 201-12. doi: 10.1159/000292777. PMID: 20215813.
  13. Miksa, S., Lutz, D. and Guy, C. (2015). New approach for a reliable in vitro sun protection factor method-Part I: Principle and mathematical aspects. Int J Cosmet Sci, 37(6), 555–566. doi:10.1111/ics.12226.
  14. Miksa, S., Lutz, D., Guy, C. and Delamour, E. (2016, May 6 e-pub). New approach for a reliable in vitro sun protection factor method-Part II: Practical aspects and implementations. Int J Cosmet Sci (2016, Oct), 38(5) 504-11. doi: 10.1111/ics.12327. PMID: 27060786.
  15. Pissavini, M., Tricaud, C., ... Matts, P.J., et al. (2018, Apr 20 e-pub). Validation of an in vitro sun protection factor (SPF) method in blinded ring testing. Int J Cosmet Sci, 10.1111/ics.12459. doi: 10.1111/ics.12459. PMID: 29676800.
  16. Batzer, J., Bleckmann, A., Lerg, H., Schwanke, F. and Schläger, T. (2016). The 'Dispersal Rate'-A product dependent characteristic to predict the reliability of the calibrated in vitro SPF on WW5 plates. Int J Cosmet Sci, 38(3), 294–304. doi:10.1111/ics.12293.
  17. Miura, Y., Takiguchi, Y., ... Shirao, M. et al. (2008). Algorithm for in vitro sun protection factor based on transmission spectrum measurement with concomitant evaluation of photostability. Photochemistry and Photobiology, 84, 1569-75.
  18. Herzog, B. and Osterwalder, U. (2015). Simulation of sunscreen performance. Pure Appl Chem.
  19. Ferrero, L., Pissavini, M., Marguerie, S. and Zastrow, L. (2003). Efficiency of a continuous height distribution model of sunscreen film geometry to predict realistic sun protection factors. J Cosmet Sci, 54, 463-481, (2003)
  20. BASF website. (accessed on 2020, Feb 16). Sunscreen Simulator. https://www.sunscreensimulator.basf.com/Sunscreen_Simulator/computation
  21. DSM website (accessed on 2020, Feb 16). Sunscreen Optimizer. www.sunscreen-optimizer.com
  22. DIN (2017). DIN EN 17156. Cosmetics–Analytical methods–LC/UV method for the identification and quantitative determination in cosmetic products of the 22 organic UV filters in use in the EU. https://standards.globalspec.com/std/13321401/din-en-17156
  23. Mildau, G. (2015, May). Aktuellen stand der amtlichen beurteilung von sonnenschutzcremes in Europa. (Current status of the official assessment of sun protection creams in Europe). In: Schmidt-Lewerkühne, H., Mensch & Sonne, DGK Symposium. Kongresszentrum Darmstadium, Darmstadt. SÖFW J 141, 11/15.
  24. CVUA Karlsruhe, Germany. (2020, Mar 9). https://www.ua-bw.de/pub/default.asp?subid=2&Lang=DE
  25. Osterwalder, U., Janssen, A. and Schlifke, A. (2018). A new option for SPF assessment? COSSMA 1-2, 20-24. https://www.cossma.com/ingredients/article/a-new-option-for-spf-assessment-35315.html
  26. Rohr, M., Ernst, N. and Schrader, A. (2018, May 23 e-pub). Hybrid diffuse reflectance spectroscopy: Non-erythemal in vivo testing of sun protection factor. Skin Pharmacol Physiol, 31(4), 220-228. doi: 10.1159/000488249. PMID:29791917.
  27. Cole, C., Silverman, J. and Bonitatibus, M. (2019). Evaluating sunscreen ultraviolet protection using a polychromatic diffuse reflectance device. Photodermatol Photoimmunol Photomed, 35(6), 436–441. doi:10.1111/phpp.12496.
  28. Reble, C., Gersonde, I., Schanzer, S., Meinke, M.C., Helfmann, J. and Lademann, J. (2018). Evaluation of detection distance-dependent reflectance spectroscopy for the determination of the sun protection factor using pig ear skin. J Biophotonics, 11(1), 10.1002/jbio.201600257. doi:10.1002/jbio.201600257.


Table 1. SPF Error

Overall Determined SPF Error as a Function of Repetition

Table 2. Reference Standard Sunscreens

Reference Standard Sunscreens

Figure 1. Right vs. Wrong

Depiction of "roughly right" versus "precisely wrong."

Figure 2. SPF Assessment Methods

SPF assessment methods

Figure 3. Experimental Setup

Experimental setup of H-DRS measurement.

Next image >