Intersecting the Senses: Synesthesia to Connect Cosmetics with Emotion

Editor's note: Learn more on this topic from the authors in our exclusive Author Commentary interview.

The term synesthesia is a fusion of two Greek words meaning "together" and "sensation," which refers to the perception of the same stimulus by two different senses. In some individuals, synesthesia is a physiological condition through which experiences related to perception, such as color or taste, are caused by stimuli that would not normally be associated with that experience.

Synesthesia also can be used as an approach to combine different sensations, or in some cases feelings, and for this reason it has widely been used in music, painting, literature and poetic metaphors.¹

For example:

• Soft as a summer whistle;
• And when he came out of the water and approached, I felt the warmth of his eyes, the scent of his words, the saltiness of his caresses, the sound of her beauty, the brightness of his embrace...
• In suspended sound colors, eyes hear, ears look…
• Francisco de Quevedo's: I listen to the dead with the eyes; and
• Joan Manuel Serrat's: Your name tastes like grass.

Synesthetic experiences demonstrate three primary characteristics: they are provoked by a stimulus, they are conscious perceptions and they are automatic.² They are an important trend emerging in sectors such as food, beverages and of course, cosmetics.

For example, in 2016, studies were published showing how a given choice for music can improve the taste of meals.³ In Europe, the Hotel Café Royal opened its Green Bar, in which the cocktails served are inspired by and taste like some of the most representative notes of fine perfumes.⁴ Another bar, Bart’s, recently opened in London featuring a “mood menu,” whereby the cocktails are mixed with specific olfactory notes to improve the consumer’s mood.⁵ And recently, scientists from the University of Oxford found that the taste of chocolate can be altered in conjunction with changes in music or sound; thus, the auditory sense influences the taste and feel perception of the taste buds.⁶

In personal, home and health care, in 2016, the first "beauty bar" of Latin America, "We Love Nails," was launched in Bogotá, where clients could sample beverages and experience new concepts in nail care. Also, Apple Inc. launched a candle scented like a MAC computer upon opening the box for the first time.⁷ In 2017, a fragrance company launched the “scenting memory” awakenings smell kit, designed for Alzheimer’s and dementia patients to help them practice their memory.⁸ And also debuting last year was the olfactory clock designed by Patrick Palcic using perfume to tell time.⁹

Some published studies of synesthesia have been based on Charles Spence's theory of multisensory integration (see Figure 1).¹⁰ This states that when sensory stimuli are mixed properly, there is an effect of super-additivity, whereby the final stimulus is perceived synergistically via the mammalian superior colliculus brain structure—i.e., the source of multisensory cells, in comparison with how stimuli are perceived individually.¹¹

As stated, synesthesia is understood as the perception of an experience achieved by stimuli not normally associated with that experience.² Recent studies indicate it is a neurological phenomenon but its causes are not entirely clear.¹² In fact, a particularly interesting component of synesthesia that has been identified from a neurophenomenological point of view is the emotional force exhibited by synesthetic experiences. Thus, the emotional component of synesthesia has been reconsidered from a neuroscientific perspective.¹³

Without a doubt, synesthesia presents a great opportunity for innovative concepts in the cosmetics field. Considering this potential to intersect and amplify sensorial experiences, a study was undertaken to develop a product design model based on synesthetic evaluations of tactile, audio, scent and taste stimuli, and supported by neuroscience techniques (implicit) and explicit evaluations. For the described studies, specific motivations were considered (see Table 1).

Panel and Methods

A group of 33 volunteers between the ages of 20 and 40 was recruited. Each was a regular user of cosmetic products. Participant reactions were evaluated by eye tracking and EEG, in addition to providing their subjective ratings, as they experienced combinations of four differently textured test formulas, four musical arrangements, six fragrances, two flavors and different visual cues, executed in three stages:

1. Touching-hearing synesthetic relation
   (n = 14);
2. Smelling-hearing synesthetic relation
   (n = 14); and
3. Tasting-touching synesthetic relation (n=16).

'Liking' for the floral note significantly increased with all four music selections.

Touching-hearing Relation Protocol

For the touching-hearing measurements, as participants experienced tactile and visual stimuli, an eye tracking device^a coupled to software recorded their responses. Four test products having different textures were developed. One product at a time was presented to the panelists for ratings on moisturization and softness, during which one of four different musical arrangements was played (see schematic in Figure 2).

This was repeated with the same formula but different music. Subjects were unaware they were experiencing the same formulas and rating them again for hydration and softness properties. This process was repeated for all four musical arrangements and product samples so in total, participants experienced 4 × 4 = 16 formulas/musical arrangement combinations.

Sample formulas: The evaluated formulas included the following.

• Formula A: 30% cyclopentasiloxane (and) dimethicone crosspolymer (and) dimethicone/vinyl dimethicone crosspolymer (and) dimethiconol dispersion;
• Formula B: 30% petrolatum dispersion;
• Formula C: hydrogel formulation in liquid format; and
• Formula D: an acrylic emulsion.

Sample sounds: Regarding the music compositions, Table 2 shows characteristics of the “Four Seasons” compositions by Antonio Vivaldi in 1723.¹⁴ This music was chosen because it is programmatic and evokes different ideas and images for each season of the year. Each has a varied tempo (shown below), sound and interpretation, ranging from acute to severe, to allow for well-differentiated sounds associated with different mental states. (Editor's note: Click through to our digital edition to hear audio samples.)

• Spring I: relates to joy, 120-128 (fast pace);
• Summer III: expresses tension and force, 168-200 (very fast pace);
• Autumn II: presents calmness, 66-76 (moderate pace); and
• Winter I: signifies contemplation, 40-60 (slow pace).

Touching-hearing Synesthesia Calibration

To calibrate the synesthetic experience of panelists, a figure/ratio test was first performed. This comprised tracking the eye movements of panelists as they experienced pairs of visual stimuli and considered whether each was concordant or discordant.¹⁵ The resulting shape of their recorded saccade patterns—i.e., rapid movements of the eyes between fixation points, provided insight on the subjective levels of perceived agreement or disagreement between the stimuli, which helped to classify their synesthetic response.

For example, the first pair of visual stimuli included an image of a waterfall and the printed word hydration. The second pair included an image of cotton balls and the printed word softness. For these two concordant pairs, 82% of panelists tended to focus more on the images. In contrast, a third pair included an image of a desert with the printed word hydration. For this discordant match, only 18% of panelists focused on the image; the larger majority focused on the word (see Table 3).

This served as the criteria to evaluate and interpret touching-hearing eye-tracking tests. More specifically, as panelists experienced tactile, visual and taste stimuli, if their eyes focused more on the image representing the desired trait, the more concordant their experience and therefore, the greater the synesthetic effect.

Multisensory integration refers to the final stimulus being perceived synergistically, compared with effects of
individual stimuli.

Results: Touching-hearing Relation

Following the calibration step, the sample formulas were presented with music and the same visual stimuli as described, and panelists evaluated the formulas in terms of perceived hydration and softness. Figure 3 shows the calibration and assessment process; Table 4 shows results of the synesthetic responses of panelists based on eye-tracking; Table 5 provides a comparison between these implicit responses and explicit panelist ratings.

It was interesting to see music change the perceptions of hydration and softness. Specifically, the silicone dispersion gave the softest and most hydrating impression with the summer music, whereas both the dispersion of petrolatum and the emulsion registered the highest softness and hydration with the autumn music. The liquid hydrogel was rated best for hydration and softness with the winter arrangement. None of the textures demonstrated increases in the perception of hydration and softness with the spring music.

These initial findings open the door to developing novel concepts, such as seasonal cosmetics linked to textures and music. It also suggests a new method for evaluating product perception: now, under the influence of music.

Smelling-hearing Relation Protocol

To assess relationships between the experience of smelling and hearing, brain waves were recorded instrumentally by electroencephalography (EEG)^b. This approach is based on the principle that the brain emits signals of different frequencies (in Hz) depending upon a given stimulus, which are captured and recorded (see Figure 4).

Sample scents: Six fragrances were used for the smelling-hearing relation tests: (1) citrus; (2) spicy; (3) floral; (4) woody; (5) Perfume 1, with floral accords of jasmine and sandalwood; and (6) Perfume 2, with citrus notes of bergamot, orange blossom, vanilla, patchouli and musk. As panelists experienced a given scent, they also simultaneously rated it for fragrance intensity and liking/pleasure on a scale of 1 to 10, from lowest to highest.

The process was divided into five stages, whereby the panelists randomly and blindly evaluated the same fragrances but while listening to different music—the same four seasons arrangements noted above. This protocol, shown in Figure 5, comprised:

• Stage 1. Evaluation of fragrances without music.
• Stage 2. Evaluation of fragrances while listening to spring.
• Stage 3. Evaluation of fragrances while listening to autumn.
• Stage 4. Evaluation of fragrances while listening to summer.
• Stage 5. Evaluation of fragrances while listening to winter.

Rating Results: Smelling-hearing Relation

Table 6 shows the panelist ratings for fragrance perceptions, which indicated the citrus and woody notes showed no significant change in intensity or liking when paired with any musical arrangements. In contrast, liking for the floral note significantly increased (p < 0.05) with all four music selections; its perceived intensity was not impacted by music. A significant increase (p < 0.05) in spicy note liking was observed with the autumn music selection; however, significant increases for both intensity and liking resulted from pairings with the summer and winter music.

Perfume 1 demonstrated significant improvements (p < 0.05) in intensity only when coupled with summer and winter music. Finally, for Perfume 2, the perception of liking significantly improved (p < 0.05) when paired with autumn, summer and winter music selections.

A statistically significant difference in the softness rating for petrolatum was observed as panelists tasted sweet chocolate.

EEG Results: Smelling-hearing Relation

The results from the EEG measurements in this study are presented in Figure 6. For all fragrances, changes in brain activity in terms of liking were noted with the addition of music, in comparison with the initial evaluation without music. This technique identifies and measures the strength of micro-electric fields generated by coherent activity between thousands of neurons. Therefore, detection of brain electrical signals favors the description of behavioral changes such as reactions to visual, audio, sensory or emotional stimuli, such as frustration and interest, in mere in milliseconds.

For example, Figure 7 shows how frustration and interest curves change during the synesthetic experience. These changes are quantified to evaluate the liking of a fragrance while the panelist is listening to music.

The olfactory notes showing the greatest variation in perception when paired with different music arrangements were: spicy, floral and Perfume 1 (floral). Those demonstrating the least amount of change due to musical influence were: citrus, woody and Perfume 2 (citrus). Under the conditions of this study, these results identify a trend for greater sensitivity with floral fragrances than citrus fragrances.

Table 7 shows a comparison between extrinsic ratings and intrinsic EEG results for the overall smelling-hearing evaluation.

Tasting-touching Synesthesia Calibration

For the tasting-touching tests, as participants experienced stimuli, the same eye tracking device^a coupled to software was again used to record their responses. Before testing began, the synesthetic experiences of the panelists were again calibrated similarly to described above using a figure/ratio test,¹³ to find agreement/disagreement based on the shape of panelists’ saccade patterns (see Figure 8).

For the tasting-touching evaluation, however, some changes were implemented. First, an image of a lime or chocolate paired with the words acid (acido) or sweet (dulce) was shown to the panelists, to create concordant and discordant responses. Following these flavor-related pairings, panelists viewed an image of cotton balls always paired with the concordant word softness (suave) as they simultaneously tasted the chocolate or acid gum and experiences the different textures on their skin.

This synesthesia calibration of the taste-touch panel provided an interesting response pattern. Similar to before, concordance between the lime or chocolate image with the appropriate word acid (acido) or sweet (dulce) was once again reflected in the panelists’ eyes focusing on the images; this was true for 94% of the panelists (see Figure 9). Also similar to before, where discordance was felt between the flavor-related images and printed words, the majority of panelist eye traffic focused on the discordant word.

For the tactile portion, as expected, after panelists viewed concordant taste-related pairs, as they viewed the image of cotton balls with the concordant word softness (suave) and simultaneously experienced the taste and tactile stimuli, their eyes were drawn to the cotton ball image. However, after viewing discordant pairs, seeing the image of cotton balls with the same concordant word softness (suave) drew eye traffic toward the word softness rather than the image of the cotton ball.

This served as the criterion to evaluate and interpret the tasting-touching eye-tracking tests, described next. Specifically, as panelists experienced a product, if they were more focused on the image of cotton balls than the word softness (suave), this indicated the formula texture provided a perceived softness. If panelists focused more attention on the word softness (suave) than the image of cotton balls, this indicated discordance and the formula did not provide a good perception of softness.

A particularly interesting component of synesthesia is the emotional force exhibited by the experience.

Tasting-touching Relation Protocol

The panelists then felt one of the four formulas previously described while tasting an acidic gum or sweet chocolate and viewing the image of cotton balls and the word softness (suave). They evaluated the formulas once again for both hydration and softness properties using a perceptio questionnaire.

Unaware to panelists, the same formula was used while the flavor was varied; this was repeated for all four formulas. The objective was to determine how the perceived sensation of softness and hydration were modified in relation to the taste stimuli. Figure 10 shows the process.

Rating Results: Tasting-touching

Figure 11 and Figure 12 provide a comparison of the ratings for hydration and softness, respectively. A statistically significant difference (p < 0.05) was observed in the softness rating for the petrolatum dispersion when tasting the sweet chocolate.

Eye-tracking Results: Tasting-touching

Figure 13 presents a summary of the tactile results obtained for the evaluation of the four products as panelists experienced different flavors. Here, while tasting acidic flavor, panelist responses showed the tactile experience of the emulsion was the most concordant with softness (suave); therefore, it was the most synesthetic sample to the acid sensation. While tasting the sweet flavor, panelist responses indicated the tactile experience of the petrolatum dispersion was the most concordant with softness (suave). Therefore, it was the most synesthetic sample to the sweet sensation.

Conclusions

The three cases evaluated here allowed for the identification and determination of the influence of synesthetic touching-hearing, smelling-hearing and tasting-touching relationships present during the implicit and explicit evaluations of different cosmetic formulas.

Varied music and flavor notes provoked greater sensitivity in the perceptions of sensory attributes. Differences in the liking of olfactory families were found to be influenced by different sounds, as monitored EEG. Also, basic sweet and acidic flavors can change the perceptions of hydration and softness in some kinds of formulations, evidenced by implicit eye tracking measurements and explicit evaluations. The sensory profile of some formula excipients could even be linked to different musical arrangements by explicit and implicit evaluations.

Thus, by combining an implicit and explicit formulation design with the objective of finding synesthetic patterns, the present work opens doors to novel product concepts, benefits and textures. Synesthetic design is a new world to discover and develop. To the extent that new methods are designed to link explicit and implicit evaluations, cosmetic chemists will have a powerful tool that will allow them to create interesting textural experiences enhanced in new ways by other senses.

Acknowledgements: The authors wish to thank the Center for Innovation and Development of Belcorp and Neurocorp for their collaboration with the design of these readings.

References

1. retoricas.com/2011/10/ejemplos-de-metafora-sinestesica.html (Accessed Feb 14, 2018)
2. B Stein, The new handbook of multisensory processing, Masachussets Institute of Technology (2012)
3. http://yucatan.com.mx/gastronomia/alimentos-gastronomia/la-musica-influiria-en-el-sabor-de-la-comida (Accessed Feb 2017)
4. luxuryretail.es/cocteles-inspirados-perfumes-givenchy/ (accessed Feb 2017)
5. telegraph.co.uk/food-and-drink/cocktails/i-tried-londons-new-happiness-boosting-cocktails---but-did-they (Accessed Feb 2017)
6. F Reinoso, 'Smooth operator': Music modulates the perceived creaminess, sweetness and bitterness of chocolate, Appetite 108 (2017) pp 383–390
7. theverge.com/tldr/2016/10/4/13160850/new-mac-candle-twelve-south-apple (Accessed Feb 14, 2018)
8. perfumerflavorist.com/networking/news/company/Scenting-Memory-Essential-Awakenings-Smell-Kit-Designed-for-Alzheimers-and-Dementia-Patients-433897713.html (Accessed Feb 14, 2018)
9. dezeen.com/2017/01/22/copper-clock-patrick-palcic-perfume-tell-time-design-homeware-imm-cologne-2017 (Accessed Feb 14, 2018)
10. N Holmes and C Spence, Multisensory integration: Space, time and superadditivity, Curr Biol 15(18) (Sep 20, 2005)
11. E Du Plessis, The branded mind, Erik du Plessis Ed, USA (2011)
12. H Melero, Synesthesia: A return to the body, Theoria et Historia Scientiarum, vol X, Universidad Complutense de Madrid (2013)
13. R William, Synesthesia: From cross-modal to modality-free learning and knowledge, Leonardo 48(1) pp 48–54 (2015)
14. https://es.wikipedia.org/wiki/Las_cuatro_estaciones. (Accessed Feb 2017)
15. J Jiménez, Application of eye-tracking methodology for fragrance evaluation, IFSCC Magazine 2 23–26 (2015)