Assistive technology refers to a field of technology that aids or assists individuals in a multitude of different ways: from the completion of daily chores to travel and education. Typically, assistive technology is designed for individuals with impairments or disabilities but if the technology is inclusive, it can be used by the wider population.
The World Health Organization reports that, as of 2020, there were approximately 2.2 billion visually impaired (VI) or blind individuals globally.1 Visual impairments can range from color blindness and presbyopia to glaucoma. In addition, vision deteriorates with age. Given the immense range of visual impairments, and that their degree of severity can vary from person to person, visual impairment and blindness is a niche issue that requires a diverse range of options and the personalization of assistive technologies.
There are two schools of thought in relation to disability management: medical and social models of disability. The medical model has focused primarily on the impairment and rehabilitation of the disabled individual, rather than the improvement of their environment and the elimination of social barriers. The social model, on the other hand, aims to empower the individual by giving them the independence, equal accessibility and social benefits to seamlessly integrate into society.2
Some experts in disability studies say that the social model of disability is yet to penetrate the employment and labor market because a large focus is still on the individual’s impairment.3 At the same time, disabled consumers are often overlooked by consumer brands and retailers alike.
In relation, the Purple Pound refers to the spending power of disabled households in the UK. According to Vogue UK, “The cosmetic industry has mostly overlooked this £249 billion (~US $347.6 billion) consumer market [of disabled users].”4 Although this term is not used in the United States, the disabled household disposable income in this region is still significant, amounting to $490 billion.5
Importance of Cosmetics
Cosmetics have historically played a role in “women's consumer culture (as) a means of self-reinvention and transformation, and allow[ed] women to articulate different aspects of the self.”6 However, there is not much reporting on cosmetics for disabled individuals, and much less on VI and blind individuals. This could be due to the specific challenges each disability group has and each individual’s unique experience with cosmetics.
Studies show that facial attractiveness influences overall physical attractiveness more than body type or physique, and decorative cosmetics (makeup) assist individuals in temporarily altering their appearance and facial features to heighten facial attractiveness.7-9 For the VI and blind consumer, carrying out a beauty or skin care regimen is not as straightforward as it is for sighted individuals, not only because self-appearance is subjective and requires some degree of personal judgment, but also because facial makeup is heavily reliant upon color.
Nonetheless, blind individuals have been found to pay a great deal of attention to their appearance because they know people around them can see them. Yet an obvious and major problem for completely blind users is that there is no way for them to check or confirm their appearance; hence “blind people rely on sighted or partially sighted people to act as their mirrors. And what’s more, sighted people are not exactly ‘reliable’ mirrors.”10
Reducing the reliance of blind and VI consumers on sighted people, friends and family could help them to achieve a more independent life. And combining cosmetic and assistive technologies can make beauty more inclusive by empowering disabled individuals and giving them an equal opportunity to experience makeup and grooming independently.
Recently, some cosmetic brands have started to address the lack of inclusivity in the industry. For example, some have focused on giving the consumer better control during application11, 12 while L’Occitane13 has designed Braille packaging for its products, which is helpful for VI and blind consumers who can read Braille.
There are currently very few products or services combining assistive technology and cosmetics but the general trend toward personalization could be a driver of future developments. Hence, this paper explores two questions:
- What assistive technologies and cosmetic technologies are available to the VI or blind consumer?
- What design considerations for cosmetic purchases and use can be potentially implemented to support the VI and blind consumers?
A combination of a semi-systematic literature review and primary data collection via a small online survey was applied. The review aimed to answer the first research question while the purpose of the survey was to answer the second research question.
Semi systematic review: The following databases were searched: ACM Digital Library, Cochrane Library, IEEE Xplore, Spie Digital Library, NCBI, Research Gate and Elsevier ScienceDirect and KOSMET. A generic internet search was also conducted to find articles that provided information on cosmetics designed for VI or blind users or disabled-friendly brands. The key search terms combining both fields—i.e., assistive technologies and cosmetics—are presented in the word cloud in Figure 1.
The inclusion criteria were: peer-reviewed papers, business articles, conference presentations and posters. Also, all sources must have been published in English from the year 2000 to the current date. Studies based on sighted participants (unless related to cosmetics) and studies on populations < 18 were excluded. The process of searching, search term expansion and data retrieval followed two iterative phases and is illustrated in Figure 2.
Survey: Participants were recruited from previous University College London (UCL) studies, the Global Cooperation on Assistive Health Technology (GATE) community, and the World Blind Union—a sight loss organization. The survey was approved by the UCL Research Ethics Committee (Ethics number: UCLIC_1920_006). The survey was designed and launched on Microsoft Forms in July 2020 and was left open for responses for two weeks post-publishing. Descriptive data analysis was conducted via Microsoft Excel while the qualitative data from the survey was examined using the principles of thematic analysis.
Results: Semi-systematic Review
The two rounds of searching produced approximately 200 abstracts and other articles, and 12 web pages. These were reviewed against the inclusion and exclusion criteria and the final number of papers considered was 31. The results were grouped into five categories of specialized assistive technologies, described next, and one of the cosmetic technologies to assist blind and VI consumers, described later.
1. Navigation toward and within the cosmetic store: The first step in cosmetic use is buying the product, either in-store or online. Tasks that VI and blind individuals typically undertake around in-store shopping were identified as: “shopping list preparation, getting to the supermarket [or store], finding products in-store, getting to cash registers, paying, getting to the exit, getting home.” These tasks required an assistive system incorporating “mobile product selection, store navigation, product search, product identification, utilization of existing devices and minimal environment adjustment.”14 A range of navigation systems were identified, all aiming at providing real-time guidance at the start of the shopping experience. Ranging from smartphone applications to navigation robots, the systems’ common outputs were cues via sounds or verbal instructions.15-19
In-store shopping is a different experience to online shopping, and the type of content and website accessibility features are important for the VI or blind user’s online shopping experience. Experience with online stores appeared under-researched, however; one study on blind individuals’ interactions with visual content on social networks ascertained that they first relied on web accessibility features and then on the people around them. This study highlighted that VI and blind users sought independence and self-reliance but generally relied more on family and less on friends. Finally, the study highlighted that newly designed sites with fewer accessibility features forced blind users to invest a lot of time in understanding and familiarizing themselves with the site via the use of screen readers.20
2. Finding the product: Finding the right cosmetic product depended on previous experience at the store, guidance by sales assistants, technology embedded in the store or assistive devices in possession of the individual. The technologies identified were either based on text recognition or visual feature detection for the identification of products on the shelves.21-24
3. Obtaining product information and reading labels: This process involved choosing the specific product to meet the needs of the consumer, which could present challenges if a number of similar alternatives are available. The identified technologies were reliant on smart mobile applications using text or barcode reading and converting the information to real-time audio or tactile feedback.24, 25
4. Finding and using products at home: It is assumed that a VI or blind person would be able to locate objects at home following personalized home environment principles. However, object detection systems to locate and identify products at home have been reported.24 In addition, artificial intelligence (AI)-enabled personal assistants such as the Apple HomePod or Amazon Alexa devices allow human interaction based on different sensory modalities via multi-modal dialogue processing. This means that VI or blind users could use input modes such as body movement or speech to interact with the personal assistant, which could assist with locating cosmetic products.26
5. Cosmetic application by VI and blind users: Applying cosmetic products—makeup, in particular, requires precision and technique that, depending on the severity of vision loss, can differ significantly from person to person. Support organizations were identified that specialize in assisting blind and VI users of cosmetics. VisionAware,27 for example, an organization that encourages independent living after sight loss, emphasized the use of systematic routine and suggested numerous tried-and-tested makeup application techniques for individuals living with visual impairments or blindness. An example of a makeup application sequence was: “cleanse, moisturize [and] (apply) foundation, powder, blush, eye makeup, lipstick.”27
However, various issues such as mismatched foundation or concealer color, the running of eye products or bleeding of lipstick arise with makeup use. To overcome these challenges, smartphone applications connecting VI and blind individuals with sighted volunteers were identified. Based on video calls, these enabled a sighted person to observe and provide feedback to the VI or blind person as to how “ready” (makeup or hair) he/she is. For example, a popular free app is BeMyEyes28 while Aira29 is an on-demand service including, among other options, a similar mobile application.
While the above technologies and services reduce reliance on friends and family, further automation is now possible due to advances in computer vision and AI. The conversion of visual information into auditory feedback is already being used for wayfinding (see #1, “Navigation toward and within the cosmetic store,” above). The first-ever assistive technology focused on makeup for the blind and VI consumers described a process of lip recognition from a face image, followed by auditory feedback on the lipstick color and application.30 A more recent and innovative approach was sonification: the transformation of shape and color into sound and specifically into a musical format for visual rehabilitation.31
- Bhowmick, A. and Hazarika, S.M. (2017). An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends. Available at: http://link.springer.com/10.1007/s12193-016-0235-6
- World Health Organization (Accessed 2021, Mar 3). Blindness and vision impairment. Available at: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment
- Ishikiriyama, J. and Suzuki, K. (2017). An interactive virtual mirror to support makeup for visually impaired persons. In: 2017 IEEE International Conference on Systems, Man and Cybernetics (SMC) pp 1393-8.
- Hersh, M. and Johnson, M.A. (2010). Assistive Technology for Visually Impaired and Blind People. Springer Science & Business Media, New York p 743.
- Oliver, M. (2013, Jul 22). The social model of disability: Thirty years on. Available at: https://doi.org/10.1080/09687599.2013.818773
- Jacson, L. (2019, Sep 17). The rise of disability friendly beauty brands. Available at: https://www.vogue.co.uk/beauty/article/the-rise-of-disability-friendly-beauty-brands
- DiverseAbility (Accessed 2021, Mar 3). The buying power of people with disabilities. Available at: https://diverseabilitymagazine.com/2018/09/buying-power-people-disabilities/
- Liu, C., Keeling, D.I. and Hogg, M.K. (2016, Jan 1). Strategy narratives and wellbeing challenges: The role of everyday self-presentation. Available at: http://www.sciencedirect.com/science/article/pii/S0148296315003276
- Korichi, R., Pelle-De-Queral, D., Gazano, G. and Aubert, A. (2008). Why women use makeup: Implication of psychological traits in makeup functions. Available at: https://pubmed.ncbi.nlm.nih.gov/18408870/
- Peters, M., Rhodes, G, and Simmons, L. (2007, Jun 1). Contribution of the face and body to overall attractiveness. Animal Behaviour 57 937–42.
- Little, A.C., Jones, B.C. and DeBruine, L.M. (2011, Jun 12). Facial attractiveness: Evolutionary based research. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3130383/
- Kaplan-Myrth, N. (2010, Jun). Alice without a looking glass: Blind people and body image. Available at: https://doi.org/10.1080/713650612
- Beauty Packaging (2019, Mar 28). Grace Beauty makes makeup more user-friendly. Available at: https://bit.ly/3e66Ly8
- Kohl Kreatives (Accessed 2021, Mar 3). Available at: https://www.kohlkreatives.com
- Major, K. (2017, Mar 29). L’Occitane review—Bringing Braille labels to visually impaired customers. Available at: https://unseen-beauty.com/2017/03/29/loccitane/
- Kulyukin, V, and Kutiyanawala, A. (2010). Accessible shopping systems for blind and visually impaired individuals: Design requirements and the state of the art. Available at: http://benthamopen.com/ABSTRACT/TOREHJ-3-158
- Ahmetovic, D., Gleason, C., Ruan, C., Kitani, K., Takagi, H. and Asakawa, C. (2016, Sep). NavCog: A navigational cognitive assistant for the blind. Available at: https://dl.acm.org/doi/10.1145/2935334.2935361
- Guerreiro, J., Ahmetovic, D., Kitani, K.M. and Asakawa, C. (2017, Oct). Virtual navigation for blind people: Building sequential representations of the real-world. Available at: https://dl.acm.org/doi/10.1145/3132525.3132545
- Voykinska, V., Azenkot, S., Wu, S. and Leshed, G. (2016, Feb). How blind people interact with visual content on social networking services. Available at: http://dl.acm.org/citation.cfm?doid=2818048.2820013
- Cecílio, J., Duarte, K. and Furtado, P. (2015). BlindeDroid: An information tracking system for real-time guiding of blind people. Available at: https://bit.ly/3kF6mno
- Azenkot, S., Feng, C. and Cakmak, M. (2016). Enabling building service robots to guide blind people a participatory design approach. In: 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) 3-10.
- Kulyukin, V.A. and Gharpure, C. (2006). Ergonomics-for-one in a robotic shopping cart for the blind. Available at: http://portal.acm.org/citation.cfm?doid=1121241.1121267
- Alhalabi, W. and Attas, D. (2016). Toward device assisted identification of grocery store sections and items for the visually impaired. Available upon request through ResearchGate at: https://bit.ly/3dZOceS
- Devi, P., Saranya, B., Abinayaa, B., Kiruthikamani, G. and Geethapriya, N. (2016). Wearable aid for assisting the blind. Available at: https://bit.ly/3bRZX4k
- Jabnoun, H., Benzarti, F. and Amiri, H. (2017, Mar 17). A new method for text detection and recognition in indoor scene for assisting blind people. Available at: https://bit.ly/3uS0pby
- Chincha, R. and Tian, Y. (2011). Finding objects for blind people based on SURF features. In: 2011 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW) 526-7.
- Shilkrot, R., Huber, J., Liu, C., Maes, P. and Nanayakkara, S.C. (2014, Apr). A wearable text-reading device for the visually-impaired. Available at: http://dl.acm.org/citation.cfm?doid=2559206.2579520
- Këpuska, V. and Bohouta, G. (2018). Next-generation of virtual personal assistants (Microsoft Cortana, Apple Siri, Amazon Alexa and Google Home). In: 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC) 99-103.
- VisionAware (Accessed 2021, Mar 3). VisionAware website. Available at: https://visionaware.org/
- Be My Eyes (Accessed 2021, Mar 3). Bringing sight to blind and low-vision people. Available at: https://www.bemyeyes.com/
- Aira (Accessed 2021, Mar 3). Connecting you to real people instantly to simplify daily life. Available at: https://aira.io/