Unilever has partnered with the accessibility app specialist Be My Eyes, the developer of accessibility solutions apps that connects people who are blind or low-vision to a world network of volunteers and AI-powered virtual assistance, to expand their use of AI technology to add powerful new tools to the cooking experience of their products for shoppers who are blind or have low vision.
The new partnership builds on addition of on-pack Accessible QR (AQR), developed by computer vision specialists Zappar, to Unilever’s Persil and Colman’s products in the UK last year and is part of Unilever’s global connected pack strategy, which includes using new digital experiences and technology to evolve and differentiate the way shoppers interact with and use Unilever’s products.
Unilever is initially trialling the Be My Eyes technology in the UK through their Colman’s Singapore Noodles Meal Maker, with the intention of rolling it out across more brands in 2024. Scanning the AQR code on the front-of-pack, shoppers can now be directed to the Be My Eyes app for free and connected to either a volunteer to read and describe cooking instructions or to a virtual AI-chat bot which uses Chat GPT-4 capabilities to address recipe or cooking questions.
This is the first time the Be My Eyes AI technology has been integrated with a food product, offering an AI-assisted cooking experience at home. Unilever is working with Be My Eyes to train their volunteers and provide specific Colman’s product information and data to the Chat GPT-4 AI bot. Unilever’s customer support team will also be integrated into the Be My Eyes app experience, allowing shoppers the option to be automatically connected to the Colman’s careline team for further support if needed.
The Be My Eyes app is the only 100% free app that allows users with blindness, low-vision or deaf-blindness, to receive live accessible information from volunteers, trained customer support representatives and artificial intelligence (AI).
Rachana Dongre, Senior Digital Engagement and Strategy Lead Nutrition & Ice Cream, Unilever said, “We’ve accelerated digitising our packs to offer new opportunities for brand engagement and elevated shopping experience and now we’re also focusing on how we can also use digital experiences to make our products more accessible. Zappar’s AQR codes means we can support blind and low vision shoppers to have equal access to information and integrating Be My Eyes into these codes offers a totally new way to make the full experience of our products more inclusive, from the shopping aisle right through to cooking at home.”
“The 285 million blind and low vision consumers in the world buy the same products as everyone else,” said Be My Eyes CEO Mike Buckley, “The difficulty in reading package ingredients and cooking instructions has long marred the cooking experience of these consumers. Unilever’s commitment to use new Zappar technology with Be My Eyes is pioneering a new era in product accessibility and disability convenience.”
Caspar Thykier, Zappar’s co-founder and CEO, said “Our integration with Be My Eyes is a really significant step towards making access to information a basic human right. By combining Zapvision’s computer vision technology with Be My Eyes’ new OpenAI, ChatGPT4 Vision integration, we’re excited to underpin a more conversational and assistive experience for Be My Eyes users with unambiguous, structured and accurate data.
“This is the start of a new type of ‘product as a service’, levelling the playing field still further for food brands such as Unilever’s Colman’s products. We’re so excited for our continued partnership with Unilever and now Be My Eyes, providing further access to information for the blind.”
The new Be My Eyes service is integrated into the AQR codes Unilever began adding to their products last year, including Colman’s and Persil products in the UK. The AQR codes, which are developed by Zappar, work by adding a dot-dash pattern around one corner of the code, which make them detectable from further away. The Zappar AQR code can be scanned through accessibility apps from 1.1 metres (over 7x that of normal QR scanning) to help blind consumers find their desired product in sore or at home.
Unilever uses these AQR codes to provide product information such as usage direction, recycling, and ingredient information and nutritional value, interacting with standard Smartphone accessibility through the Be My Eyes app to share information via audio description or displaying information in larger text.
Unilever will be working with Be My Eyes and Zappar to collate user feedback to optimise the technology with the aim to roll out the accessible technology to other Unilever products. Unilever’s ambition is to continue to partner with accessibility experts and apps to support wider integration of AQR technology across different products, categories and countries and to work with other businesses using this technology to share best practice.
