Long read
Using augmented reality in the food factory
Despite the stilted start, technology has become more commonplace in the world of food and drink, and seen the sector introduce and achieve some incredible triumphs – from portable food fraud detection devices, to autonomous tractors and intelligent stock management systems. But one technology which appears to be all the rage in hospitality has yet to make significant headway in the production plant.
Before entering the food sector, I worked as a technology journalist and would often hear about the latest kit entering the scene. The ‘eXtended realities’ community (or XR for short) featured frequently in press releases and there were those who claimed such tech would be revolutionary in upskilling and training staff in fields such as medicine and engineering. So why not food?
With worker shortages, retention and effective training being some of today’s key challenges for food and drink, could AR be the key?
Speaking to Professor Bob Stone, Emeritus Professor at the University of Birmingham, specialising in XR and telepresence applications and human factors, he explained his belief that AR is likely to change the way humans access and interact with information on a daily basis, but expressed uncertainty over whether it will transform the food and drink sector to the point at which some headlines suggest.
“Can we say with confidence that AR is, now a real ‘game changer’ and is, now, delivering convincing results and ROIs in the F&B sector today? I would have to say no,” Stone told Food Manufacture, as he reasoned that there is limited evidence to suggest otherwise.
He pointed to regular reports of AR used to create brand awareness online or bring menus to life, but described these as examples which have “little real relevance on the factory floor”.
“Many of the articles one reads online – and this does not just apply to food and drink – are nothing more than collections of marketing sound bites, with lots from senior employees making bold and fantastical statements about the exploitation of AR,” Stone ventured. “For example, one company claims: ‘With a vast number of companies in the food industry adopting AR for its operations … there is no doubt augmented reality will be used in the food industry in full fledge soon’. Vast? No doubt? Full-fledged? Plenty of conjecture, but time and time again, no case studies, no evidence.”
He added: “One paper presented at the 54th CIRP conference on Manufacturing Systems in 2021 concluded that ‘Although the food industry has seen a rise in the adoption of Industry 4.0 technologies, the implementation of AR remains significantly low … Apart from the gaming industry, no other industry has figured out the AR-based business model that will last for significant years’.
EyeSucceed
Conversely, however, one company does appear to be making strides in AR and food.
NSF International in partnership with TeamViewer has created a hands-free, interactive and intuitive AR device that it says has already reaped results.
“A leading quick-service restaurant client saw that using EyeSucceed contributed to avoiding lost revenue due to equipment downtime and reducing maintenance costs by up to 75%,” NSF’s John Rowley, senior vice president of global food and ISR told Food Manufacture.
Essentially, the duo has developed a training modality that uses AR smart glasses paired with proprietary software to improve virtual and in-person training, consistency and operational efficiency – its name, EyeSucceed.
Commenting on the potential benefits of such technology, Rowley said: “The use of voice- and eye-controlled AR applications giving step-by-step instructions to workers on smart glasses, such as EyeSucceed, can support training employees at their workstations, streamlining the onboarding process and reducing the pressure and costs of peer-to-peer training. The inbuilt corrective intelligence detects employee errors and equipment deviations in real time. The result is a properly trained employee, reducing food safety and employee safety risks.”
According to Rowley, the technology has been designed to reduce maintenance and travel costs, and improve training and employee retention, whilst also creating opportunity for streamlined practices, such as the creation of self-audits to make reporting easier and potentially eliminating the need for on-site visits.
The challenges of AR in factories
But as Stone explains deploying AR is not without its challenges.
“Many years ago, whilst involved in a Human Factors study of the deboning process line of a well-known supplier of processed chicken products to fast food outlets, I remember thinking that there must be a more intuitive, safe and hygienic way of training employees, and visiting researchers like me,” said Stone.
“I often wonder, had the technology been available, whether or not AR would have significantly improved training, simplified the carcass preparation process, provided templates for special cuts, minimised errors and accidents, enhanced hygiene practices and so on. Could AR have been used to overlay a set of manual handling instructions, possibly even a voice-activated checklist of functions related to food processing or quality control activities, for example?
“In providing such functions, perhaps AR could help in reducing the incidences of product recalls by minimising contamination, spoilage, or, as was a significant issue with chicken deboning, wasteful breast cuts. Of course, now that chicken processing has become much more automated, these issues are of less concern than back then, but AR wearables in an environment that was cold, damp and dangerous? Perhaps not. But the concept would have been very interesting to evaluate.”
Alongside considerations around health and safety, which Stone pointed to, Rowley also flagged a strong internet connection as vital.
“Following the pandemic, we have seen that the food industry is looking for even more ways to leverage new technologies to improve training and efficiency while maintaining quality and consistency. However, as with the adoption of any new technology, there is a learning and adoption curve,” explained Rowley.
“To use the devices to their full potential, including live streaming functionalities, you need a reliable, strong Wi-Fi connection throughout a facility, which has been a challenge some of our clients had to overcome.
“The success of any disruptive technology depends on its ability to deliver demonstrated use cases in saving money or solving problems, along with a proven return on the financial investment needed. When tools such as smart glasses were first introduced more than a decade ago, they were launched without having effective use-case solutions.”
Indeed, one only needs to consider the laser, which when first invented, appeared to have no application.
An appetite for AR
Stone added that the aftermath of Covid and the effect on product prices as a result of the war in Ukraine, means that many eyes are now turning towards the promises of Industry 4.0.
“AR appears regularly within articles summarising Industry 4.0, more as a concept than an actual working solution on multiple factory floors. Nevertheless, AR is gaining some traction, particularly in the provision of maintenance support and supervision, or collaboration, for local or remote employees, using so-called ‘digital twins’.”
However, he acknowledged that many of these examples existed in the aerospace and automotive industry and so far, in-depth coverage of similar applications in food and drink is limited.
Commenting on the potential advantages of AR, Stone said: “The honest answer to this question is that we simply don’t know.”
He added that there are many publications which boast the advantages of this technology but few that offer “properly executed case studies to which one can turn to for reliable evidence to support the adoption of AR in one’s own business”.
The only way is up
However, along with this technology finding better use cases within the sector as Rowley stated, Stone said that since its introduction in the 1990s, AR has improved significantly.
“Just look at the technology on show at this year’s Computer Electronics Show (CES) in Las Vegas,” Stone said. “New, lightweight AR glasses, improved fields of view, improved high-contrast colour displays, optics that allegedly adapt to highly variable environmental lighting conditions.
“And we should not forget that AR is not just about wearable displays but can work equally as well – if not better for certain applications – when implemented using smartphones and tablets. There have even been early developments in AR contact lenses reported over the past two years, but the key player in this field has changed developmental focus. Thus, the future of this technology is unclear. Such lenses may well help to overcome the ongoing privacy issues associated with AR glasses, but it remains to be seen how well they are adopted – in both consumer and industrial settings.”
His advice to interested parties is to treat the data “carefully” as it doesn’t always “stand up to academic or scientific scrutiny”.
