by Peter H. Diamandis, MD: Augmented reality (AR) has already exceeded over 2,000 AR apps on over 1.4 billion active iOS devices…
Even if on a rudimentary level, the technology is now permeating the consumer products space.
And in just the next four years, the International Data Corporation (IDC) forecasts AR headset production will surge 141 percent each year, reaching a whopping 32 million units by 2023.
AR will soon serve as a surgeon’s assistant, a sales agent, and an educator, personalized to your kids’ learning patterns and interests.
In this fourth installment of our five-part AR series, I’m doing a deep dive into AR’s most exciting industry applications, poised to hit the market in the next 5-10 years.
Let’s dive in.
Healthcare
(1) Surgeons and physicians
Whether through detailed and dynamic anatomical annotations or visualized patient-specific guidance, AR will soon augment every human medical practitioner.
To start, AR is already being used as a diagnosis tool. SyncThink, recently hired by Magic Leap, has developed eye-tracking technology to diagnose concussions and balance disorders. Yet another startup, XRHealth, launched its ARHealth platform on Magic Leap to aid in rehabilitation, pain distraction, and psychological assessment.
Moreover, surgeons at the Imperial College London have used Microsoft’s HoloLens 1 in pre-operative reconstructive and plastic surgery procedures, which typically involves using CT scans to map blood vessels that supply vital nutrients during surgery.
As explained by the project’s senior researcher, Dr. Philip Pratt, “With the HoloLens, we’re now doing the same kind of [scan] and then processing the data captured to make it suitable to look at. That means we end up with a silhouette of a limb, the location of the injury, and the course of the vessels through the area, as opposed to this grayscale image of a scan and a bit more guesswork.”
Dramatically lowering associated risks, AR can even help surgeons visualize the depth of vessels and choose the optimal incision location.
And while the HoloLens 1 was only used in pre-op visualizations, Microsoft’s HoloLens 2 is on track to reach the operating table. Take Philips’ Azurion image-guided therapy platform, for instance. Built specifically for the HoloLens 2, Azurion strives to provide surgeons with real-time patient data and dynamic 3D imagery as they operate.
Moreover, AR headsets and the virtual overlays they provide will exponentially improve sharing of expertise across hospitals and medical practices. Niche medical specialists will be able to direct surgeons remotely from across the country (not to mention the other side of the planet), or even view annotated AR scans to offer their advice.
Magic Leap, in its own right, is now collaborating with German medical company Brainlab to create a 3D spatial viewer that would allow clinicians to work together in surgical procedures across disciplines.
But beyond democratizing medical expertise, AR will even provide instantaneous patient histories, gearing doctors with AI-processed information for more accurate diagnoses in a fraction of the time.
By saving physicians’ time, AR will therefore free doctors to spend a greater percentage of their day engaging in face-to-face contact with their patients, establishing trust, compassion, and an opportunity to educate healthcare consumers (rather than merely treating them).
And when it comes to digital records, doctors can simply use voice control to transcribe entire interactions and patient visits, multiplying what can be done in a day, and vastly improving the patient experience.
(2) Assistance for those with disabilities
Today, over 3.4 million visually impaired individuals reside in the US alone. But thanks to new developments in the AI-integrated smart glasses realm, associated constraints could soon fade in severity.
And new pioneers continue to enter the market, including NavCog, Horus, AIServe, and MyEye, among others. Microsoft has even begun development of a “Seeing AI” app, which translates the world into audio descriptions for the blind, as seen through a smartphone’s camera lens.
During the Reality Virtual Hackathon in January, hosted by Magic Leap at MIT, two of the top three winners catered to disabilities. CleARsite provided environment reconstruction, haptic feedback, and Soundfield Audio overlay to enhance a visually impaired individual’s interaction with the world. Meanwhile, HeAR used a Magic Leap 1 headset to translate vocals or sign language into readable text in speech bubbles in the user’s field of view. Magic Leap remains dedicated to numerous such applications, each slated to vastly improve quality of life.
(3) Biometric displays
In biometrics, cyclist sunglasses and swimmer goggles have evolved into the perfect medium for AR health metric displays. Smart glasses like the Solos ($499) and Everysight Raptors ($599) provide cyclists with data on speed, power, and heart rate, along with navigation instructions. Meanwhile, Form goggles ($199)—just released at the end of August—show swimmers their pace, calories burned, distance, and stroke count in real-time, up to 32 feet underwater.
Accessible health data will shift off our wrists and into our fields of view, offering us personalized health recommendations and pushing our training limits alike.
Retail & Advertising
(1) Virtual shopping
The year is 2030. Walk into any (now AI-driven, sensor-laden, and IoT-retrofitted) store, and every mannequin will be wearing a digital design customized to your preferences. Forget digging through racks of garments or hunting down your size. Cross-referencing your purchase history, gaze patterns, and current closet inventory, AIs will display tailor-made items most suitable for your wardrobe, adjusted to your individual measurements.
An app available on most Android smartphones, Google Lens is already leaping into this marketplace, allowing users to scan QR codes and objects through their smartphone cameras. Within the product, Google Lens’s Style Match feature even gives consumers the capability to identify pieces of clothing or furniture and view similar designs available online and through e-commerce platforms.
(2) Advertising
And these mobile AR features are quickly encroaching upon ads as well.
In July, the New York Times debuted an AR ad for Netflix’s “Stranger Things,” for instance, guiding smartphone users to scan the page with their Google Lens app and experience the show’s fictional Starcourt Mall come to life.
But immersive AR advertisements of the future won’t all be unsolicited and obtrusive. Many will likely prove helpful.
As you walk down a grocery store aisle, discounts and special deals on your favorite items might populate your AR smart glasses. Or if you find yourself admiring an expensive pair of pants, your headset might suggest similar items at a lower cost, or cheaper distributors with the same product. Passing a stadium on the way to work, next weekend’s best concert ticket deals might filter through your AR suggestions—whether your personal AI intends them for your friend’s upcoming birthday or your own enjoyment.
Instead of bombarding you at every turn on a needed handheld device, ads will appear only when most relevant to your physical surroundings— or toggle them off, and have your personal AI do the product research for you.
Education & Travel
(1) Customized, continuous learning
The convergence of today’s AI revolution with AR advancements gives us the ability to create individually customized learning environments.
Throw sensors in the mix for tracking of neural and physiological data, and students will soon be empowered to better mediate a growth mindset, and even work towards achieving a flow state (which research shows can vastly amplify learning).
Within the classroom, Magic Leap One’s Lumin operating system allows multiple wearers to share in a digital experience, such as a dissection or historical map. And from a collaborative creation standpoint, students can use Magic Leap’s CAD application to join forces on 3D designs.
In success, AR’s convergence with biometric sensors and AI will give rise to an extraordinarily different education system: one comprised of delocalized, individually customizable, responsive, and accelerated learning environments.
Continuous and learn-everywhere education will no longer be confined to the classroom. Already, numerous AR mobile apps can identify objects in a user’s visual field, instantaneously presenting relevant information. As user interface hardware undergoes a dramatic shift in the next decade, these software capabilities will only explode in development and use.
Gazing out your window at a cloud will unlock interactive information about the water cycle and climate science. Walking past an old building, you might effortlessly learn about its history dating back to the sixteenth century. I often discuss information abundance, but it is data’s accessibility that will soon drive knowledge abundance.
(2) Training
AR will enable on-the-job training at far lower costs in almost any environment, from factories to hospitals.
Smart glasses are already beginning to guide manufacturing plant employees as they learn how to assemble new equipment. Retailers stand to decimate the time it takes to train a new employee with AR tours and product descriptions.
And already, automotive technicians can better understand the internal components of a vehicle without dismantling it. Jaguar Land Rover, for instance, has recently implemented Bosch’s Re’flekt One AR solution. Training technicians with “x-ray” vision, the AR service thereby allows them to visualize the insides of Range Rover Sport vehicles without removing their dashboards.
In healthcare, medical students will be able to practice surgeries on artificial cadavers with hyper-realistic AR displays. Not only will this allow them to rapidly iterate on their surgical skills, but AR will dramatically lower the cost and constraints of standard medical degrees and specializations.
Meanwhile, sports training in simulators will vastly improve with advanced AR headset technology. Even practicing chess or piano will be achievable with any tabletop surface, allowing us to hone real skills with virtual interfaces.
(3) Travel
As with most tasks, AI’s convergence with AR glasses will allow us to outsource all the most difficult (and least enjoyable) decisions associated with travel, whether finding the best restaurants or well-suited local experiences.
But perhaps one of AR’s more sophisticated uses (already rolling out today) involves translation. Whether you need to decode a menu or access subtitles while conversing across a language barrier, instantaneous translation is about to improve exponentially with the rise of AI-powered AR glasses. Even today, Google Translate can already convert menu text and street signs in real time through your smartphone.
Manufacturing
As I explored last week, manufacturing presents the nearest-term frontier for AR’s commercial use. As a result, many of today’s leading headset companies—including Magic Leap, Vuzix, and Microsoft—are seeking out initial adopters and enterprise applications in the manufacturing realm.
(1) Design
Targeting the technology for simulation purposes, Airbus launched an AR model of the MRH-90 Taipan aircraft just last year, allowing designers and engineers to view various components, potential upgrades, and electro-optical sensors before execution. Saving big on parts and overhead costs, Airbus thereby gave technicians the opportunity to make important design changes without removing their interaction with the aircraft.
(2) Supply chain optimization
AR guidance linked to a centralized AI will also mitigate supply chain inefficiencies. Coordinating moving parts, eliminating the need to hold a scanner at each checkpoint, and directing traffic within warehouses will vastly improve workflow.
After initially implementing AR “vision picking” in 2015, leading supply company DHL recently announced it would continue to use the newest Google smart lens in warehouses across the world. Or take automotive supplier ZF, which has now rolled out use of the HoloLens in plant maintenance.
(3) Quality assurance & accessible expertise
AR technology will also play a critical role in quality assurance, as it already does in Porsche’s assembly plant in Leipzig, Germany. Whenever manufacturers require guidance from engineers, remote assistance is effectively no longer remote, as equipment experts guide employees through their AR glasses and teach them on the job.
Transportation & Navigation
(1) Autonomous vehicles
To start, Nvidia’s Drive platform for Level 2+ autonomous vehicles is already combining sensor fusion and perception with AR dashboard displays to alert drivers of road hazards, highlight points of interest, and provide navigation assistance.
And in our current transition phase of partially autonomous vehicles, such AR integration allows drivers to monitor conditions yet eases the burden of constant attention to the road. Along these lines, Volkswagen has already partnered with Nvidia to produce I.D. Buzz electric cars, set to run on the Drive OS by 2020. And Nvidia’s platform is fast on the move, having additionally partnered with Toyota, Uber, and Mercedes-Benz. Within just the next few years, AR displays may be commonplace in these vehicles.
(2) Navigation
We’ve all seen (or been) that someone spinning around with their smartphone to decipher the first few steps of a digital map’s commands. But AR is already making everyday navigation intuitive and efficient.
Google Maps’ AR feature has already been demoed on Pixel phones: instead of staring at your map from a bird’s eye view, users direct their camera at the street, and superimposed directions are immediately layered virtually on top.
Not only that, but as AI identifies what you see, it instantaneously communicates with your GPS to pinpoint your location and orientation. Although a mainstream rollout date has not yet been announced, this feature will likely make it to your phone in the very near future.
Entertainment
(1) Gaming
We got our first taste of AR’s real-world gamification in 2016, when Nintendo released Pokémon Go. And today, the gaming app has now surpassed 1 billion downloads. But by contrast to VR, AR is increasingly seen as a medium for bringing gamers together in the physical world, encouraging outdoor exploration, activity, and human connection in the process.
And in the recently exploding eSports industry, AR has the potential to turn player’s screens into live action stadiums. Just this year, the global eSports market is projected to exceed US$1.1 billion in revenue, and AR’s potential to elevate the experience will only see this number soar.
(2) Art
Many of today’s most popular AR apps allow users to throw dinosaurs into their surroundings (Monster Park), learn how to dance (Dance Reality), or try on highly convincing virtual tattoos (InkHunter).
And as high-definition rendering becomes more commonplace, art will also grow more and more accessible.
Magic Leap aims to construct an entire “Magicverse” of digital layers superimposed on our physical reality. Location-based AR displays, ranging from art installations to gaming hubs, will be viewable in a shared experience across hundreds of headsets. Individuals will simply toggle between modes to access whichever version of the universe they desire. Endless opportunities to design our surroundings will arise.
Apple, in its own right, recently announced the company’s [AR]T initiative, which consists of floating digital installations. Viewable through [AR]T Viewer apps in Apple stores, these installations can also be found in [AR]T City Walks guiding users through popular cities, and [AR]T Labs, which teach participants how to use Swift Playgrounds (an iPad app) to create AR experiences.
(3) Shows
And at the recent Siggraph Conference in Los Angeles, Magic Leap introduced an AR-theater hybrid called Mary and the Monster, wherein viewers watched a barren “diorama-like stage” come to life in AR.
While audience members shared the common experience like a traditional play, individuals could also zoom in on specific actors to observe their expressions more closely.
Say goodbye to opera glasses and hello to AR headsets.
Final Thoughts
While AR headset manufacturers and mixed reality developers race to build enterprise solutions from manufacturing to transportation, AR’s use in consumer products is following close behind.
Magic Leap leads the way in developing consumer experiences we’ve long been waiting for, as the “Magicverse” of localized AR displays in shared physical spaces will reinvent our modes of connection.
And as AR-supportive hardware is now built into today’s newest smartphones, businesses have an invaluable opportunity to gamify products and immerse millions of consumers in service-related AR experiences.
Even beyond the most obvious first-order AR business cases, new industries to support the augmented world of 2030 will soon surge in market competition, whether headset hardware, data storage solutions, sensors, or holograph and projection technologies.
Jump on the bandwagon now—the future is faster than you think!
