The Future of Captions: Augmented Reality

Interest in augmented reality (AR) has grown exponentially in recent years and we can expect this trend to continue. This growth can be attributed in part to its great commercial value and future potential. Not only does this breakthrough technology allow to us to enhance reality, but it also provides many solutions. With new applications such as Pokemon Go and AR-enabled maps, AR is constantly pushing the boundaries of what’s possible. As more and more businesses begin to adopt AR, it will likely be everywhere rather than constrained to a few gag uses. 

Already, in conjunction with captions and subtitles, AR presents new opportunities for creating assistive devices. This article will explore how captions are currently being used in AR and where the technology is heading in the future.

What Is Augmented Reality And How Does It Work?

AR allows us to superimpose digital elements onto our real world environment. These digital elements can then be viewed on various devices such as phones, tablets and lenses. To create this enhanced reality, localization and depth tracking software is employed along with various other components. Some of these components include processors, cameras and sensors. The cameras and sensors scan the user’s environment and send the data to the processor. Once processed, the AR is able to locate objects and generate 3D information. Mirrors may also be used to facilitate proper image alignment, thus improving the viewing experience.

Currently, there are many different types of AR technology on the market. Marker-based AR is the most popular and it requires a special visual object to run. Often these objects come in the form of QR codes. When the object is scanned, a 3D model or animation is generated. Other types of AR technology are Markerless AR, Projection-based AR and Superimposition-based AR. 

The Difference Between AR And VR

With the rapid growth we are seeing in both sectors, it’s easy to see why you may confuse AR with Virtual Reality (VR). Although both use technology to create an immersive experience for users, there are key differences between the two.

In VR, the virtual reality completely supersedes your actual environment. Users are completely immersed in a world that is entirely computer generated. VR headsets are connected to a PC and completely block out your surroundings to transport your somewhere else. In fact, when turned off, wearing a VR headset is akin to wearing a secure blindfold. While most VR headsets are linked to PCs, there has been a rise in smartphone enabled experiences.

Unlike VR, AR enhances your real world environment rather than crafting an entirely new one. Text, images and 3D graphics are overlaid on top of your surroundings using portable devices like smartphones and tablets.

VR provides a more immersive experience while AR provides users with more freedom. Despite their differences, one thing is certain. Our future will enjoy a mix of both technologies and what they have to offer.

The Usage of Captions and Subtitles in AR 

At the moment, the biggest usage of captions and subtitles in AR is for assistive technology. Huge strides have been made to improve the lives of those who are deaf or hard of hearing. 

One such example of this that has received lots of attention is the use of smart glasses in theatres. This technology, dubbed Open Access Smart Capture, was developed by National Theatre and will change performances forever. Using AR alongside theatre projection systems, theatres can fully integrate captions into the artistic elements of performances. First, a 3D model of the stage set is used to build a digital twin of the physical thing. Next, the digital twin is aligned with its physical counterpart through the use of location markers. Employing location markers ensures that the captions will always be visible from any seat. Captions are then added to the digital version and everything else is removed barring the captions. Thus, when a user puts on the connected smart glasses, they will see perfectly displayed captions.

This means that deaf and hard of hearing audience members will not have to constantly switch their focus between the action onstage and the captions to the side. As a result, they can enjoy a significantly more immersive viewing experience. Providing access to culture and technology is a human right and these glasses will contribute greatly to cultural diversity.

The Future Of Captions And Subtitles In AR

As more companies begin to invest in AR, the technology is developing quickly.  Currently, AR is making massive strides when it comes to live captioning.

Universities have been working on using live captioning with smart glasses to help deaf and hard of hearing users navigate conversations. To start, a mic is calibrated to detect human speech. The detected speech is then processed by a smartphone sized micro-computer that the user can keep in their pocket. Finally, the resulting text is transmitted to a display that superimposes the text on the speaker. Although this setup achieves limited latency, the captions are quite obtrusive. Furthermore, the user’s own speech is also captioned which can create confusion. However, experts are working tirelessly to perfect the technology.

The Future of Augmented Reality Captions and Subtitles

AR technology is developing at a breakneck speed and the realm of possibilities is continuously expanding as it does. Right now, AR-based assistive technology is helping to improve the lives of many and in the future this will only become more prevalent. Adding captions and subtitles to technological innovations is a great way to increase inclusivity. Fortunately, with Rev, it doesn’t need to be complicated. You can get accurate and affordable captions on our platform.