The New iPad Pro’s LiDAR Sensor Changes the Augmented Reality Game

Learn how a time-of-flight sensor improves augmented reality.

For a major launch such as this, Apple normally holds a press conference in their Cupertino headquarters. However, due to growing health concerns and widespread retail shutdowns, the company opted instead for an online-only launch for their latest device.

Sporting a powerful 8-core processor and LiDAR scanner, the iPad Pro looks more like a computer replacement than ever before.

Improvements

This launch, undoubtedly, increased the “pro” factor of the iPad Pro. The redesigned model of the iPad Pro was originally launched in 2018, which was a significant improvement over previous generations, as it featured smaller bezels, a new form factor, Face ID, and an advanced processor. This year, Apple kept the same form factor but made significant under-the-hood improvements.

Cameras and Sensors

The most visible change to the second-generation iPad Pro is the addition of a camera and LiDAR sensor. The “camera bump” on the device more closely resembles the iPhone 11 and iPhone 11 Pro lineup. The first camera is a standard 12MP Wide camera, and the second camera is a 10MP Ultra Wide camera. You’ll learn more about the LiDAR sensor a bit later.

This arrangement allows you to take better-quality photos and videos with your iPad Pro—but do you really want to be the person who uses a 12.9″ tablet to videotape a concert? A more plausible use for the upgraded cameras would be for stationary recordings, such as interviews.

Microphones

Going hand-in-hand with the upgraded cameras on the new iPad Pro, it also has an upgraded array of microphones. An increase from three mics on the previous generation of the device, the new version has five microphones and promises “pro-audio” with “studio-quality mics.”

The new audio-video upgrades of the device shows that Apple is targeting the film industry with their new product, for on-the-go shoots or quick editing. However, it’s unclear how much better the quality is from the previous generation . We’ll learn more when people begin using it and sharing content.

Processor and Wi-Fi

The latest version of Apple’s iPad Pro also gets upgrades on the inside. Apple’s custom A12Z Bionic Chip sports an eight-core GPU and CPU, which will significantly increase the performance of the device in gaming and video editing. In addition, it features enhanced LTE support up to one gigabit, and a new Wi-Fi 6 module, which isn’t even present on their Mac lineup yet.

Cursor and Magic Keyboard

One of the upgrades which makes the iPad Pro more like a computer is the added support for a cursor and the trackpad on Apple’s new Magic Keyboard. This allows for a more computer-like interaction with the device, while maintaining a fluid, touch-based interface. In other words, they’ve made sure that their cursor support doesn’t take away from their touch-first device.

The new keyboard has a “floating design,” which means that it suspends the device in the air for more flexibility in terms of viewing angles. In addition, it provides an additional USB-C port, in case you need more than one. Of course, it has a new trackpad to control the cursor and a backlit keyboard for nighttime use.

Augmented Reality

So now you’ve gotten a quick look at how the iPad Pro has changed—but what’s with all the buzz about augmented reality (AR)? Well, it turns out that the LiDAR sensor on the new device makes AR a whole lot more accurate and stable. This sensor allows the iPad to obtain a precise map of the surroundings, similar to how Face ID measures the contours of your face.

What is LiDAR?

You may be wondering what this “magical” new sensor is and how it works. The acronym LiDAR stands for Light Detection and Ranging. A LiDAR scanner fires hundreds of thousands of laser beams and measures the time it takes for that light to return. Using this data, it’s able to measure how far away each point is, resulting in an accurate 3D map of the surroundings. Because of this, LiDAR scanners are often referred to as time-of-flight sensors.

The technology is currently used for a variety of different tasks. For years, space satellites have used LiDAR to map the surface of the Earth. More recently, LiDAR has made its way into driverless vehicles to precisely map cars, traffic lights, and roads. Now, the technology is being used on the new iPad Pro—and likely on future smartphones and tablets—to improve its ability to create immersive augmented reality experiences.

LiDAR and AR

Previously, devices could only assess the surrounding area using their built-in cameras, but with the addition of a LiDAR Scanner, the iPad Pro can use a precise 3D map to place virtual objects, instead of a rough, camera-based model of the surroundings. This, in turn, allows objects to actually stay where they were placed, as opposed to moving around.

Developers who’ve used ARKit to develop AR-enabled apps for the iPad will automatically get the benefits of the newly improved hardware, and if you’re thinking of building an AR-based app, now is a great time to jump in. In the image shown above, all of the objects you see are virtual, and the user is playing on an empty rug. This illustrates the improvements made to the augmented reality technology on the new iPad Pro.

Future Devices

There’s no doubt that the iPad’s use of LiDAR technology will revolutionize the landscape of augmented reality on mobile devices. Over the next few months, we can expect other device manufacturers to follow suit and release their own products with such technology. In terms of Apple, we can expect them to introduce such technology in future version of their iPhones and possibly on other devices as well.

Conclusion

It’s clear that Apple’s launch brought the iPad Pro closer to a “pro” device and a laptop replacement, while keeping it’s reputation as a user-friendly tool for the masses. The new sensors, cameras, and processors provide developers a great new platform to engage their users and provide them a multi-purpose device great for both work and play.

As we learn more about this new device, we’ll know more about the great ways that LiDAR improves augmented reality experiences, and we can trust that our great developer community will create new, innovative apps that take advantage of this technology.

Be sure to smash that “clap” button as many times as you can, share this tutorial on social media, and follow me on Twitter.

Fritz

Our team has been at the forefront of Artificial Intelligence and Machine Learning research for more than 15 years and we're using our collective intelligence to help others learn, understand and grow using these new technologies in ethical and sustainable ways.

Comments 0 Responses

Leave a Reply

Your email address will not be published. Required fields are marked *

wix banner square