Why Apple’s ARKit Will Help Businesses (Finally) Take AR to the Masses

Two weeks ago at its annual Worldwide Developer’s Conference (WWDC), Apple announced a new framework called ARKit that will dramatically simplify the process of building augmented-reality apps for iPhone and iPad.

Although Apple is late to the AR game, the tech giant has the key ingredients to help developers and businesses bring AR to the masses. More on that later.

What does ARKit do?

SOURCE: iOS App Store

For those unfamiliar with the term, “augmented reality” refers to the process of mapping digital objects and information into three-dimensional space, and overlaying those objects on top of a person’s field of view (either through the on-screen camera of a phone/tablet, or through a headset). Pokémon Go is the most frequently referenced recent example of this tech: You can see the world around you through your iPhone screen, along with the game controls and creatures to catch.

Historically, building an app with enough understanding of the device’s surroundings to make this type of experience possible has required enormous engineering overhead. Only a small number of developers had the expertise and the time to create such an app.

Apple will lower the barrier to entry with ARKit. The framework, included with Apple’s fall 2017 release of iOS 11, handles much of the heavy lifting that developers previously had to do to support AR. The framework includes powerful tools to help process user context, with underlying technologies such as:

• Visual Inertial Odometry (VIO) to understand where you are in space and where a device camera is looking.

• Scene understanding and scale estimation via plane detection and hit testing.

• Light estimation, so that as a scene darkens, virtual objects also darken to make sure they continue to look natural within their environment.

Also, because Apple builds the hardware and the operating system, ARKit can coordinate between the camera, motion sensors, CPU, and graphics chip in ways that other third-party developers are unable to do. This means that Apple is able to serve up AR information in a manner that takes a dramatically lower toll on battery life and performance than previous approaches. And it gives app developers more resources to work with when building their own custom experiences on top of ARKit.

The benefits of this optimization between hardware and software are significant. Apps built with ARKit can be used on any iPhone or iPad with an A9 chip or newer under the hood. I’ll explain why that matters in a bit.

But AR has been hyped for years. Is ARKit really going to make a difference?

Apple is rarely the first-to-market with any technology. But the company has an incredible track record of creating ecosystems that provide a foundation for new products (e.g. iTunes for iPod and the App Store for the iPhone).

The same is true with ARKit. Without going into the decades-long history of AR tech, there are a number of household brand names currently in the space.

Snapchat has included the ability to overlay amusing computer-generated effects on top of selfies for several years, and in April announced its World Lenses feature, which allows additional 3D objects to be added into Snaps.

In April, Facebook announced its Camera Effects platform and related AR Studio for developers to build frames and effects for use with Facebook’s apps.

Microsoft released its HoloLens developer kit in March of 2016, and a handful of businesses are already investing in hardware and development for the platform, with specific use cases such as education, enterprise hands-free training and on-the-job heads-up-display tools.

And Google first released its AR platform Tango back in 2014 and has made steady advances every year since. The newest release of Tango, unveiled at the most recent Google I/O in May, has capabilities that objectively are far beyond what Apple announced at WWDC. Google’s AR technological maturity is evident in its Visual Positioning Service (VPS) and high-resolution mapping of indoor spaces to produce floorplans and 3D models, to name just a few highlights. Google also demonstrated several partnership projects in development with global brands such as BMW, Pottery Barn, and Lowe’s.

SOURCE: Pottery Barn

So, what makes Apple’s ARKit different?

While other companies have been in the AR space for much longer, Apple is once again in a position to do what it is has executed time and again: allow others to lay a foundation, then swoop into a market with exactly the right ingredients for mass adoption where others have fallen short. In the case of AR, what are those key ingredients? Two words: Scale and Sales.

Key Ingredient 1: Scale

Each of the current players in AR has inherent self-limiting constraints on its total addressable market. For example:

• For Microsoft, cost is a barrier to adoption, with the required HoloLens hardware costing up to $5,000.

• For Google, Tango can only run on phones and tablets with Tango-specific hardware, of which there are only two devices available for sale.

• For Snapchat and Facebook, all AR experiences are retained within their respective apps, so developers are limited in what types of experiences make sense within a social network context. Businesses that create AR experiences for their customers on top of these networks — though massive — will always be at the mercy of Snapchat and Facebook.

On the other hand, recall earlier that I mentioned Apple’s ARKit will run on any iPad or iPhone with an A9 processor or newer. That means every iPhone released since the iPhone 6s in 2015, and every iPad (except mini) released since the iPad Pro (12.9-inch) in 2015. Translation: in the fall when iOS 11 is released, there will immediately be an audience of hundreds upon hundreds of millions of customers who already own everything they need to use augmented reality apps.

In addition, because ARKit is available for all developers and businesses to use within their own independent apps, there aren’t as many branding or experience constraints as building on social platforms. App makers can build rich custom experiences limited only by their imagination.

Key Ingredient 2: Sales

In addition to a massive audience of customers with AR-ready hardware already in-hand, Apple has another very convincing incentive for developers to get on board: money. In 2016, Apple brought in an estimated 60 percent of all mobile app sales, according to BMO Capital, and in June Apple announced that App Store earnings for developers have now exceeded $70 billion.

No other company has announced anything even close regarding developers’ earnings from their app platforms.

The constraints of scale and sales faced by the other players in AR present an uphill battle to reach a critical mass of developers. It’s much easier to justify the business case for building an iOS AR experience when the reach is greater and it takes less time/money because the toolkit and hardware ecosystem are more complete.

Apple has a huge, incredibly loyal customer base with a proven will and means to invest in the app ecosystem. Developers and companies who choose to build for ARKit can do so with confidence that if they build a compelling experience, there will be a paying audience ready and waiting.

So, what does ARKit mean for developers?

ARKit provides app designers and developers with the freedom to focus on building great experiences and great content without the heavy lifting it took in the past. No more spending weeks or months rolling your own AR renderer or world-tracking engine, performing object mapping and spatial tracking using computer vision and physics algorithms. Instead, you can jump right in at the fun part: designing and building the experience you envision.

ARKit also builds upon many of Apple’s existing technologies, including Metal, SpriteKit, and SceneKit, and also offers integrations with Unity and Unreal Engine. This means you’ll need to learn the specifics of the ARKit API, but you may already have all of the other skills you need to start building right away.

What does ARKit mean for businesses?

The potential use cases for AR are significant, from retail/virtual shopping, to training, to customer engagement, to gaming, and beyond. One of our recent hackathon projects resulted in a proof-of-concept app that could recognize photos of fruits and vegetables and make recommendations for recipes with those ingredients. A grocery store, for example, could use something similar to build brand loyalty and offer additional value for their customers. For a few more demos to jump-start your imagination, watch the WWDC keynote video from about the 1:28:30 to 1:32:30 timestamp.

Apple is also adding many other features in iOS 11 to help consumers locate and communicate with businesses, such as indoor mapping of malls and airports in their Maps app, and Business Chat via iMessage (in beta until 2018). It’s a small leap to envision ways in which this connection to brands could be extended within Apple’s Maps app through use of ARKit in a follow-on update to iOS. In fact, some have been anticipating this type of integration for a few years.

As 9to5mac wrote in 2015, “Pointed towards a cafe, for example, the screen could show a virtual view of menu items or daily specials. If the user points her phone toward a street, a virtual outline of local businesses, restaurants, shopping stores, or coffee shops could appear.”

Beyond these integrations, the possibilities become even more powerful when considering combinations of ARKit with other new frameworks that Apple opened to developers this year — including machine learning through CoreML, object and face detection through the Vision framework, built-in QR code recognition, natural language processing, and more.

Now is the time for companies to begin experimenting and thinking about whether there is a fit for AR within their business model. With ARKit, Apple has offered up the right ingredients for businesses and developers to build compelling AR experiences — along with the scale and sales for the masses to follow.

Thinking of building an augmented reality experience?

If you have an idea for an AR app and want some advice, we’re happy to help. In addition to our design and development services, we offer free consultations. So, let’s set up a time to talk!