Apple Unleashes AR Backed by 'The Walking Dead,' Ikea, 'A Very Hungry Caterpillar'

Augmented Reality apps feature furniture, animated meme, zombies and more

Apple Unleashes AR Backed by 'The Walking Dead,' Ikea, 'A Very Hungry Caterpillar'

If the past few years were the age of visor-wearing virtual reality – a brief period that saw the multi-billion-dollar purchase of a VR company by Facebook, a hybrid, phone-powered attempt at mainstream acceptance for the tech and then sudden, momentous price-drops for the pricey headgear – then Apple seems to be betting that 2018 will be the kick-off of a much more successful age of augmented reality.

Apple rolled out details about its support for augmented reality earlier this year during the company's annual Worldwide Developers Conference. Software baked into the upcoming iOS 11 will allow supported iPhones and iPads to run augmented reality programs and make it easier for developers to create them.

The unveiling – lost as it almost was amid the minutely detailed additions coming with the operating system – still underlined the importance the company is putting into the technology.

Earlier this month, Apple significantly increased its presence with the technology, essentially doubling down on what it thinks will be a pivotal, perhaps even transformative technology.

A group of Apple-invited developers joined Greg Joswiak, Apple's vice president of iOS, iPad and iPhone marketing, in New York City last week at press events designed to immerse journalists into Apple’s view of AR.

Joswiak spoke on background, but the developers there from more than half-a-dozen well-known companies were more than willing to share their own insight into the importance of the tech as they hawked their virtual goods.

The hours long session included demonstrations from Giphy, Ikea, Touchpress, Food Network, two game developers and AMC networks. The result was a look into technology that through the iPhone and iPad will allow users to plop virtual Ikea couches into their real homes, fill the air around them with sparking, eye-rolling animated memes turned into floating Gifs and fight the walking dead alongside the likes of Daryl, Rick and Michonne.

The embargo for those sessions lifts today, coincidentally the same day that Google announced its own push for more deeply engrained support for augmented reality on Android phones. With so much going on in the past month around virtual reality and now augmented reality, we seem perched on the verge of a realities war.

Augmented reality, like virtual reality, is a technological way for software developers to mess with what you see and hear. In the case of virtual reality, everything a user sees is created and manipulated by the software and then shown to a user, typically, through a headset that includes a wide-angle screen, which is strapped to your face. With augmented reality, developers create objects that are overlaid onto the real world. That view can be seen through glasses, headsets or, in the case of Apple's upcoming AR support, a phone. The phone shows what it is pointed at using its built in camera and then the software adds elements to that real world, live shot.

Virtual reality made a big push in the past year thanks to Facebook's sudden interest in the technology and its decision to purchase Oculus for $1 billion. The injection of money perked the interest of a number of other companies that saw the reemerging tech as a chance to hop on new form of entertainment.

Facebook, Google and Apple are all showing big interest in AR this year.

PlayStation, HTC, Samsung, Google and Facebook, as well as some smaller companies, all released virtual reality headsets. While the technology continues to thrive, interest in it seems to have cooled recently, with a number of the headsets receiving price drops and rumors that HTC is trying to rid itself of the VR arm of its company.

Meanwhile, the year has seen a sudden interest in augmented reality, which is seen by some as an easier, less expensive reality technology to invest in or support.

Facebook expressed interest in AR earlier this year as did Google and, of course, Apple.

The Demos
Apple kicked off the morning with perhaps the strangest concept of the day: Augmented reality, animated gifs.

"We love GIFs because they help us communicate, communicate a lot," says Ralph Bishop, head of design at Giphy. "Our first goal was to help you find and share these things with our help. After that we wanted to help you make your own gif with our own app. As we move into augmented reality, we want to be there, too."

The Giphy World app, which is hitting the app store "soon," launches with a live view through your device's camera. Users can then simply touch and drag animated gifs into the world as seen through the lens. Moving the camera around lets you change where you place the GIF and the app keeps track of everything you put into the world. Once finished, you can record the scene as a video and share it with a friend or you can place gifs into a friend's world and let them hunt around for them using the app.

“These are like treasure maps for your friends to look for and find,” Bishop said.

Ikea’s demonstration was much more down to earth. The Ikea Place app allows users to shop for furniture from the store and then, using a devices camera, drop the furniture into the real world scene to see how it looks or fits in a room.

The app knows the exact size of the furniture and how big it should be relative to the real world furniture around it. It can also drop new furniture right on top of your old stuff, if you’re looking to replace what you already own. The sizing is 98 percent accurate, says Michael Valdsgaard, leader of digital transformation at Ikea.

Once placed, a user can walk up to and around the furniture to examine everything from what it looks like from underneath to the stitching.

"You're going to look back on a moment like this and think, 'This is where it started,'" Valdsgaard says, enthusing about the coming of the tech to iPhone and iPad. "This is a complete game changer."

The least interesting of the tech demos was a strange little addition to Food Network's In the Kitchen app. With it, users can create and customize virtual desserts on a counter top and then share them with friends.

The mix of non-gaming augmented reality apps shown during the day was as hit or miss as it was diverse. Where some of the apps, like Ikea’s furniture placer, was so fine-tuned that the AR furniture viewed through the phone screen seemed to have weight, others like Food Network's app featured a baffling lack of originality made worse by an augmented reality cupcake gently twitching around on the real table.

The mix of good and bad highlights one of the struggles Apple may face once a new flood of apps, this time relying on augmented reality, hit its already overwhelmed store front. Perhaps not surprising, the best demonstrations of the day all ended up being sorts of games.

AR Gaming
The Very Hungry Caterpillar, a much-loved children’s story by Eric Carle, is the sort of creation that really doesn't need any sort of touching up. It’s done just fine relying on paper and the printed word to get its point across for more than 45 years now.

And yet, the augmented reality take on the classic was the only thing shown last week that seemed to elicit genuine smiles and looks of wonder among the gathered press.

"The book is a book," says Barry O’Neill, Touch Press CEO, who started working with Carle back in 2015. "You don't touch it. It's a wonderful, wonderful experience."

But the folks at Touch Press apparently couldn't help themselves and decided to try something new with Apple's AR tech and put the children into the world of the caterpillar.

In the experience, children use an iPhone or iPad to first watch the caterpillar hatch from a colorful egg in their own house. As the caterpillar grows, children can watch it eat from digital trees or feed the creature bits of fruit. They can also play with it using digital toys. Eventually, the caterpillar goes to sleep, forming a new egg which hatches to release a beautiful butterfly.

Following the butterfly with an iPhone, O'Neill showed it gently floating up, up, up to the ceiling to join a host of similar butterfly gently flying around the room. Then a new egg forms and the cycle starts all over again.

"This is what we've done in a few weeks, really," O'Neill says.

Next, developer Next Games and executives from AMC showed off an upcoming, location-based augmented reality game based on The Walking Dead.

Think of it as Pokémon Go, only instead of tracking down and capturing cute, colorful creations, you hunt down and kill zombies, sometimes with the help of life-sized recreations of the show’s main characters like Rick, Daryl and Michonne.

In the demo, Mikael Achren, creative director at developer Next Games, showed the room around us, then turned to spot a full-sized zombie through his phone, walking toward him. He aimed a crossbow, seen jutting out from his view on the phone, and tapped the screen to shoot a bolt into the zombie and kill it. Soon the room was filled with shuffling zombies and Achren was swiveling back and forth, taking them down with shots from the crossbow. After a few minutes he noted that Michonne had joined him and turns to show her nearby, dispensing zombies with her Katana.

Once an encounter was complete, Achren took a picture of digital Michonne standing next to the real Clayton Neuman, vice president of games and entertainment applications at AMC.

Achren says the game, which doesn't yet have a release date, will be free to play with some content available for purchase with real money. It will track where a player is in the world and deliver zombies that fit in with the setting. That location-based gaming is one of the bigger draws for creating this game, he adds.

"With The Walking Dead, you're always wondering what the world would be line in your location," he says. "I always wonder what the world would be in Helsinki. It's one of the biggest questions fans of the show have. This will give you insight into what is happening in your own neighborhood."

Once the game launches, it will feature a variety of ways to do away with zombies including grenades, the Katana and guns.

Finally, longtime augmented reality game creator Climax Studios showed off its take on an AR puzzle game.

In Arise, players will have to readjust their line of sight by walking around, tilting the iPhone or iPad and viewing a floating puzzle world to help a hero get to the top of a series of islands. During the demo, we watched as a tiny hero, its head lost in a Spartan helmet, walked around the island, pausing at seemingly insurmountable passes until the camera was moved to change the perspective and create a path. Once the hero was guided to the top of the island, he grabbed a glowing relic and the floating land slowly transformed into a giant golem.

Simon Gardner, CEO at Climax Studios, says the studio decided to hold a game jam, crunching on ideas and turning them into prototypes, shortly after Apple released the software for its augmented reality.

While the studio has experience in virtual reality and augmented reality games, the sheer potential size of Apple's player base can't help but attract developer attention, Gardner says. "On launch day, when they release iOS 11, hundreds of millions of people will be able to try these apps. That's more than any console or store. That's very attractive."