Trying Star War's New Augmented Reality Game, Lightsaber in Hand

Hands-on with Star Wars: Jedi Challenges

Darth Maul stands about three feet in front of me, his back is about 16 inches from the window overlooking New York City's Avenue of the Americas. He's sort of uncomfortably jammed between the couch and the desk that's been pushed to one side of the office. He doesn't seem to mind, though. All of his focus seems to be on me and the lightsaber I hold in my hand.

Just to his side, certainly within lightsaber striking distance, two people from Disney sit on the couch. They're slightly bent down, intently staring up directly at my eyes. They're looking at the silvered visor that covers my face, they see what I see, but smaller, tiny reversed images of the augmented world in which I find myself immersed.

Available this holiday season, Disney's latest attempt at video games completely eschews the console for a smartphone-powered augmented reality headset, light-up beacon and a game-controller built into a very authentic looking lightsaber. And if it's successful, it's likely you'll be seeing versions of this AR entertainment pop-up tied to everything from Marvel's superheroes to Disney movies.

Star Wars: Jedi Challenges
The idea started about a year and a half ago with the light saber, says Mike Goslin, vice-president of advanced development at Disney Consumer Products and Interactive Media. "My team comes up with new businesses that are experiences," he says. "We wanted to do a light saber experience that hadn't been done before."

The team started with a physical light saber and tied it to audio, haptics and LED lights, but the experience still wasn't gelling Goslin says.

As a small, quick, responsive team, they tend to make use of everything, whether or not it turns into a product. So with the current system not working, they started to look around at other tech that might make the project come together. They looked at the Internet of Things, connected play, AI, virtual reality and augmented reality.

Ultimately, the team settled on creating an AR, not VR system for the project.

"We love VR, but one of the things about AR is that it's more social," he says. "I can see my friends. I can move around. I'm more connected with my environment. It's more mainstream. We love VR too, but we think AR is a more mass market technology."

Once the team settled on augmented reality, they started testing all of the products in development at the time.

"We experimented with a lot of things," Goslin says, "But we concluded it was best if we started creating a headset ourselves."

The team built out a prototype headset that is powered by a smartphone and then they took that prototype to Lenovo. "We ended up partnering with Lenovo because our strategy is complimentary to there's," Goslin says. "We went to them for manufacturing and design for manufacturing. We had a prototype, they made it something that would scale."

The team felt it was important for the end product to cost less than $200, which is one of the primary reasons they decided to have the headset powered by a smartphone. "We wanted it to be a great experience and as scalable as possible, but we wanted to do it for under $200," he explains. "We’re trying to break in a new category, and we didn't want to have people sweat the price." It also helped how ubiquitous high-quality smartphones are these days, he adds.

The Force Was With Me
At first glance, the Lenovo Mirage AR headset looks like a virtual reality headset. It still straps onto your face and has what appears to be a large screen jutting from the front. But instead of solid plastic, the front is a silvered, transparent piece of plastic. Before using the system, you have to pop a phone into a tray and slide it into the top of the headset, placing the front of the camera facedown. The headset includes two built-in fisheye sensors to provide inside-out positional tracking and pairs with a single tracking beacon that you place on the floor or a table in front of you.

The headset also pairs with a surprisingly detailed lightsaber designed as a replica of the one used by Anakin Skywalker, Luke Skywalker and Rey. The lightsaber is both a controller and a pointer, used to navigate menus, issue commands and, of course, smack away lasers or parry and attack in duels. Once activated, players can see the lightsaber emerge from the hilt and feel strikes and blocks, thanks to built in haptics.

I spent about 30 minutes messing around with the gear in a Rolling Stone office with its desk pushed to one side, leaving me about four feet of room in which to play. Once the beacon was placed on the floor and the headset turned on, a translucent map of the star system popped up in front of me, hovering a few feet above the beacon. I could walk around it, walk closer or retreat and it mostly stayed still, though it did jitter a little bit. Unlike with VR, I could still see the office and the visitors from Disney as I inspected the map.

The planets offer up access to the system's three main types of gameplay: holochess, lightsaber battles and a sort of boiled down real-time strategy game.

The lightsaber battles are the most expected experience provided by the system. Players can either take on key villains from the Star Wars universe in a duel, or hold their own against an army of laser-shooting, advancing robots.

My time in the duel had my taking on a life-sized Darth Maul. The force alerted me to incoming strikes, showing me how and where to parry his attacks. So by holding the lightsaber in my hand to match a floating diagonal line, I could parry his attack. After several parries, the Force showed an attack symbol on Maul, letting me know I could unleash on him. I didn’t have to wait to attack, but attacking once he was sort of stunned, did much more damage. I also had the ability to use a single force power (I selected one that slowed time down a bit when the match started) by pressing a button on the lightsaber. In a different encounter, I used my saber to block incoming fire from robots, either taking them out with their own shots, or knocking off limbs and head if they got to close.

While it was a fun, physical experience, I found that the lightsaber tended to get a little out of alignment over time and that the game seemed to jitter a bit as I played. Both are issues that Goslin says the team is working to fine-tune before launch.

Holochess brings up the familiar holochess table first seen in the Millennium Falcon in Star Wars: A New Hope. Once the table and the pieces appear, you simple use the lightsaber to select which piece to move and play the game following the pretty basic instructions. It's a neat bit of fan service, but not the sort of game that will occupy much of a player's time.

I was surprised to find myself most interested in the game’s strategic combat mode, which has you controlling action-figure-sized characters on the floor or table in a real-time strategy game. In this mode, you're essentially acting as the general, marshaling the forces of the Republic, Rebel Alliance and Resistance against the Empire, Separatists and First Order in mini-battles. It's amazing to kneel down on the floor and what an entire strategic map sort of float up out of the carpet. Once the game got going, I was placing turrets, dropping in rally points, issuing fire commands and even bringing in a hero. The whole thing looks like a holographic version of playing with your old action figures, but if they moved on their own and could blow stuff up. It was incredible.

All said, Goslin says it takes the quality assurance team about 12 hours on average to play through all of the content once.

AR for Doctor Strange, Iron Man, Frozen?
The team is already working on new content for the game, though they're not sure yet if it will be given away as free add-ons or sold.

Top of the list is support for multiplayer, so if you and a friend both have the system, you can play holochess together, lay on the floor and battle it out with holographic action figures and, yes, fight one another with lightsabers you can see and feel in your hands..

They're also looking into releasing more physical lightsabers. I was really impressed with the attention to detail in the one included with the game. Goslin says that’s a common reaction and they hope to be able to deliver more lightsabers, perhaps packed in with original content, for fans once the systems are out.

And it's not just Star Wars the team is working on. Goslin says Disney views this as an entire new category of play.

They started with Star Wars because, he says, it's the easiest to understand for those not accustom to augmented reality. They sell you a lightsaber, a headset and the fantasy that you're being trained to be a Jedi. Once you wrap your head around how augmented reality works and what it can deliver, in theory you’re more open to other experiences in augmented reality too.

"Obviously there's lots of stuff we can do with AR," Goslin says. "But we didn't want to confuse the message so we led with Star Wars."

The new category, in Disney's eyes, is called AR for entertainment. "We are going to learn a lot about the category from this launch," he says. "Is there demand for more premium features? Do we need a lot more content? Does it need to be more social or need more multiplayer?"

The team is already exploring something Goslin calls "two-handed play." What that means is that you don't necessarily have anything in your hands, but maybe you're wearing something on them. He was deliberately vague about what that meant, but he mentioned the work after I asked him about this tech coming to Marvel and its many franchises. Imagine, he says, what we could do with a game that has you playing as Doctor Strange, using your hands to summon spells.

It sounds like, in that scenario, one might wear special gloves, though Goslin declined to tell me if that was accurate when I asked.

"It would be a different kind of AR entertainment," he says. "Something that is wearable, but that has open-handed play."

He also mentions in passing the possibilities of building experiences around properties like Iron Man or Disney's movie Frozen. And Disney is very behind the concept of AR entertainment.

"People at the highest levels of the company believe in this."