Why Square Enix is Turning Manga Into Immersive VR Comics

The first chapter of 'Project Hikari' hits early next year

Square Enix hopes to release its experimental blend of traditional Japanese manga and virtual reality to the public early next year, the project lead on the title announced during New York Comic-Con tonight.

In Project Hikari, players strap on a virtual reality headset and are dropped in front of the panels of the manga title Tale of Wedding Rings. The panels then come to life, sometimes moving to swallow up a viewer, pushing them inside the panel, sometimes simply turning the panel into an animated scene from the traditionally static manga.

No price has yet been set, but that first release will likely be the first of four chapters found in the original manga’s initial volume.

The team, which works at Square Enix’s Advanced Technology Division in Tokyo, said they started work on the concept back in 2013 after getting a chance to check out the Oculus Rift development kit headset for the firs time.

“We thought that it was going to change things,” Kaei Sou, project lead, told an audience during a New York Comic-Con panel on the experience. “So we talked to our CEO. We wanted to do something different, something story focused, something unique to a company like our’s. So we talked about it and he gave us the OK to do the prototype.”

The team settled on using Tale of Wedding Rings both because they liked it and the original artists were very open to Square Enix experimenting with it using emerging technology.

“We want to let the user to be able to step inside the manga stories,” Sou said, after showing the audience a concept video, which showed two people reading a manga only to have it slowly take over the world and allow them entry into the story they were reading. “We believe that with a lot of the advancement with modern technology we can achieve a lot of the things you can see in the video.”

Sou said that a while the sorts of things that have been inspired by Japanese manga, like movies and video games, continue to advance in the way they portray the source material, those original manga haven’t really changed at all.

“This art form has been around a long time,” he said.

So the team set about trying to take the flat, static, black and white art of traditional manga and turn it into something that was immersive and welcoming to readers.

The team took that initial effort to the Tokyo Game Show in 2016.

“We got a lot of positive impressions,” Sou said. “We got interviewed by TV stations in Japan. It was really positive feedback.”

But the team wasn’t happy with their efforts yet, so they went to work making it better.

Take Two
“Manga has a very distinct look,” Sou said. “We wanted to make sure we were true to the fans. If we don’t make it look like the manga everyone is going to be disappointed. The visual aspect is something we really focused on.”

The team rebuilt the manga using the Unreal game engine, essentially recreated the flat, hand-drawn panels in three dimensions.

“When we first created the characters in the engine it looked like this,” said Setbon Elie, a character artist on the title, showing an image of what looked almost like a 3D grey sculpture of a character.

The slowly stripped almost everything away from that original creation and then slowly layered back in detail.

Where the TGS build of the experience worked, the team felt it didn’t really deliver on the artistry of the original.

So they worked to recapture the sense of hand drawing, the lines that weren’t quite straight, the subtle shading., Hisami Miyamoto, the concept’s environmental artist, said. They also went back and added hatching, a distinct sort of shading used in most manga.

As they worked to perfect the look of the artwork in virtual reality, they also had to tackle a slew of other, new problems.

Matching something up to look exactly like it does in a panel didn’t always work because often the person experiencing the manga can now walk around it and view it from all angles. Sometimes what looks fine from a locked, specific angle, looks absolutely wrong once the camera controls are handed over to the viewer.

The team also had to create whole environments that were never seen inside the manga because of tight framing.

Miyamoto estimated that 70 percent of what a person will experience in Project Hikari wasn’t actually present in the manga, it had to be created by the developers.

The team continues their work finalizing it, but everyone seems happy with how the project is coming along, even those who didn’t think virtual reality was a good match for manga.

“I wasn’t convinced at first, to be honest,” said Elie. “But then I got to try it and realized it was really something else and that it delivered an experience you couldn’t achieve with anything else.

“You get to experience something you really like from the inside. Being able to watch the story from every angle is really cool.”

Manga VR
While the team were coy about where they might take this technology once Project Hikari is completed, it seems likely that what the team is building isn’t simply a single manga in virtual reality, but a system in which to create a slew of such creations.

Project lead Sou said he’d love to experiment with using the system to bring a horror manga into virtual reality.

“A lot of the early results of what we got were just perfect for horror,” he said.

The team also have thought about colorizing manga in the experience using special techniques, like using water color. Character artist Issei Koda said he’d like to have movable speech bubbles and add more detailed animations, such as sweat, to the creations.

“My goal,” said environmental artist Miyamoto, “is to take an environment that if you took a snapshot of it, it could stand on its own as a piece of artwork.”

Sou said that the experience would release first on HTC Vive and Oculus Rift, but couldn’t give an exact release date.

“We are working on bringing a full version early next year,” he said. “As quickly as possible.”