Pokémon Go May Not Truly Be Augmented Reality, and That’s OK
Two weeks ago, the app maker Niantic, a spin-off of Google, released Pokémon Go — and now there are monsters everywhere. After downloading the game on their phones, players can traverse the real world hunting for creatures, the little “pocket monsters” that originally took the world by storm in the late 1990s. Once a Pokémon is found, users simply hold up their phones to see it overlaid in their real-world environment. They can use Poké balls, also collected virtually at real locations, to catch the creature and continue the hunt.
With 15 million downloads, the app is more popular than Tinder, and has quickly made “augmented reality,” or AR, a household term.
Coined by a Boeing researcher, Tom Caudell, more than two decades ago, the term is used to describe computer graphics, sounds, and other information used to change some aspect of the world around us in real time. In contrast with virtual reality, which creates a whole new environment around the user, augmented reality changes and builds on what the user senses in the real world. But whether or not Pokémon Go fits the bill depends on whom you ask.
“The question of whether it is AR certainly depends on what definition and criteria of AR you adopt,” Tony Liao, an assistant professor of communication at the University of Cincinnati who studies augmented reality, wrote in an email. “[B]ut the game does show digital graphics on top of real physical space, so at the very least it is mixed reality.” And he added that the app could become so associated with AR that it “becomes the de facto definition.”
But another expert, Ronald Azuma, currently the principal engineer and research manager at Intel Labs, has a much stricter definition of augmented reality. In 1997, he wrote that a true AR application must have three characteristics: It must combine the real and the virtual, it must be interactive in real time, and it must be in 3D. Azuma said in an email that he was unavailable for interviews.
These three points can seem simple enough, but they are hugely difficult to apply. For instance, in Pokémon Go, the characters appear as 2D images overlaid onto the camera-view behind them, but not really interacting with their real-world surroundings. To do that — to have the Pokémon hide behind an actual tree, say, or stay steady on a flat surface — the computer has to understand what the real objects are and how to interact with them. That’s something our brains can do easily; computers, not so much.
“You can imagine that if I change the lighting in the room, our eyes and our brains adapt and we can still tell what everything is,” said Blair MacIntyre, an associate professor at Georgia Tech and the director of the Augmented Environments Lab there. “But from the computer’s point of view, all the colors have changed. The image is completely changed. Shadows fall on objects — it is very hard for computers to separate that stuff out.”
Still, MacIntyre said that, like Liao, he would probably consider Pokémon Go at least a type of augmented reality.
“Some of my colleagues would roll their eyes, but I would say this is AR,” he said. “It is just a really weak form of it.”
The game also uses the camera only when users actually locate and click on a Pokémon, a feature that can be turned off so the characters simply appear against a background generated by the game. Stronger forms of AR are out there, however. Headsets from companies like Microsoft and Meta make for a more immersive experience. In a few years, MacIntyre said, such technology may come to be used for many things, like providing directions, allowing users to repair machinery without training by overlaying instructions directly onto the equipment, or even helping with remote surgery.
So Pokémon Go may not be true AR, but Shira Chess, an assistant professor of entertainment and media studies at the University of Georgia, says the game does give us a glimpse of what our future with AR might look like.
“We are hitting a point where we’re inseparable from our phones, she said. The little screen has become “part of our bodies, and it is part of how we respond to the world.”
While walking around campus playing Pokémon Go not long ago, Chess said she “heard a bird tweeting, and my first thought was ‘oh wait, where’s that?’ and then I realized that that was not a Spearow, that was actually a bird.”