Future of the NFL: The Virtual, Augmented, 3D, 360-Degree Football Experience

Tom Brady is standing on the pool table in your man cave.

Brady looks real enough to reach out and touch as he waits for the snap behind the Patriots line. His receivers hover over the corner pockets. Rob Gronkowski is standing on your pool table as well, though that's somehow less surprising.

In front of you is a live three-dimensional rendering of Gillette Stadium during a playoff game. Or maybe you are watching on your dining room table or your back porch deck. Heck, maybe you want to bring Brady with you into the bathroom instead of waiting for a break in the action. Nothing creepy about that. Just reach out and grab the whole stadium, shrink it, carry it with you, spread it out again in the tub.

While you are at it, spin your hands to change the camera angle or use a finger to call up a fantasy stat update in the vanity mirror.

No, you have not become Tony Stark. You are just a football fan of the future. Not the distant future either: a near future when Brady might still be leading the Patriots to playoff games.

You will be equipped with the latest virtual and augmented reality glasses. The NFL's broadcast partners will live-stream 360 degree high-definition video and/or fully modeled digitized renderings of the action into your home. Whether you watch the game from the virtual 50-yard line, hover behind Brady's shoulders or sit like an Olympian god over a chessboard of mere mortals is up to you.

And it will happen sooner than you think.

The Virtual Ecosystem

Max Cohen is not just a vice president of mobile technologies at Oculus. He is also the commissioner of a fantasy football league. And fantasy football is really just a primitive form of virtual reality. It's an imaginary experience that can feel real when the rookie running back you discovered wins a game for you with a Monday night touchdown.

"It creates all of these emotional connections to the league," Cohen said of fantasy football. "And that's the best thing that VR can offer: that emotional connection you don't get when you are just watching a TV screen."

Oculus is best known for the Rift headset, a popular device often associated with early-adopting techies or twitchy tweens gulping down energy drinks for all-night video game marathons. It's the kind of tech older consumers might think of as a generation away, or go the way of 3DTV or laserdisc players in a few years.

But virtual reality is about much more than goofy goggles for a niche marketplace. According to Cohen, Oculus is not really in the headset, hardware, software or first-person-shooter business.

"We're in the ecosystem business," he said.

That ecosystem is a web of hardware providers, software developers, streaming platforms, camera manufacturers, production companies and everything else consumers need to experience virtual reality without breaking the bank or rewiring their homes.

Like any good ecosystem, there are symbiotic relationships among the organisms. Oculus weaves through the environment, cooperating with companies which superficially appear to be rivals.

"If you lower the friction of developing VR content, that brings users in, that helps the companies be successful," Cohen said.

With companies working together, it's no wonder that the ecosystem is also evolving rapidly.

"What I might have told you was 10 years away last year is now three years away," Cohen said.

"Imagine a VR fantasy app which has tags for all your players and cameras in every end zone," Cohen said. "You can put on your headset and see a touchdown as if you are standing next to a goal post."

Sounds great. Is that something we might see in 2026?

"That's something we could actually do today."

Oculus now has over one million monthly users, but an upcoming industry explosion could make that number seem puny. The Samsung Gear—powered by Oculus tech and retailing at around $99—is putting headsets over more eyeballs every day. Sony's new PlayStation VR headset ($399) has the potential to put virtual reality in the livings rooms in as many as 45 million PS4 users. Other companies, from Microsoft to (probably) Apple, are in the process of leaping into the market.

It's all happening so fast that Cohen cautioned me to think in terms of months, not years, when projecting the market.

"I don't think it makes sense yet to offer, say, an NFL Sunday Ticket VR package to customers. This year," he said.

So...virtual reality NFL broadcasts could make sense next year?

"Some people will dip their toes into things this year," Cohen speculated. "And then it will double, and double again and double again."

Sounds like a tiny organism in an ecosystem evolving into a dinosaur, and beyond.

"Imagine you could pay 20 dollars for a seat the 50-yard line," Cohen asked. "That's not five to ten years away. It's sooner than that."

In a way, it's already happening.

Craving the Experience

I am standing on the field at the Patriots practice facility during a minicamp. At least it looks, sounds and feels like that's where I am. When I turn my head to the left I see three wide receivers. A fourth is to my right. When I turn completely around I see Brady drop and scan the field; I hear his voice in whichever ear is closest to him.

Thirty seconds ago I was in a Philadelphia hotel lobby with Chris Little, head of partnerships for STRIVR, a virtual reality content provider. Then he slipped goggles over my head. A Patriots logo and a lozenge-shaped START button appeared in front of me. I reached forward instinctively to click it.

My hands did not appear before my eyes. Oops, I guess I am not Tony Stark just yet.

"You did what everyone tries to do," Little explained.

The real start button is located near my right temple on the goggles. With a click, I enter a scene out of The Matrix, only with less Keanu Reeves and more Gronk. The Patriots tight end welcomes me to the virtual reality demonstration. Seconds later, I am experiencing football from angles I have never seen before.

The images and sounds are fully three-dimensional, with 360 degree range. Not only can I see Brady and Gronk running routes and blockers fending off defenders, but coaches milling around and trees swaying in the distance. Whirling about the Sheraton lobby with a gizmo over my eyes and pointing at illusory football players while Little patiently explains the STRIVR technology, I must look like an imbecile. But at virtual Patriots camp, I feel like I could clean the sod from Brady's cleats.

Little describes himself as "realistically focused and practical," so he doesn't sound worried that the loopy reporter spinning about and pointing to virtual Patriots will announce that fully immersive VR NFL broadcasts will be available by next Sunday. He's quick to point out all the things the STRIVR system can't do yet. I cannot walk around my virtual Patriots environment, for example. Everything I see was pre-recorded weeks earlier. And the image quality isn't perfect.

"Camera technology is not strong enough yet," he said. "Outside of about 20 yards, it's a really diminished view."

It's true. The receivers look a little grainy when they are about 25 yards away. But they really look like they are 25 yards away. My brain thinks it could throw passes to them.

STRIVR creates 3D virtual training simulators for NFL and college teams, as well as for other sports. Little shared some of the research behind the VR training advantage.

"Human beings retain 20 percent of what they read or hear, 40 percent of what they observe, and 90 percent of what they experience," he explained. "That's why experiential VR is so powerful."

The numbers sounded a little like slick marketing pseudo-statistics until I donned the headset and watched some defenders approach me. I have watched blitzes from a hundred different angles, but with the headset on, I felt that I was being blitzed. My brain didn't want to take my eyes off the aggressive men coming straight at me, which is exactly what a young quarterback must be trained to do.

It wasn't much of a leap for STRIVR to go from training simulations to virtual reality experiences for fans. After my trip to Patriots camp, Little took me behind the goal at a USA Soccer match and behind the plate at Coors Field. I stood in the University of Arkansas football locker room and on the sideline as the band marched, looking real enough that I thought I could grab a trombone.

"Those sorts of experiences, in an immersive nature, are what fans crave," Little said.

STRIVR creates fan experiences that allow visitors to Madison Square Garden to play goalie for the virtual New York Rangers. STRIVR technology lets Carson Palmer take extra reps against a "live defense" without needing live defenders. So what's standing between us and weekly Sunday NFL virtual reality broadcasts?

"VR is one of those things that everyone wants to talk about, but no one is really writing checks yet," Little warned.

"To create titles and content, there has to be an audience," he continued. "There has to be an opportunity to make money, right? That hasn't quite come together yet. So the overall experiences at a volume level are pretty low, because we are waiting for the markets to develop."

The leagues and broadcasters will be ready to blast us with games, highlights and other goodies the moment they do.

The Future is Last Friday

There's a dirty little secret about virtual reality, and Michael Davies, senior vice president of field and technical operations at Fox Sports, knows it.

"If you talk to anybody who has never put a headset on they will say, 'virtual reality is great because it gives me the best seat in the house,'" Davis said. "Well, sometimes the best seat of the house in virtual reality kinda sucks."

Little offered a similar caution:

"Everyone thinks that VR is like chocolate sauce. You just put it on top of something and it makes it better. That's really not the case."

Virtual reality could end up a lot like the current state of 3D movies. One week you are thrilled by James Cameron's Avatar. The next you are plunking down an extra $20 to watch Glorified Toy Commercial 2 with the kids in goofy glasses. Worse still: It could end up like 3DTV, abandoned on the roadside by consumers, despite the promise of NFL players seemingly leaping from your television screen.

No one in the ecosystem wants virtual reality to become some costly gimmick, which is why Davies and the Fox Sports Lab are already working to provide high-quality content before the average home consumer is even ready to receive it.

When I interviewed Davies in late summer, Fox was planning for a major live virtual reality broadcast of the Bundesliga Soccer League opener the following week. All a user needed was a top-tier Samsung smartphone and headset and a downloadable app. Then, presto.

"It streams the experience to your headset, and you will be there in Munich," Davies said.

In other words, the future of virtual reality was literally next Friday. Which means it has already passed.

Fox's production partner in the VR ecosystem is NextVR, a company that not only produces content but supplies it through a downloadable app. In addition to Fox, NextVR has partnerships with NBC, Live Nation and Bleacher Report parent company Time Warner, among other broadcasting giants. When the NFL dabbled in VR demonstrations at Super Bowl 50, they were using NextVR's tech.

NextVR supplies Fox with the cameras and production equipment. The cameras are small, meaning they can overlay their VR broadcast over any television broadcast without crowding the sidelines with gear, a major stumbling block for 3D sports broadcasts of the past.

"It's the size of a breadbox," said Brad Allen, executive chairman of NextVR of his company's stereoscopic cameras. "It's small enough that it sits on the scorer's table during an NBA game."

The cameras also don't require an operator. After all, you decide where to look, not a cameraperson.

The NextVR technology is so new that just last January they were still introducing the proof of concept. By August, they were broadcasting soccer as well as dozens of other live events, from Big East basketball to live concerts.

Still, the audience size doesn't quite merit the effort of live broadcasts just yet.

"One of the big reasons that you do VR right now is to get the word out that you are doing VR," Allen said, calling it "VR for PR."

But producers are also working out the technical and not-so-technical kinks. According to Davies, early VR broadcasts from a stationary point in the arena, as great as that point may have been, "kind of left you in the woods a little bit."

Davies added: "We learned that you gotta give the viewer more. There's value in production. There's value in multiple cameras. There's a place for augmented reality."

The Augmented World

For those of you who don't speak techie, augmented reality sounds like something that could fracture the fabric of space-time. But it's actually both common and relatively simple.

Virtual reality puts you inside the stadium. Augmented reality puts the stadium (or the players) into the real world around you. In Little's STRIVR demonstration, for example, a glowing ball with a "heat trail" directed my eyes to long passes, home runs and shots on goal, making sure I wasn't looking at clouds instead of the action (unless I really wanted to).

If glowing balls and heat trails sound familiar, it's because augmented reality got its start when Fox made the puck glow during NHL broadcasts in the 1990s. The glowing puck was ridiculed by just about everyone, but it started an augmented reality revolution.

"The glowing puck was the foundation for the technology that made the yellow [first-down marker]," Davies said. "That technology was the basis for the pointers in NASCAR and the range-finder in golf. There's a lineage for this technology that begot the more modern ones."

Augmented reality has the potential to make the best seat in the virtual house far, far superior to the best seat in the actual house. You can "sit" at midfield with highlights of other games on the "Jumbo Screen," fantasy stats hovering to your left and your social network feed to your right. Mike Pereira might pop into view in the seat in front of you during a replay review.

You don't have to be surrounded by pixelated strangers, either. Right now, VR users are isolated inside their headsets, but augmented reality can change that. "If I want to watch something with my wife, the TV screen is easier...today," said Cohen.

"But I don't think that will always be the case. I do think there will be a time where I can be in a hotel in London, she could be in our house in California, and we can get the sense that we are sitting on the couch together.

"We're putting together the groundwork, the foundation, so that you can experience anything, anywhere with anyone, and there is no barrier of distance."

In the short term, the experts envision what Davies calls a "dip-in, dip-out" virtual reality experience: You may watch the game on your HD television, then switch to the headset to experience an Odell Beckham catch, a postgame highlight reel, or the Super Bowl halftime show. As headsets grow more plentiful and comfortable, image resolutions grow crisper and producers master the logistics of live broadcasts, you and your buddies will be able to high-five each other along the virtual 50-yard line.

But why stop there? If the reality is truly augmented, why stop at the best seat in the house, or whichever seat a producer chooses for you? Why not be able to move from midfield to the end zone to the blimp to inside the quarterback's helmet with a wave of the hands?

"There's an expectation that virtual reality has to be interactive," Davies said. "You have to be able to do something."

According to Davies, that level of interaction is on the horizon.

"I think we'll be able to capture so much data at so much resolution that you'll be able to synthesize views that you could never get," he said

In other words, you can go places where there are no cameras and see vantage points no camera could ever capture.

"You would be able to see what the quarterback really saw," Davies said.

"We've seen technologies where that is starting to percolate," he added. "But that's waxing pretty futuristic."

In the virtual ecosystem, "futuristic" means that it's already about to happen.

The Box of Possibility

One of those places trying to take VR into the future is EON Sports. The company can do most of the things that the other VR providers do, creating virtual experiences at the local stadium or streaming highlights and inside-the-huddle videos to fans, but Brendan Reilly, CEO of EON Sports, brushes past these now semi-common applications.

"Throwing a pass to your favorite receiver is just the tip of the iceberg," he said. "There are way cooler things that we can design and create."

Long term, EON is in the business of doing precisely what Davies described: taking the real world, digitizing it completely, and blasting all of that information to your smartphone in real time so you can do anything you want with the images.

If the entire three-dimensional stadium environment, moving players and all, is turned into digital data like a Madden video game, then (like in Madden) you can place the camera in places where no cameras exist, like inside Tom Brady's eye sockets. Except that the game is real, and live, and the images look 100 percent life-like, not like Madden's (certainly impressive) video game avatars.

Reilly is based in Kansas City, so you'll have to forgive him undercutting the awe-factor of his technology by using the Chiefs as an example.

"You can have Alex Smith stand in your living room, and get play-by-play stats as they happen when he's standing right there interacting with you," he said.

"You can experience what it's like to be the left tackle. Or the field-goal kicker. Or the quarterback."

This augmented reality experience isn't really designed for a VR headset, which isolates the user from the outside world. It's made for something more streamlined, like a pair of glasses that can project three-dimensional images onto the real world. So you can see the game from any angle and see your friends, stick the stadium on your pool table, or stand next to Alex Smith...or some other quarterback. Glasses that can do all of that are coming.

"We're in the brick cellphone stage of VR hardware," Allen of NextVR said. "It's a given that in a couple of years, they are going to look like a pair of Oakley glasses."

In fact, Reilly, Cohen and Allen all told me that contact lenses are coming. The game will be beamed straight to your eyeballs, overlaying the world around you.

So when can you expect to ditch the headset?

"That's not 10 years away," Reilly said of everything but the contact lenses. "That's more like three years away."

There goes the virtual reality industry smashing through its own timeline again.

The Revolution Won't Be Televised

If you are waiting for the catch, the technical, legal or financial hurdle standing between you and 360-degree, fully-immersive virtual NFL content, keep waiting.

Fox has the rights. NextVR has the apps, cameras and trucks. STRIVR has the storytelling know-how. EON has the vision. Oculus provides the whole ecosystem. Everyone from Samsung to Google has an inexpensive gizmo for you to click over your smartphone and slide over your forehead. And everyone plays together nicely, so you're unlikely to get stuck with the obsolete, incompatible brand. And the NFL has already dipped several toes in the water, from team-based experiences like my virtual trip to Patriots practice to demos during the Super Bowl.

Every month, the resolution gets crisper, the bandwidth requirements less intense, the headsets lighter, the production more sophisticated, the cost lower. It's not going the way of 3D television. It's more likely to make your HDTV look as quaint as an evening newspaper.

"TV is dead," Reilly said. "Not in the story. Not in the content. But the actual structure itself is going to be gone."

"You can have a headset on, and if you want a 96-inch TV, you just move your hands and spread it out."

The industry is just waiting for us—several million of us—to get out our credit cards. Once the user total reaches a tipping point, major profitability will follow. Then, it will all be as easy and commonplace as streaming Netflix or Hulu is now. Sponsors will pay big bucks to fill your virtual stadium with augmented advertisements, probably ones tailored exactly to your spending habits.

Perhaps the only catch is that we are talking about 2018 or 2019, not 2026, which is a future too far-flung for even virtual reality experts to imagine.

"Ten years is too far," said Little of STRIVR. "Once you reach the point in technology when money starts to arrive, then the ramp changes."

Piles of money are starting to appear beneath that ramp's fulcrum.

"When you start talking 10 years, it's like: holy smokes," said EON's Riley. "We're going to be able to teleport by then."

"Virtual reality is not where it's gonna be in six months, but you don't get into this business if you don't like change," said Davies a few months ago, meaning the industry is in a totally different place now.

"Who knows?" Davies chuckled. "Maybe we will come back to putting a glowing puck into the VR. It can all come full circle."

The return of the glowing hockey puck? Oh Brave New World that hopefully won't have that particular wondrous thing in it.

Mike Tanier covers the NFL for Bleacher Report. Follow him on Twitter @MikeTanier.