Scientists Tell Us How Your Old Pics Will Change Time-lapse Photography

No time for time-lapse? New research just might be able to change that.
timelapseinline
University of Washington; Google Inc.

Google and the University of Washington are breathing new life into old photographs. That’s 86 million old photographs, to be precise, shuffled and blended to create an array of stunning time-lapse videos that give new insights into how landmarks, monuments, and more have evolved.

Traditional time-lapse sequences require painstaking---or at least patient---work. Their most common subjects are familiar by now; the aurora borealis, clouds or stars (or both) traversing the sky, the stops and starts of city traffic. You anchor your camera, and you wait. And wait. And wait. And after you’ve waited long enough to gather a critical mass of footage, you compress and compile it all into a bite-sized, super-speed version of that particular corner of the word.

The process has gotten easier recently, thanks largely to smartphone software. iOS 8 has a time-lapse feature built right into the camera app, and rival Hyperlapse projects from Instagram and Microsoft smooth out and shorten even shaky mobile moving footage. The accessibility of these tools, though, still doesn’t solve one of the biggest problems facing time-lapse: It requires time, and lots of it.

It was the tedious process that inspired Google and the University of Washington. "I always loved time-lapse photography, but never thought about making them myself due to all the logistical challenges," explains Ricardo Martin, who worked alongside researchers David Gallup and Steven Seitz. "As our tools for working with internet imagery got more and more sophisticated, we found that lo and behold, it might be possible to create time-lapses from photos that people have already taken."

It is possible---if you’re able to sift through tens of millions of photographs, sorted by timestamp and geolocation, to discover the time-lapses that are already hiding there. It’s like going through mountains of paper shreddings and taping them back together into novellas.

The team calls the process "time-lapse mining," and points to just how much more efficient the pseudo-crowdsourced results can be than the time-worn, traditional method.

"Whereas before it took months or years to create one such time-lapse, we can now almost instantly create thousands of time-lapses covering the most popular places on Earth," the three wrote in a recently published paper on the project.

Getting to this point was not at all easy. What sounds like a fairly straightforward two-step process (1. find photos; 2. stitch them together) actually poses fairly intense computational challenges every step of the way. Those tens of millions of photos add up to hundreds of terabytes to crunch through, says Martin. And making those photos line up to look like they were shot from a single vantage point takes a significant amount of digital trickery.

"Synthesizing a photorealistic time-lapse is also very challenging," explains Martin, "as the input photos look very different from each other … Photo timestamps might be wrong and many photos contain occluders, like people posing in front of monuments, that our time-lapse successfully ignores."

Reference image and computed depthmap for Briskdalsbreen Glacier and Goldman Sachs Tower scenes. Warmer colors represent
pixels closer to the camera.


UNIVERSITY OF WASHINGTON; GOOGLE INC.

It’s a complicated business, but also fully automated: Just feed the algorithm, and in a few hours it’ll spit a video right back for you. Their efforts yielded nearly 11,000 time-lapses, most of which seem to be cover temporal stretches of five and 10 years. Among the highlights are the erosion of the Briksdalsbreen Glacier in Norway, the rise of New York City’s Goldman Sachs Tower, and a Swiss Guard at the Vatican who remains still enough over six years that he becomes every bit as much a part of the time-lapse as the iron door frame around him.

In a random sample of 500 time-lapses, the researchers found nearly half to be "good and interesting," which is to say they have no visible artifacts, are photorealistic, and actually show something changing over time; Martin points out that indoor scenes will often simply look like still photographs, since the changes even over a span of years can be too minute to notice.

The team hopes to be able to automatically filter out the boring results. In the meantime, some of those "failures" are captivating in their own right. The position of the famous Wall Street Bull has actually moved slightly over the years, creating a blur effect and a quick reexamination of your assumptions about statues and mobility. And while the researchers cast a four-year look at Hong Kong’s Wan Chai skyline as unsuccessful because it mixed day and night photographs, the pulsing neon result infuses the city with more life than a daytime or nighttime-only vantage would.

Map of the location of discovered time-lapses. Europe
contains the highest density of time-lapses, while few exist in Africa
and South America, as there are fewer photos available.


University of Washington; Google Inc.

The most exciting part of the project is how much potential it has going forward. Since it relies on timestamped online photos, most of the time-lapses it produces don’t go back much more than 10 years, if that. Increasingly, though, our photographs end up not in dusty, plastic-lined albums but on Flickr and other digital repositories.

What could that mean for the future? Not just the ability to capture time-lapses of locales that wouldn’t be practical with a static camera, but productive scientific research and discovery. Or for us civilians, a more thorough, more educational, more tangible reckoning of how our favorite places have changed in our lifetimes.

"Think of the signs that show how a viewpoint used to look many years back," says Martin. "In the future, we won’t have to show how it looked back then, by using the available photos in the internet, we will show videos of how the place transformed from what it looked back then to what it looks now."

The best part, of course, is that the very nature of this technique means that new time-lapses are being constructed every day. Somewhere in your next batch of vacation photos, one image could someday become a single stitch in a much larger tapestry.