These Industrial Robots Get More Adept With Every Task

Vicarious, a secretive 10-year-old startup backed by Mark Zuckerberg, Elon Musk, and Jeff Bezos, reveals its progress and an initial customer.
robot sorting products
Vicarious' AI software can neatly pluck items from a jumbled collection like this box of lip balm.Photograph: Phuc Pham

At the offices of startup Vicarious in Union City, where the San Francisco Bay Area’s sprawl abuts rolling hills, 10 robot arms tirelessly place travel-sized beauty products into bins on a conveyor belt. Each gray arm ends in a suction-cup-tipped finger that makes a high-pitched whine as it plucks items such as antiperspirant or hand lotion from crowded boxes.

Vicarious buys standard industrial robots, enhances them with its software, and contracts them out the way a temp agency does workers—charging per task completed or at an hourly rate. In Baltimore, Vicarious robots assemble sampler packs for makeup company Sephora, work previously done exclusively by humans. Vicarious CEO and cofounder D. Scott Phoenix says the deal demonstrates his business model: Create artificial intelligence software that makes industrial robots smart enough to perform jobs previously done only by people.

Vicarious hasn’t previously discussed its customers or robots publicly but has earned itself an air of mystery among AI and robot experts since its founding in 2010. The startup has raised more than $130 million, according to data service PitchBook. Its investors include some of Silicon Valley’s most famous names and deepest pockets—venture firm Founders Fund, cofounded by early Facebook investor Peter Thiel, and billionaire entrepreneurs Mark Zuckerberg, Elon Musk, and Jeff Bezos.

Instead of placing these items into boxes the robots throw them with a firm flick to extend their range.

Photograph: Phuc Pham

The startup is pursuing its own path in artificial intelligence, looking beyond the technology driving high-profile projects such as content moderation at Facebook and automated driving at Tesla. Phoenix says only a fresh approach to AI can resolve what he calls a paradox of modern society. Robot arms and grippers have been around for a long time, and components such as motors, sensors, and microcontrollers have never been so cheap or capable. But even inside factories and warehouses, robots are restricted to certain tightly controlled tasks because their software must be specifically programmed for every situation and can’t adapt to unexpected variability.

“We're paying people trillions of dollars a year to do stuff that robots have been physically capable of doing for the last 30 or 40 years,” Phoenix says. Anyone who can make industrial robots more adept—and Vicarious is not the only one trying—could transform the economy by shifting the balance of labor between people and machines.

Deep Learning and Its Limits

When you hear a CEO or politician talk of the growing power of artificial intelligence, they are generally referring, even if they don’t know it, to a technique called deep learning. Since 2012, when researchers showed it could make computers much better at interpreting images and text, the technique has rewired the technology industry. Deep learning powers face-swapping photo filters and self-driving cars; it is why Alphabet CEO Sundar Pichai opined at Davos this year that AI is “more profound than fire or electricity.”

Vicarious uses deep learning for some things, such as in its robots’ vision systems, but believes other ideas are needed to make computers truly smart. Phoenix started the company in 2010, before the deep learning era, convinced that infusing AI into robots could transform the economy. His cofounder was Dileep George, a software engineer turned researcher who had recently completed a PhD thesis at Stanford titled “How the Brain Might Work.” It used observations from neuroscience to guide the design of AI algorithms. Since then, deep learning has swept through Silicon Valley, and Vicarious has published a series of papers highlighting its limitations and advocating an alternative approach.

Deep learning software makes sense of data like images or audio by looking for statistical patterns it has extracted from past data. Apple’s Photos app can automatically create an album of your pets because it has deep learning algorithms trained on thousands or millions of labeled images of cats and dogs. One way to make a robot grasp objects is to program it to try different approaches and use deep learning on its successes and failures to determine a good claw hold.

This kind of statistical pattern matching has found many, profitable, uses. But George points out that it doesn’t let computers reason about the world, intuit the cause of events, or handle situations outside their past experience. “Just scaling up deep learning is not going to solve those fundamental limitations,” George says. “We’ve made a conscious decision to find and tackle those problems.” Vinod Khosla, the billionaire investor whose firm Khosla Ventures has invested $25 million into Vicarious, says he had trouble finding AI experts to help vet the company as a potential investment. "Everyone knows deep learning, but not this other stuff," Khosla says.

article image
Supersmart algorithms won't take all the jobs, But they are learning faster than ever, doing everything from medical diagnostics to serving up ads.

A provocative paper Vicarious presented in 2017 at a leading deep learning conference illustrates its approach to AI. The company designed experiments that exposed the inflexibility of deep learning software from Alphabet’s DeepMind research group that learned to play vintage Atari games such as Breakout better than top gamers. Vicarious showed how these superhuman AI players crumbled if a game was trivially altered, such as by increasing the brightness of colors or subtly changing the size of objects.

The startup’s own software could handle such changes because they did not affect its understanding of the mechanisms at work in the game. Though the software also learned from past data, it was primed to pick up the causal relationships between objects and events in the game and could use that knowledge to adapt to small changes it hadn’t previously experienced.

Brenden Lake, an assistant professor at NYU, says the paper demonstrated something the field of AI needs to figure out, as talk grows of deep learning hitting its limits. “A key part of human intelligence is building flexible models of the world that can be used in a variety of situations,” Lake says. “I think people are realizing you can’t get there with large-scale pattern recognition systems trained on large data sets for one specific task.”

A Flick, and a Miss

The robotic arm, taller than a person, stacking boxes in one corner of Vicarious’ cavernous factory looks like it’s playing a particularly boring videogame. Whirring and hissing, it picks up cubic boxes and stacks them into a neat grid on a wooden pallet, a common warehouse and factory operation called palletizing. Nearby, a line of robot arms sorts cosmetics into boxes with flair, using firm flicks of their suction fingers to throw items like tubes of lotion into boxes just beyond their reach.

Vicarious is not the only startup using AI to teach industrial robots new tricks. Many, including some featured in WIRED, rely heavily on deep learning. Alphabet recently unveiled a fleet of robots that rove around two of its offices collecting waste and sorting it into trash, recycling, and compostable items.

Phoenix says his robots are distinguished by their flexibility—born of algorithms like those that allowed his Atari bots to adapt to tweaks to a game. Robotic arms that stack pallets are usually paired with expensive feeders that position every incoming box or bin identically. Vicarious’ software is flexible enough to pick up boxes that aren’t perfectly positioned, Phoenix says, and can grab them from an ordinary table. It takes a reporter only about a minute using a touchscreen interface to reprogram the arm to palletize its boxes into a squiffy, blocky take on the WIRED logo.

Lila Snyder, who leads Pitney Bowes’ business handling ecommerce logistics for brands such as Bloomingdales, says she was struck by seeing Vicarious robots get visibly more able at laying out products into boxes in just hours when Pitney Bowes began working with the startup last fall. “I’ve seen robotic arms do work before, but it’s odd to watch one get better at a task,” she says. “Vicarious is allowing us to automate things we were unable to automate in the past.”

This large robotic arm can put boxes into a neat stack on a pallet ready to be transported elsewhere.

Stefanie Tellex, a robotics professor at Brown and cofounder of the startup Realtime Robotics, says Vicarious stands out from most other startups trying to breathe more intelligence into industrial robots. “Most companies don’t write cool academic papers and at the same time try to deliver things to clients, but Vicarious is trying to do both,” she says.

Phoenix says he’s investing in fundamental research because he wants to see his robots take on higher value work than packing boxes, such as complex assembly in manufacturing that currently requires people. That will require more progress on Vicarious’ take on AI.

As WIRED toured the startup’s warehouse, one robotic arm miscalculated, flicking a tube of lotion along a smooth arc that overshot its mark and ended on the scuffed concrete floor. “That was too hard,” Phoenix said, in a deadpan tone. The robots are coming—but there’s still work to do.


More Great WIRED Stories