Artificial intelligence is awakening the chip industry’s animal spirits

SUPERCOMPUTERS usually fill entire rooms. But the one on the fifth floor of an office building in the centre of Bristol fits in an average-sized drawer. Its 16 processors punch more than 1,600 teraflops, a measure of computer performance. This puts the machine among the world’s 100 fastest, at least when solving certain artificial-intelligence (AI) applications, such as recognising speech and images.

The computer’s processors, developed by Graphcore, a startup, are tangible proof that AI has made chipmaking exciting again. After decades of big firms such as America’s Intel and Britain’s ARM ruling the semiconductor industry, the insatiable demand for computing generated by AI has created an opening for newcomers. And it may even be big enough to allow some startups to establish themselves as big, independent firms.

New Street, a research firm, estimates that the market for AI chips could reach $30bn by 2022. That would exceed the $22bn of revenue that Intel is expected to earn this year from selling processors for server computers. It could swell further, argue the authors of a recent report by UBS, an investment bank. AI processors, they believe, will create their own demand; they allow firms to develop cleverer services and devices, which will collect even more data, generating a need for even brainier chips.

To understand what is going on it helps to make a short detour into zoology. Broadly speaking, the world of processors is populated with two kinds of animal, explains Andrew Feldman, chief executive of Cerebras, an American competitor to Graphcore. One sort of chip resembles hyenas: they are generalists designed to tackle all kinds of computing problems, much as the hyenas eat all kinds of prey. The other type is like cheetahs: they are specialists which do one thing very well, such as hunting a certain kind of gazelle.

For much of computing history, hyenas named “central processing units” (CPUs) have dominated the chip savannah. Becoming ever more powerful according to Moore’s law, the rule that the performance of processors doubles every 18 months, they were able to gobble up computing tasks, or “workloads”, in the jargon. This is largely why Intel, for instance, in the early 1990s became the world’s biggest chipmaker and stayed that way for decades.

But in recent years the world of number-crunching has changed radically. Moore’s law has started to peter out because making ever-denser chips has hit physical limits. More importantly, cloud computing has made it extremely cheap to amass huge amounts of data. Now more and more firms want to turn this asset into money with the help of AI, meaning distilling data to create offerings such as recognising faces, translating speech or predicting when machinery will break down.

Such trends have altered the chip-design habitat. First to benefit were “graphics processing units” (GPUs), a kind of hyena which are mainly made by Nvidia. Originally developed to speed up the graphics in video games, they are also good at digesting reams of data, which is a similar computational problem. But because they are insufficiently specialised, GPUs have been hitting the buffers, too. The demand for “compute”, as geeks call processing power, for the largest AI projects has been doubling every 3.5 months since 2012, according to OpenAI, a non-profit research organisation (see chart). “Hardware has become the bottleneck,” says Nigel Toon, the chief executive of Graphcore.

The response from various firms has been to design processors from the ground up with AI in mind. The result of Graphcore’s efforts is called an intelligent processing unit (IPU). This name is not just marketing: on GPUs, memory (the staging area for data) and brain (where they are processed) are kept separate—meaning that data constantly have to be ferried back and forth between the two areas, creating a bottleneck with data-heavy AI applications. To do away with it, Graphcore’s chips do not just have hundreds of mini-brains, but the memory is placed right next to it, minimising data traffic.

Graphcore’s chip can also hold entire neural networks, computational models inspired by structures in biological brains, which are used in many AI applications. Having such models, which can be immensely complex with billions of parameters, sit in the chip allows them to be “trained” more quickly—the act of feeding them with lots of data (pictures of cats, say), so they learn to recognise them. The set-up also simplifies what is known as “inference”, when the model applies what it has learned (spotting cats, for instance).

Cerebras is going further still. It is not only designing a new processor, which is similar to Graphcore’s, but a specialised AI computer as well. Putting a new chip on a circuit board, as Graphcore does, that is added into an existing system limits specialisation and optimisation because of constraints in power, cooling and communication, says Mr Feldman. But this means that he has a steeper hill to climb: while Graphcore has already delivered a first batch to customers, Cerebras has yet to announce when its product will be available.

Although Graphcore and Cerebras were early to see the need for specialised AI chips, they are by no means alone. Dozens of startups are creating what are known as “application-specific integrated circuits” (ASICs). These are meant to do inference in all kinds of connected devices, from smartphones to sensors, known as the “edge”. The processors come with trained AI models baked in, for instance to let a video camera recognise faces without having to upload the entire footage.

Big cloud-computing providers have also joined the fray, deeming AI chips important enough to develop their own. In May Google launched the third generation of its Tensor Processing Units (TPUs), the previous versions of which already power many of its services, including search and Street View. Amazon, Facebook and Microsoft, too, are developing processors. Apple, for its part, ships its iPhone X with an AI chip that helps the device recognise the owner and read his facial expressions.

Firms that ruled the world of hyenas, notably Intel, are now acquiring designers of cheetahs. It has spent billions in recent years buying AI-related startups, including Nervana Systems and Mobileye. The idea, says Gadi Singer, in charge of the firm’s AI products, is to have an entire portfolio of processors, each with its own specialisation—for neural networks, for self-driving cars and for inference at the edge.

If the history of other semiconductor markets, such as networking processors, is any guide, the new field of AI chips could consolidate before too long, perhaps with one or two processor architectures winning the day. There is already talk that big cloud-computing firms, such as Amazon, are interested in buying startups, including Cerebras and Graphcore. And incumbents are trying to catch up. Intel has developed a program that ties together all its AI chips; Nvidia has tweaked the architecture of its processors, which is said to now match the performance of Google’s TPUs.

But there are forces that push toward fragmentation. Specialisation in AI chips can go very far, just as with animals (cheetahs are the only large cats whose claws do not retract, so they are ready to accelerate and catch a gazelle at all times). Pierre Ferragu of New Street says that ever more demanding AI workloads needing special treatment, fast-evolving algorithms, and tech giants designing their custom chips all may lead to a world in which lots of processor architectures thrive.

China, too, is likely to inject more diversity. The government has plans to spend tens of billions to create a national semiconductor industry in an effort to be less dependent on Western imports. According to some estimates, hundreds of firms are developing ASICs. Alibaba has announced that it is working on its own AI chip, called Ali-NPU (which stands for neural processing unit). Cambricon, a startup based in Shanghai, recently unveiled a chip that is similar to Graphcore’s and Cerebras’s. The chip kingdom is unlikely to become a dull monoculture again anytime soon.