DeepMind AI finds a new way to multiply numbers and speed up computers

But the mathematician Volker Strassen proved in 1969 that multiplying a matrix of two rows of two numbers by another of the same size does not necessarily imply eight multiplications and that, by a clever trick, it can be reduced to seven. This approach, called Strassen’s algorithm requires a few more additions, but this is acceptable because additions in a computer take much less time than multiplications.

The algorithm has been the most efficient approach on most matrix sizes for over 50 years, although some slight improvements that are not easily adapted to computer code have been found . But DeepMind’s AI has now discovered a faster technique that works great on current hardware. The company’s new AI, AlphaTensor, started with no knowledge of the solutions and faced the problem of creating a working algorithm that completed the task with the minimum number of steps.

He found an algorithm to multiply two matrices of four rows of four numbers using only 47 multiplications, which surpasses Strassen’s 49 multiplications. He also developed improved techniques for multiplying matrices of other sizes, 70 in total.

AlphaTensor discovered thousands of working algorithms for every matrix size, including 14,000 for 4×4 matrices alone. But only a small minority were better than the state of the art. The research is based on AlphaZero DeepMind’s game model, and lasted two years.

Hussein Fawzi from Deepmind says the results are mathematically sound, but far from intuitive to humans. “We don’t really know why the system created this, basically,” he says. “Why is this the best way to multiply matrices?” It’s unclear. »

“Somehow neural networks have an intuition of what looks good and what looks bad. Honestly, I can’t tell you exactly how it works. I think there’s theoretical work to be done out there on exactly how deep learning manages to do this stuff,” Fawzi says.

DeepMind has found that the algorithms can increase computing speed by 10-20% on certain hardware such as an Nvidia V100 graphics processing unit (GPU) and a Google tensor processing unit (TPU) v2, but there is no guarantee that these gains would also be seen on common devices like a smartphone or laptop.

James Knight from the University of Sussex, UK, claims that a range of software running on supercomputers and powerful hardware, such as AI research and weather simulation, is actually large-scale matrix multiplication.

“If this kind of approach was actually implemented there, it could be a kind of universal acceleration,” he says. “If Nvidia implemented this in its CUDA library [un outil qui permet aux GPU de fonctionner ensemble], it would reduce a certain percentage of most deep learning workloads I would say. »

Oded Lachish de Birkbeck, University of London, says the new algorithms could increase the efficiency of a wide range of software because matrix multiplication is such a common problem – and other algorithms are likely to follow.

“I think we’ll see AI-generated results for other problems of a similar nature, although rarely something as central as matrix multiplication. There is a significant motivation for such technology, because fewer operations in an algorithm not only means faster results, it also means less energy expended,” he says. If a task can be performed a little more efficiently, it can be performed on less powerful, less power-hungry hardware, or on the same hardware in less time, using less power.

But DeepMind’s progress doesn’t necessarily mean that human coders are out of work. “Should programmers be worried? Maybe in the distant future. Automatic optimization has been practiced for decades in the microchip design industry and is just another important tool in the coder’s arsenal,” says Lachish.

Journal reference: Nature , DOI: 10.1038/s41586-022-05172-4

We would love to thank the author of this post for this amazing web content

DeepMind AI finds a new way to multiply numbers and speed up computers


Find here our social media accounts as well as other pages related to it.https://www.ai-magazine.com/related-pages/