Intel is releasing a whole arsenal of AI chips and systems hoping something will stick in the market. Its latest entry is a neuromorphic system called Hala Point. The system includes Intel’s research chip called Loihi 2, an exotic AI chip inspired by the design and functioning of the human brain.
“This rivals and exceeds levels achieved by architectures built on graphics processing units (GPU) and central processing units (CPU),” Intel claimed on its website.
It bears repeating: Hala Point is a research system, and the Loihi 2 chip is still in Intel’s lab and not commercially available yet. It won’t unseat the dominance of Nvidia GPUs anytime soon.
Instead, it may be a blueprint for a new computing architecture in which computing travels to the data. Today’s computing designs rely on data traveling to computing, which creates bottlenecks.
The Hala Point system and Loihi 2 system pursue a new concept of “sparse computing,” which the U.S. Department of Defense is studying as a computing architecture of the future.
The chip’s architecture is more effective at processing audio and video, which Intel highlighted in a research paper published last month.
The Promise of Intel’s Neurochip
Intel announced its first neuromorphic chip in 2017 as it encountered a predicament around scaling manufacturing capabilities and reducing chip sizes.
Current AI computing techniques involve moving data to compute, which is considered inefficient for AI due to bottlenecks in storage, memory, and processing. Loihi 2’s neuromorphic design focuses on efficient execution at the points of data, which frees up bandwidth and storage.
The proliferation of data makes current computing models unsustainable, especially for post-exascale systems, said William Harrod, a program manager at the U.S. Department of Defense’s Intelligence Advanced Research Projects Activity (IARPA), during a keynote at the Supercomputing 22 conference.
IARPA has a data-centric computing program to “define the future of computing based on the data movement problem, not on floating point units of ALUs,” Harrod said.
The program, called AGILE (Advanced Graphical Intelligence Logical Computing Environment), focuses on a new architecture that moves compute to data instead of vice versa to generate faster real-time results.
Intel, Qualcomm, and AMD are among the companies pushing for new chip designs and computer architectures for the program. The companies are submitting proposals to IARPA.
A research paper published last month by Intel’s researchers compared Loihi 2 to Nvidia’s Jetson Orin Nano. The researchers concluded that Loihi 2 had power and performance advantages over Orin Nano but that Nvidia’s chip was better when computing at scale.
False Promises of Silicon Brains
Neuromorphic chips have been under development at chip makers and universities for more than a decade.
The exotic chips were hyped as bringing a higher level of intelligence to computers, a role assumed by AI chips such as Nvidia’s GPUs. At the time, IBM and others were also developing neuromorphic chips.
Neuromorphic chips are probabilistic and factor in uncertainty and randomness in computing. These chips are designed to replicate the structure and functioning of the brain, which itself is still a mystery.
Most conventional AI chips operate on deterministic models, which are more precise and rely on machine learning and associated algorithms. However, AI chips can also incorporate probabilistic models.
Computing cores in neuromorphic chips play the role of neurons, which are interconnected (like synapses). Computing in neurons across synapses happens in parallel. The low-precision computing points reach conclusions by understanding trends and associations in the information.
A brain can recognize cats through many neurons working in parallel, and Intel’s chip is intended to function similarly.
Intel’s Hala Point scales up the number of computing neurons in its system to 1.15 billion, which is nowhere close to the scale of the human brain (approximately 86 billion). But the computational capacity is significantly larger than its 2020 neuromorphic system called Pohoiki Springs, which had 100 million neurons.
The chip can “support up to 20 quadrillion operations per second, or 20 petaops, with an efficiency exceeding 15 trillion 8-bit operations per second per watt (TOPS/W) when executing conventional deep neural networks,” Intel claimed. The claims have not been independently verified.
Origins of Neuromorphic Chips
Intel’s first silicon brain effort came out in 2017, but other chip developments were already underway. IBM was developing its own neuromorphic chips, and Qualcomm was developing its Zeroth chip.
In 2013, Qualcomm released Zeroth, and CEO Paul Jacobs teased the idea of smartphones being perceptive and anticipating the needs of smartphone users. His prediction took a while to come true, but smartphones like Google Pixel can now adapt to user needs.
Efforts in the U.S. and EU to fund brain-inspired chips in the 2010s vanished without a trace.
Starting in 2008, DARPA funded a program called Multiphase Synapse (Systems of Neuromorphic Adaptive Plastic Scalable Electronics), which involved IBM, Hewlett-Packard, Cornell, Stanford University, and other universities.
The EU funded a $1.6 billion Human Brain Project to understand how the brain works, and one component included developing a chip based on the design. The Human Brain Project has now ended.
IBM remains one of the few organizations developing a silicon brain, and last year, it released a chip called NorthPole, which focuses more on power efficiency as opposed to scaling.
Intel’s Neurochip
The Hala Point system includes the Loihi 2 chip, which is made using the Intel 4 process and has 128 cores per chip. Each chip includes up to 1 million digital neurons and 120 million synapses. The Loihi 2 throughput is achieved by electrical impulses that facilitate communication between neurons.
Loihi 2 is Intel’s second-generation neuromorphic research chip. (Credit: Intel Corporation)
The Hala Point chip has 1,152 Loihi 2 chips and 1.15 billion neurons with 128 billion synapses. The six-rack system consumes 2,600 watts of power, and the chips are organized in a mesh design, with six asynchronous parallel lanes to interconnect multiple Loihi 2 chips.
A 10Gbps Ethernet connector facilitates communication. Mesh configurations can range from one to “thousands of chips,” Intel said.
“Further development will enable applications of neuromorphic computing that overcome power and latency constraints that currently limit the real-world, real-time deployment of AI capabilities,” Intel said in a research paper.
The chips are artificial neural networks that can perform many concurrent audio and video functions. Computation is sparse and performed more at the point where data is located.
“While the GPUs, Tensor processors, and deep learning accelerators of today focus on dense matrix-based computation at a very high throughput, neuromorphic processors focus on sparse event-driven computation that minimizes activity and data movement,” Intel’s researchers wrote in the paper.
Intel’s research compared its chip on an artificial neural network versus a Jetson Orin Nano chip, which was released about a year ago.
Intel Hala Point system composed of Intel Loihi 2 neuromorphic processors. (Source: Intel Corporation)
The research paper concluded that neurons on the Loihi 2 chip activated only when needed, compared to the Jetson chip on the sparse computing model. That was more energy efficient and made better use of bandwidth and other computing resources.
However, the Jetson Orin held an advantage on larger operations when computing cores were fully fired up and operating at scale within comparable bandwidth.
Where the Chip Fits
The Loihi 2 survived Pat Gelsinger’s axe during restructuring, so the company may see value in the research. The Hala Point is deployed at Sandia National Laboratories, which tests a variety of chips and configurations for energy-efficient AI.
The chip is also part of Intel’s stable of AI chips, which includes Gaudi 3 and Ponte Vecchio GPU. It does not present any current threat to any CPU or GPU, including Intel’s offerings.
Neuromorphic is another chip concept as Intel mulls its future. Making the chip commercially viable is a challenge. Like quantum processors, these chips will require software frameworks and algorithms.
Tags:
AGILE, ALU, DARPA, Gaudi 3, GPU, Hala Point, Human Brain Project, IARPA, Jetson Orin Nano, Loihi 2, neuromorphic, Ponte Vecchio
