AI chips are about to be disrupted? The efficiency is 100 times higher than the CPU, and the power consumption is 1000 times lower than the GPU

Jessie March 12, 2024

In the last century, a project called "skunkworks" built an engineering workstation whose computing power was excellent for its time, but in order to drive this big thing, the processor and the motherboard relied on complex water-cooling systems to dissipate heat. Coincidentally, this cooling system malfunctioned, and eventually the inside of the computer completely melted down.

Now, with the increasing demand for general-purpose computing power such as Gpus, and the increasing number of edge devices deployed each year, more people are paying attention to the energy requirements and heat dissipation of computing. In other words, whoever can release more computing power at lower power consumption will win the future.

In recent days, Efficient Computer, a foreign chip startup, has moved out of "stealth mode." And a new Processor with a Reconfigurable Dataflow Processor Architecture, the Fabric architecture.

According to the company, its Fabric architecture is 100 times more efficient than market-leading general-purpose cpus and consumes 1,000 times less power than Gpus, marking the beginning of a new era of general-purpose edge computing, unlocking a wide range of applications previously limited by energy constraints.

In fact, reconfigurable chips are a chip route that has been concerned for a long time, not only favored by experts such as Wei Shaojun, but also domestic products have been available.

A chip architecture that disrupts the market

According to Exonet, Efficient's Fabric architecture was developed over seven years of research at Carnegie Mellon University, and it already has its first test chip, called Monza, but the company has yet to reveal actual performance data for chips based on the Fabric architecture.

However, it is certain that the performance of the processor under this architecture is not generally robust - that is, the efficiency described above is 100 times higher than the leading general purpose CPU on the market and the power consumption is 1000 times lower than the GPU, and the energy efficiency ratio is 100 times higher than the leading general purpose CPU on the market.

That is to say, it can not only achieve lower operating costs, but also eliminate the energy barrier of edge computing, and the entire market logic of digital chips will be subverted.

So, what exactly is reconfigurable processor architecture, and why is it so powerful?

According to Efficient, traditional general-purpose processors are capable of handling almost every possible workload and are even backward compatible with software released decades ago, which greatly increases their complexity and ultimately power consumption.

The excessive generality of these processors led to a lot of effort being spent on unnecessary internal data movement and instruction control overhead,

A reconfigurable architecture is, as the name suggests, a data flow processor architecture that can be reconfigured, tailored for a specific use case, to execute specially optimized code in parallel on its "compute structure." This is especially true in low-power embedded and edge computing, and requires proprietary software stacks (compilers) that support general-purpose programming languages.

The operating principle can be interpreted as that CPU resources are adjusted by software for specific workloads, greatly improving efficiency. At the same time, Efficient says Fabric can handle general data processing calculations, data analytics, and for AI and ML, suggesting that Efficient is dealing with an inherently parallel architecture.

Efficient's software stack supports major embedded languages, so developers of real-world applications will be able to quickly recompile code for architectural architectures. However, for this architecture, recompiling software is necessary, so software compatibility will be the limit of reconfigurable processors.

Efficient Computer has raised a $16 million seed round led by Eclipse, a venture capital firm that has funded Cerebras, FlexLogix, and Tenstorrent.

Reconfigurable chip, not FPGA

Wei Shaojun, former director of the Institute of Microelectronics of Tsinghua University and IEEE Fellow, is the founder of the first generation of reconfigurable computing architecture.

"Dynamic reconfigurable chips are often misunderstood as FPGas. Not just domestically, but internationally as well." Wei Shaojun wrote in the paper, according to the introduction, as early as the early 1960s, the concept of reconfiguration has been proposed, after nearly 60 years, the real significance of the reconfiguration problem has been a breakthrough, indicating that the technical difficulty is very high.

The so-called reconfigurable can be divided into static reconfigurable and dynamic reconfigurable, the most typical reconfigurable computing chip with static reconfigurable characteristics is FPGA, and the focus of this paper is that "reconfigurable" is a dynamic reconfigurable architecture of the processor, so collectively referred to as "reconfigurable chip".

People familiar with CPU, DSP hardware programmability is weak, software programmability is strong; ASIC, SoC software and hardware programmability are weak; FPGA and EPLD have strong hardware programmability, but weak software programmability.

Reconfigurable chips are not only software programmable, but also hardware programmable, sometimes called RCP or CGRA, etc., which is characterized by: software and hardware can be programmed, mixed granularity, the hardware function of the chip changes with the software, the application changes the software, the software changes the hardware. In addition, reconfigurable chips have many similarities with processors such as cpus, and developers do not need knowledge of underlying chip design.

In summary, the expected characteristics and potential capabilities of reconfigurable chips and traditional chips can be summarized as follows:

• Software and hardware can be programmed;
• Dynamic variability of hardware architecture and efficient architecture transformation capability;
• Both high computational efficiency and high energy efficiency;
• Intrinsic safety;
• Application simplicity, does not require chip design knowledge and ability;
• Software defined chip, that is, it is a kind of "general purpose chip", which has both universality and specificity.
• The ability to achieve intelligence, metaphorically speaking, is that the chip use process is constantly "learning" and improving itself through "education."

So the question is, since there are FPGas, there are ASics, why do we spend a lot of effort to research reconfigurable chips?

In fact, with the advancement of integrated circuit process technology to 14nm, the comprehensive cost of ASIC is as high as 150 to 200 million US dollars, usually to sell more than 30 million, in order to reasonably amortize the research and development cost to each chip, but ASIC itself is characterized by many varieties and small batches, and sales are difficult to be guaranteed. Compared with ASics, FPGas do not have the advantages of energy efficiency and computational efficiency.

Reconfigurable chip is a solution to the above problems. "Imagine if only one 'universal' chip is produced, its function can be changed through software, and when different software is written, it becomes a 'dedicated' chip," the paper says. This would be a very ideal situation. If this idea can be realized, software defined chips can be considered a reality."

The basic architecture of the reconfigurable chip still uses the basic architecture of the ASIC, the difference is that the control unit has become a universal control unit, and the universal data channel is a two-dimensional processing unit (PE) array that can be defined according to needs, which can be isomorphic or heterogeneous, and is run in a data-driven way.

Of course, nothing in the world is perfect. The overhead of reconfigurable chips is significant; for example, 25-71% of execution time is spent on refactoring in DISC II systems and 98.5% in UCLAATR.

In addition, its programming complexity is high, traditional CPU uses Java, C/C++ and other mature programming languages, reconfigurable computing requires hardware programming, usually using hardware programming languages (such as Verilog, VHDL, etc.), these languages are difficult to master.

More importantly, the process is updated every two years to follow Moore's Law, and the performance improvement brought by the architecture update is not as direct as the process update. There are too many core technologies of reconfigurable chips (configuration information is greatly reduced and "implicit configuration-data-driven" technology, configuration information efficient loading and correlation sensing cache and wheel load technology, efficient array architecture and control intensive task parallelization method, time domain and spatial cooperative mapping technology), so for so many years, reconfigurable computing has been almost buried in history.

Products are starting to appear in China

To develop reconfigurable computing is to lay out the future. China has been exploring reconfigurable chips. More than a decade ago, Chinese scholars were keenly aware of the huge potential of reconfigurable computing technology, and with the support of the key topics of the 863 Plan and major national science and technology projects, conducted long-term research, and achieved a series of research results with important international influence.

In April 2015, Tsinghua University cooperated with Intel to build new Computing hardware and software based on Intel's processor Architecture and Tsinghua's Reconfigurable Computing.

On January 24, 2018, MIT Technology Review featured a report on the research results of reconfigurable chips, arguing that this technology can dynamically adjust computing and memory parameters to meet the different needs of real-time AI software, which is a "Crowning Achievement" achieved by China.

Now, a handful of companies and products are starting to emerge in China.

AI market, or will be disrupted

It can be said that reconfigurable computing chips are crucial to the AIGC that is now exploding.

With the current semiconductor process linewidth approaching the physical limit, it is more and more difficult to improve the performance and power consumption of integrated circuits by relying on process technology. The continuation of Moore's Law through architecture innovation and the continuous gain in performance, power consumption and cost has become a hot spot of international research.

In 2015, the International Semiconductor Technology Development Roadmap (ITRS) identified coarse-grained Reconfigurable Architecture (CGRA) as one of the most promising emerging computing architectures for the future.

Coincidentally, the electronic revitalization program proposed by DARPA in the United States takes architecture innovation as one of the three key research directions.

However, when a chip enters the commercial market, it must not only face various tests of customers, but also have a better cost performance. In contrast, the current market, the number of players is relatively rare, and the technical maturity is far from the ideal degree.

However, in the increasingly slowing down of Moore's Law, the way to improve the structure is increasingly bright. It can be seen that this track may be about to break out now that the demand for AI chips is surging.

Disclaimer: This article is a reprint article, reprint this article to convey more information, copyright belongs to the original author. If the video, pictures and text used in this article involve copyright issues, please contact Xiaobian for processing. Email address: sales08@elecsaler.com