Fujitsu has unveiled new middleware that reportedly delivers over a 2x boost in GPU computational efficiency for artificial intelligence (AI) tasks during trials. This innovation was developed specifically to address the GPU limitations and shortages arising from the increasing computing demands of AI.
According to a recent press release, the primary objective of this middleware, launched today for global customers, is to enhance resource allocation and memory management across a variety of platforms and applications utilizing AI. Fujitsu has already begun pilot programs with several partners, with additional technology trials set to start this month.
Fujitsu initiated tests of its new middleware in May, collaborating with AWL, Xtreme-D, and Morgenrot. The results indicated an impressive 2.25x improvement in computational efficiency for AI workloads. Furthermore, the partners experienced a significant uptick in the number of AI processes that could be managed concurrently across different cloud environments and servers when utilizing this middleware.
Hisashi Ito, CTO of Morgenrot, commented in a press statement, “By enabling GPU sharing among multiple jobs, we achieved a notable nearly 10% reduction in overall execution time compared to executing jobs sequentially on two GPUs. This parallel processing capability facilitated simultaneous long training sessions for model development and shorter inference/testing tasks, all while operating within limited resources.”
This month, Tradom is set to start trials with its new product, while Sakura Internet is currently conducting a feasibility study to explore the technology’s potential for its data center operations, as reported by Fujitsu.
With GPUs proving to be more effective for AI processing than CPUs, their usage has surged significantly. However, this spike in demand has led to increased power consumption in data centers and caused a shortage of GPUs. As a result, companies are actively seeking alternative solutions to optimize their AI workloads.
“The rapid expansion of compute infrastructure to support training for genAI has created a major electrical power availability challenge,” mentioned a Gartner research note addressing emerging technologies for energy-efficient generative AI compute systems authored by researchers Gaurav Gupta, Menglin Cao, Alan Priestley, Akhil Singh, and Joseph Unsworth.
This situation requires those managing AI data centers to seek immediate solutions to tackle the issues at hand, which include escalating costs, inadequate power supply, and declining sustainability performance. “These challenges will ultimately be transferred to the customers and end users of data center operators,” the researchers emphasized.
Simultaneously, data centers are tasked with managing performance challenges that have arisen from the push towards GPU-assisted AI, as highlighted by Eckhardt Fischer, a senior research analyst at IDC. He stated, “Enhancements in computer systems designed to alleviate this bottleneck will typically result in a noticeable boost in output.”
The performance hurdles tied to AI and generative AI computing needs involve memory and networking, as noted by Gupta from Gartner. He emphasized that “the current pace of Moore’s Law cannot keep up with the surging demands for compute power.”
In an effort to address these issues, Fujitsu has introduced AI computing broker middleware, which leverages a blend of adaptive GPU allocation technology that the company developed in November 2023, alongside various AI-processing optimization techniques. According to Fujitsu, this innovative middleware enables automatic identification and optimization of CPU and GPU resources across multiple AI processing programs, ensuring that processes requiring high execution efficiency receive the priority they need.
Unlike traditional resource allocation methods that operate on a per-job foundation, Fujitsu’s AI computing broker allocates resources dynamically on a per-GPU basis. The goal is to enhance availability rates and facilitate the simultaneous execution of various AI processes without the need to worry about GPU memory limitations or physical resource capacity.
Gupta expressed that the idea behind the middleware is logical, particularly considering that the energy consumption of GPUs is a significant issue, making energy efficiency a critical factor.
He mentioned, “While this doesn’t directly address the shortage issue, it does enhance utilization and operational efficiency – essentially allowing more to be achieved with less, provided the technology functions as intended,” which remains uncertain given that it is still in the early stages.
Nonetheless, Gupta pointed out that if Fujitsu’s AI-driven middleware can indeed enhance memory and GPU utilization, it deserves attention. Observing its implementation and the ensuing competitive dynamics for similar technologies will be important.
