Huawei’s CloudMatrix 384: A New Contender in AI Technology

With rapid advancements in artificial intelligence, Huawei has introduced a new technology that challenges market giants like Nvidia. The CloudMatrix 384 system relies on Ascend 910C processors connected via optical links, significantly enhancing machine learning performance. In this article, we will explore the details of this new system and its impact on the market.

Architecture Concepts of CloudMatrix 384

The CloudMatrix 384 system features a distributed architecture that surpasses traditional GPU-based systems. Although individual Ascend processors may be less powerful compared to their competitors, the overall system offers improvements in resource utilization and internal runtime efficiency.

This architecture is designed to maximize the use of available resources, enabling companies using this system to enhance operational efficiency and reduce operational costs.

The Challenge to Nvidia

Through this system, Huawei aims to challenge Nvidia’s dominance in the market, despite ongoing U.S. sanctions. Huawei offers a robust system that can provide companies with an effective alternative based on modern technologies that can be adapted to meet various needs.

Huawei claims that the CloudMatrix 384 technology can be a strong competitor due to its performance enhancement and independence from U.S. technologies, offering more options for global companies.

Adapting to the MindSpore Framework

Using Huawei’s new system requires a shift in how data engineers work, as they must use the MindSpore framework developed by Huawei to achieve optimal performance with Ascend processors.

If models are pre-built using PyTorch or TensorFlow, engineers will need to convert them to the MindSpore format or retrain them using MindSpore APIs. This requires additional effort in learning new APIs and training methods.

Using MindIR for Model Deployment

MindSpore uses the MindIR format, which acts as an intermediate interface similar to Nvidia NIM. After training the model in MindSpore, it can be exported using the mindspore.export tool, which converts the trained network into the MindIR format.

Deploying the model for inference requires loading the model and converting it using MindSpore APIs, which handle model deserialization, customization, and execution.

Adapting to CANN

Huawei provides a suite of CANN tools and libraries specifically designed for Ascend processors, similar to NVIDIA CUDA functions. Huawei recommends using CANN’s analysis and debugging tools to monitor and optimize model performance on devices.

Conclusion

The CloudMatrix 384 system represents a bold step by Huawei to strengthen its position in the AI market, requiring adaptation to new programming interfaces and training methods. Although Huawei’s tools are still in development and lack the widespread support of PyTorch, the company hopes that transitioning to its technologies will yield positive results, reducing reliance on U.S.-based Nvidia.