An Energy-Efficient Parallelism Scheme for Deep Neural Network Training and Inferencing on Heterogeneous Cloud Resources

The emergence of Large Language Models(LLM) and generative AI has led to an explosive increase in computational demands across cloud computing data centers. The growing number of parameters in deep learning models results in significant power consumption problem, leading to the need for cost-effective and eco-friendly data centers. Furthermore, with the advent of multi-cloud environments, deep learning computations, not only for training but also for inference, no longer occur on a single hardware unit but are distributed across various heterogeneous hardware nodes forming clusters. In this paper, we present solutions to these challenges from a parallelism perspective. Considering the characteristics of the models, we implement data parallelism and model parallelism, partitioning models and data across heterogeneous hardware nodes for power-efficient learning and inferencing. To quantify the impact, we measured the power consumption of CPUs, GPUs, and RAM during the experiments, providing insights into the energy efficiency of the proposed partitioning strategies. Furthermore, we conducted a carbon footprint analysis, converting the measured power consumption into equivalent carbon emissions. The study highlights the necessity of partitioning research for energy-efficient learning and inferencing, addressing the identified issues.