With the rapid advancement of technology, surpassing human abilities in tasks like image classification and language processing, evaluating the energy impact of ML is essential. Historically, ML projects prioritized accuracy over energy efficiency, contributing to increased energy consumption. Green software engineering, highlighted by Gartner as a key trend for 2024, focuses on addressing this issue. Researchers have compared ML frameworks such as TensorFlow and PyTorch in terms of energy use, leading to efforts in model optimization. However, more research is needed to assess the effectiveness of these energy-saving strategies in practice.
Researchers from Universitat Politècnica de Catalunya aimed to enhance the efficiency of image classification models by evaluating various PyTorch optimization techniques. They compared the effects of dynamic quantization and torch. compile and prune methods on 42 Hugging Face models, analyzing energy consumption, accuracy, and economic costs. Dynamic quantization significantly reduced inference time and energy use, while torch. compile balanced accuracy and energy efficiency. Local pruning showed no improvement, and global pruning increased costs due to longer optimization times.
The study outlines key concepts for understanding AI and sustainability, focusing on model-centric optimization tactics to reduce the environmental impact of ML. Inference, which accounts for 90% of ML costs, is a key area for energy optimization. Techniques like pruning, quantization, torch. compile, and knowledge distillation aims to reduce resource consumption while maintaining performance. Although most research has focused on training optimization, this study targets inference, optimizing pre-trained PyTorch models. Metrics like energy consumption, accuracy, and economic costs are analyzed using the Green Software Measurement Model (GSMM) to evaluate the impact of optimization.
The researchers conducted a technology-focused experiment to evaluate various ML optimization techniques, specifically dynamic quantization, pruning, and torch. Compile in the context of image classification tasks. Using the PyTorch framework, our study aimed to assess the impact of these optimizations on GPU utilization, power consumption, energy use, computational complexity, accuracy, and economic costs. We employed a structured methodology, analyzing data from 42 models sampled from popular datasets like ImageNet and CIFAR-10. Key metrics included inference time, optimization costs, and resource usage, with results helping guide efficient ML model development.
The study analyzes popular image classification datasets and models on Hugging Face, highlighting the dominance of ImageNet-1k and CIFAR-10. The study also examines model optimization techniques like dynamic quantization, pruning, and torch. Compile. Dynamic quantization is the most effective method, improving speed while maintaining acceptable accuracy and reducing energy consumption. Torch. Compile offers a balanced trade-off between accuracy and energy, while global pruning at 25% is a viable alternative. However, local pruning shows no accuracy improvement. The findings underscore dynamic quantization’s efficiency, particularly for smaller and less popular models.
The study discusses the implications of model optimization techniques for different stakeholders. For ML engineers, a decision tree guides the selection of techniques based on priorities like inference time, accuracy, energy consumption, and economic impact. For Hugging Face, better documentation of model details is recommended to improve reliability. PyTorch libraries should implement pruning that removes parameters rather than masking them, enhancing efficiency. The study highlights dynamic quantization’s benefits and suggests future work on NLP models, multimodal applications, and TensorFlow optimizations. Additionally, energy labels for models based on performance metrics could be developed.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..
Don’t Forget to join our 50k+ ML SubReddit
Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.
Leave a comment