Neural Network Pruning Techniques for Efficient Model Compression - Kumari, Kukati Aruna; Ahamad, Shahanawaj; Patil, Trupti; Sardana, Kamal; Muniyandy, Elangovan; Pilli, Daniel (International Journal of Intelligent Systems and Applications in Engineering, 2024)
A network of neurons When it comes to meeting the growing need for deploying deep learning models on devices with limited resources, pruning has emerged as an essential strategy for model reduction. The purpose of this study is to offer a detailed review of several pruning approaches that attempt to reduce the size and computational complexity of neural networks while maintaining their predictive accuracy. Specifically, the major emphasis is placed on structured pruning techniques, which include the removal of whole neurons, channels, or layers in a methodical manner based on certain criteria. In this article, we go into the fundamental ideas that underlie magnitude-based pruning, weight clustering, and filter pruning, and we emphasize the usefulness of these techniques in achieving considerable model reduction. In addition, this work investigates the interaction between pruning procedures and fine-tuning tactics in order to reduce the possibility of accuracy loss. In addition, the research investigates unstructured pruning techniques, which entail the elimination of individual weights in order to bring about sparsity in the network. The difficulties that are connected with unstructured pruning are discussed, and methods such as iterative pruning and regularization procedures are investigated as potential ways to improve the effectiveness of this kind of pruning. The comparative comparison of these different pruning strategies gives insight on the advantages, disadvantages, and compromises associated with each of them. Additionally, we highlight recent breakthroughs, including the integration of neural architecture search with pruning and the examination of pruning in the context of specialized neural network topologies like transformers.