Deep Neural Network Training with iPSO Algorithm

Abstract

Deep learning-based methods are frequently preferred in many areas in recent years. Another issue, which is as important as deep neural networks applications, is the training of deep neural networks. Although many techniques are proposed in the literature for the training of deep nets, most of these techniques use gradient descent based approaches. In this study, differently from the conventional gradient method, Improved Particle Swam Optimisation (IPSO) algorithm is used for the training of deep neural networks. LeNet-5 network is preferred as network structure and MNIST is utilized as data set. Depending on the number of particles, a performance of up to 96.29% was achieved. In the cases after 20 particles, the average performance was over 90%.