BPGD-AG: A New Improvement Of Back-Propagation Neural Network Learning Algorithms With Adaptive Gain

  • Nazri Mohd Nawi
  • R.S. Ransing
  • Norhamreeza Abdul Hamid
Keywords: Neural Networks, Gain, Activation function, Learning rate, Training Efficiency.

Abstract

The back propagation algorithm is one of the popular learning algorithms to train self learning feed forward neural networks. However, the convergence of this algorithm is slow mainly because the algorithm required the designers to arbitrarily select parameters such as network topology, initial weights and biases, learning rate value, the activation function, value for gain in activation function and momentum. An improper choice of theses parameters can result the training process comes to as standstill or get stuck at local minima. Previous research demonstrated that in a back propagation algorithm, the slope of the activation function is directly influenced by a parameter referred to as ‘gain’. In this paper, the influence of the variation of ‘gain’ on the learning ability of a back propagation neural network is analysed. Multi layer feed forward neural networks have been assessed. Physical interpretation of the relationship between the gain value and the learning rate and weight values is given. Instead of a constant ‘gain’ value, we propose an algorithm to change the gain value adaptively for each node. The efficiency of the proposed algorithm is verified by means of simulation on a function approximation problem using sequential as well as batch modes of training. The results show that the proposed algorithm significantly improves the learning speed of the general back-propagation algorithm.

Author Biographies

Nazri Mohd Nawi
Faculty of Science Computer and Information Technology
Universiti Tun Hussein Onn Malaysia
86400, Parit Raja, Batu Pahat, Johor, MALAYSIA
R.S. Ransing
Civil and Computational Engineering Centre, School of Engineering
University of Wales Swansea
Singleton Park, Swansea, SA2 8PP, UNITED KINGDOM
Norhamreeza Abdul Hamid
Faculty of Science Computer and Information Technology
Universiti Tun Hussein Onn Malaysia
86400, Parit Raja, Batu Pahat, Johor, MALAYSIA
How to Cite
Mohd Nawi, N., Ransing, R., & Abdul Hamid, N. (1). BPGD-AG: A New Improvement Of Back-Propagation Neural Network Learning Algorithms With Adaptive Gain. Journal of Science and Technology, 2(2). Retrieved from https://publisher.uthm.edu.my/ojs/index.php/JST/article/view/262
Section
Articles

Most read articles by the same author(s)