BPGD-AG: A New Improvement Of Back-Propagation Neural Network Learning Algorithms With Adaptive Gain

Authors

  • Nazri Mohd Nawi
  • R.S. Ransing
  • Norhamreeza Abdul Hamid

Keywords:

Neural Networks, Gain, Activation function, Learning rate, Training Efficiency.

Abstract

The back propagation algorithm is one of the popular learning algorithms to train self learning feed forward neural networks. However, the convergence of this algorithm is slow mainly because the algorithm required the designers to arbitrarily select parameters such as network topology, initial weights and biases, learning rate value, the activation function, value for gain in activation function and momentum. An improper choice of theses parameters can result the training process comes to as standstill or get stuck at local minima. Previous research demonstrated that in a back propagation algorithm, the slope of the activation function is directly influenced by a parameter referred to as ‘gain’. In this paper, the influence of the variation of ‘gain’ on the learning ability of a back propagation neural network is analysed. Multi layer feed forward neural networks have been assessed. Physical interpretation of the relationship between the gain value and the learning rate and weight values is given. Instead of a constant ‘gain’ value, we propose an algorithm to change the gain value adaptively for each node. The efficiency of the proposed algorithm is verified by means of simulation on a function approximation problem using sequential as well as batch modes of training. The results show that the proposed algorithm significantly improves the learning speed of the general back-propagation algorithm.

Downloads

Download data is not yet available.

Author Biographies

  • Nazri Mohd Nawi
    Faculty of Science Computer and Information Technology
    Universiti Tun Hussein Onn Malaysia
    86400, Parit Raja, Batu Pahat, Johor, MALAYSIA
  • R.S. Ransing
    Civil and Computational Engineering Centre, School of Engineering
    University of Wales Swansea
    Singleton Park, Swansea, SA2 8PP, UNITED KINGDOM
  • Norhamreeza Abdul Hamid
    Faculty of Science Computer and Information Technology
    Universiti Tun Hussein Onn Malaysia
    86400, Parit Raja, Batu Pahat, Johor, MALAYSIA

Downloads

Issue

Section

Articles

How to Cite

Mohd Nawi, N., Ransing, R., & Abdul Hamid, N. (2010). BPGD-AG: A New Improvement Of Back-Propagation Neural Network Learning Algorithms With Adaptive Gain. Journal of Science and Technology, 2(2). https://publisher.uthm.edu.my/ojs/index.php/JST/article/view/262