Performance Analysis of Different Neuron Activation Function Implementations on FPGA using Artificial Neural Network

Abstract

Artificial Neural Network (ANNs) is a part of a computing system that design to process data information in the same way as the human brain working. It has been trained to process a large amount of data in the network and successfully applied in many applications such as pattern recognition. The high computational technique and processing power required to develop Artificial Neural Network architecture allow it to be embedded into Field Programmable Array (FPGA). This allows for even more parallelism in the network implementation. Furthermore, the activation function selection is also significant in Artificial Neural Network as it influences the network's performance. This work aims to implement different neuron activation functions on an Artificial Neural Network architecture. The design has been chosen to implement on FPGA, specifically the Virtex-7 board based on Xilinx FPGA. Xilinx Vivado software was used for synthesizing and implementing the architecture to analyse the performance of each neuron activation function. At the end, the comparison of different activation function reveals that there is not a significant difference between the activation function tested. Every activation function has almost given the same impact on a network once it has been trained.