CMOS BASED IMPLEMENTATION OF HYPERBOLIC TANGENT ACTIVATION FUNCTION FOR ARTIFICIAL NEURAL NETWORK

Author

Fatemeh Mohammadi Shakiba, Southern Illinois University CarbondaleFollow

Date of Award

5-1-2018

Degree Name

Master of Science

Department

Electrical and Computer Engineering

First Advisor

Tragoudas, Spyros

Abstract

FATEMEH MOHAMMADI SHAKIBA, for the Master of Science degree in MAJOR Electrical and Computer Engineering, presented on 4/11/2018, at Southern Illinois Uni- versity Carbondale. TITLE: CMOS BASED IMPLEMENTATION OF HYPERBOLIC TANGENT ACTIVA- TION FUNCTION FOR ARTIFICIAL NEURAL NETWORK MAJOR PROFESSOR: Dr.Spyros Tragoudas In this thesis, an efficient implementation for the established hyperbolic tangent acti- vation function is proposed. The efficiency of this design is considered in multiple aspects such as power consumption, simplicity of implementing, compatibility with different designs and accuracy. Considering all of these parameters, CMOS technology is chosen to be used for its implementation. This activation function is designed to resemble its mathematical definition, to reach the highest possible extent. Existing CMOS-based designs for hyperbolic tangent activation function is either power consuming for satisfying a reliable accuracy, or not compatible with both digital and ana- log circuits. Hence, we tried to implement a super fast, low power design to solve these problems. We experimented the shallow and deep neural networks with various parameters for two image processing open datasets, the MNIST and notMNIST with more than 60000 images, to present the classification accuracy of the proposed design. [5, 8] This activation function is suited for current based artificial neural networks architectures. According to symmetry feature of hyperbolic tangent function, we proposed a tricky method to have a very small circuit to simulate just half of the function, which is much faster and needs less power than existing circuits that produce a complete form of hyperbolic tangent.