Accuracy Considerations in Deep Learning Using Memristive Crossbar Arrays

Author

Bijay Raj Paudel, Southern Illinois University CarbondaleFollow

Date of Award

5-1-2023

Degree Name

Doctor of Philosophy

Department

Electrical and Computer Engineering

First Advisor

Tragoudas, Spyros

Abstract

Deep neural networks (DNNs) are receiving immense attention because of their ability to solve complex problems. However, running a DNN requires a very large number of computations. Hence, dedicated hardware optimized for running deep learning algorithms known as neuromorphic architectures is often utilized. This dissertation focuses on evaluating andenhancing the accuracy of these neuromorphic architectures considering the designs of components, process variations, and adversarial attacks. The first contribution of the dissertation (Chapter 2) proposes design enhancements in analog Memristive Crossbar Array(MCA)-based neuromorphic architectures to improve classification accuracy. It introduces an analog Winner-Take-All (WTA) architecture and an on-chip training architecture. WTA ensures that the classification of the analog MCA is correct at the final selection level and the highest probability is selected. In particular, this dissertation presents a design of a highly scalable and precise current-mode WTA circuit with digital address generation. The design is based on current mirrors and comparators that use the cross-coupled latch structure. A post-silicon calibration circuit is also presented to handle process variations. On-chip training ensures that there is consistency in classification accuracy among different all analog MCA-based neuromorphic chips. Finally, an enhancement to the analog on-chip training architecture by implementing the Convolutional Neural Network (CNN) on MCA and software considerations to accelerate the training is presented.The second focus of the dissertation (Chapter 3) is on producing correct classification in the presence of malicious inputs known as adversarial attacks. This dissertation shows that MCA-based neuromorphic architectures ensure correct classification when the input is compromised using existing adversarial attack models. Furthermore, it shows that adversarialrobustness can be further improved by compression-based preprocessing steps that can be implemented on MCAs. It also evaluates the impact of the architecture in Chapter 2 under adversarial attacks. It shows that adversarial attacks do not uniformly affect the classification accuracy of different MCA-based chips. Experimental evidence using a variety of datasets and attack models supports the impact of MCA-based neuromorphic architectures and compression-based preprocessing implemented on MCAs to mitigate adversarial attacks. It is also experimentally shown that the on-chip training improves consistency in mitigating adversarial attacks among different chips. The final contribution (Chapter 4) of this dissertation introduces an enhancement of the method in Chapter 3. It consists of input preprocessing using compression and subsequent rescale and rearrange operations that are implemented using MCAs. This approach further improves the robustness against adversarial attacks. The rescale and rearrange operations are implemented using a DNN consisting of fully connected and convolutional layers. Experimental results show improved defense compared to similar input preprocessing techniques on MCAs.