Transformer-based structural response prediction and transfer learning: A numerical analysis

Author

• Sudip Kumar Bogati, Southern Illinois University CarbondaleFollow

Date of Award

5-1-2026

Degree Name

Master of Science

Department

Civil Engineering

First Advisor

Sen, Debarshi

Abstract

Prediction of structural response has evolved from Finite Element Analysis (FEA) to Deep Learning (DL) models. These conventional FEA methods are computationally intensive, require a full understanding of the structural model, and pose significant challenges in modern Structural Health Monitoring (SHM) scenarios. To overcome these issues, various DL models, such as Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTMs), and Transformers, have been introduced. Traditional methods like RNNs and LSTMs capture temporal dependencies but struggle to capture global patterns in long-term sequences and noisy data. Therefore, in this study, we analyze a Sequence-to-Sequence (Seq2Seq) transformer architecture that offers a more effective solution by using a multi-layer encoder-decoder system to map past responses to future. A simply supported beam is modelled in MATLAB, and its structural response (training data) is computed based on the Finite Element Method (FEM). Various numerical experiments are conducted to check the accuracy and efficiency of the proposed framework. We evaluate the model across varying Signal-to-Noise Ratios (SNRs) and different numbers of prediction points, with a fixed lookback value. Finally, the knowledge gained from training the transformer model on a simply supported beam is applied to a cantilever beam using the Transfer Learning (TL) approach. The accuracy and efficiency of the trained transformer model are assessed using various metrics such as Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Time Response Assurance Criterion (TRAC). Results from these numerical experiments show that the proposed model achieves high predictive accuracy, even with noisy data and responses for unseen structures within the same population.