Performance evaluation of NLP and CNN models for disaster detection using social media data

Abstract

The use of social media data for disaster-type identification has been turning progressively important in recent years. With the extensive dependency on social networking sites, people can share real-time information and updates about disasters, making it a valuable source of information for disaster management organizations. The use of natural language processing (NLP) and computer vision techniques can help process and examine large amounts of social media data to gain valuable insights into the nature and extent of a disaster. In this study, NLP, and convolutional neural networks (CNN) models were applied to social media data for disaster-type recognition. The language models used were BERT-Base-Uncased, DistilBERT-Base-Uncased, Twitter-RoBERTa-Base, and FinBERT. Two convolutional neural network (CNN) models, Inception v3 and DenseNet were also applied. The models were evaluated on the CrisisMMD dataset. The outcome proved that the language models achieved a uniform accuracy of 94% across disaster-related tweet classification tasks, while DistilBERT-Base-Uncased demonstrated the fastest training and testing time which is important for prompt response systems. In terms of the CNN models, DenseNet outperformed Inception v3 just by a small margin of 1 or 2% in terms of accuracy, recall, precision, and F1 score. This entails that the DistilBERT-Base-Uncased and DenseNet model has the potential to be better suited for disaster-type recognition using social media data in terms of accuracy and time.