Content area
The use of social media data for disaster-type identification has been turning progressively important in recent years. With the extensive dependency on social networking sites, people can share real-time information and updates about disasters, making it a valuable source of information for disaster management organizations. The use of natural language processing (NLP) and computer vision techniques can help process and examine large amounts of social media data to gain valuable insights into the nature and extent of a disaster. In this study, NLP, and convolutional neural networks (CNN) models were applied to social media data for disaster-type recognition. The language models used were BERT-Base-Uncased, DistilBERT-Base-Uncased, Twitter-RoBERTa-Base, and FinBERT. Two convolutional neural network (CNN) models, Inception v3 and DenseNet were also applied. The models were evaluated on the CrisisMMD dataset. The outcome proved that the language models achieved a uniform accuracy of 94% across disaster-related tweet classification tasks, while DistilBERT-Base-Uncased demonstrated the fastest training and testing time which is important for prompt response systems. In terms of the CNN models, DenseNet outperformed Inception v3 just by a small margin of 1 or 2% in terms of accuracy, recall, precision, and F1 score. This entails that the DistilBERT-Base-Uncased and DenseNet model has the potential to be better suited for disaster-type recognition using social media data in terms of accuracy and time.
Introduction
In the modern day, social networking sites offer a great way to explore the interests, hobbies, and behaviors of user groups. Individuals are posting comments and multimedia information concerning personal lifestyles, likes, sentiments, and opinions more than ever before Vigna et al. (2015). On the other hand, as a result of the widespread usage of social media sites during times of crisis brought on by natural or man-made disasters, there are countless opportunities for individuals seeking information to quickly acquire insightful data Alam et al. (2017). Thus, social media can be used to forecast and identify disasters by looking at people’s posts on social media platforms Jaeger et al. (2007). Using these platforms to monitor real-time updates, images, and comments, authorities and researchers can spot early signs of crises, like unusual weather conditions, changes in sentiment, or localized emergencies. By combining this information, we’ll be able to provide early warnings and respond fast to emerging threats. One of the giant social media sites, Twitter has been used to spread news, assist in the rapid reaction to disasters, and monitor relief and recovery activities. For instance, tweets about the Virginia earthquake spread around the US more quickly than the real earthquake, according to data visualizations of this phenomenon Lotan (2011). Researchers solely focused on the creation of effective technologies like Artificial Intelligence Shaukat et al. (2020a), Time series analysis Shaukat et al. (2021a), Internet Of Things (Shaukat et al. 2021b, 2017; Tariq et al. 2023) to harness and use current data from social networking sites with the goal of humanitarian responses to help relief operations to catastrophes Castillo (2016). They created techniques like automatic information extraction from posts Yin et al. (2012), timely detection of events Ashktorab et al. (2014a), and automated image classification Alam et al. (2018). For example, the Australian Red Cross uses a computer filtration system for spam and the classification of social media posts into event types. Whereas ResilienceDirect helps to collaborate with all UK emergency services by assimilating evidence gathered from social media. Furthermore, the American Red Cross employs a monitoring system that tracks possible emergencies Pekar et al. (2020). Though the rescue request is a time- and resource-consuming process, machine learning (ML) Rabbi et al. (2024a), natural language processing (NLP), Deep Learning Shaukat et al. (2023) Shaukat et al. (2022), process mining Rabbi et al. (2024b), Network Analysis Rabbi et al. (2023) and convolutional neural networks (CNN) offer solutions.
NLP mainly focuses on human language, and computers interact by processing, analyzing, and comprehending human linguistic data using computational methods. It uses text analysis and classification to find rescue request messages on social media. This entails analyzing and classifying social media communications using NLP approaches Zhou et al. (2022). Since the beginning, NLP research has concentrated on tasks that include machine translation, information extraction, and topic modeling, to name a few Wang et al. (2024). Though during the middle of the twentieth century, only a rule-based system could be created. However, later recurrent neural networks (RNNs) were introduced, as was recently the case with opinion mining Erik and Bebo (2014). These neural networks made it feasible to complete a variety of jobs where dynamics and static values are crucial Lukauskas et al. (2022). A new type of neural network was created to tackle memory-related shortcomings, namely the long-short-term memory neural network. However, a new transformer structure was developed in 2017 Vaswani et al. (2017), which substantially altered the direction of NLP. Today, numerous models have been developed based on those structures which are being utilized by different language researchers. The GPT (Generative Pre-Training Transformer) model made significant contributions along with its successors GPT-2 Radford et al. (2019), and GPT-3 Tom et al. (2020). However, they are often used models in BERT, which is a type of transformer structural model. It is a pre-training method based on research on contextual representations Devlin et al. (2018a).
BERT (Bidirectional Encoder Representations from Transformers) and its offshoots have recently attained state-of-the-art performance on a variety of NLP tasks, such as event detection Devlin et al. (2019). The BERT model pre-trains encoders using a huge volume of textual input before making effective fine-tuning for a specific target job, including sentiment analysis and text classification. DistilBERT is a smaller and faster variant of BERT that can be used when computational resources are limited. The distillation in the model drastically reduces the model’s volume while still maintaining roughly 97 percent of the model’s correctness Lukauskas et al. (2022). Twitter-RoBERTa-Base is another pre-trained language model that has been fine-tuned for specific tasks, such as sentiment analysis and named entity recognition. FinBERT is fine-tuned using a large corpus of financial news articles and reports, allowing it to have a better understanding of financial terms, concepts, and relationships compared to a general-domain BERT model. This makes FinBERT more suitable for tasks such as sentiment analysis, named entity recognition, and question-answering in the financial domain.
On the other hand, a notable artificial neural network for classifying and recognizing images is the convolutional neural network (CNN). CNN allows extremely accurate object recognition and image categorization Shaukat et al. (2024), which led to a transformation in the computer vision field Hossain et al. (2024). It is made up of a number of layers that use different functions to transform one volume of activations into another. It consists of a series of layers, each of which uses a differentiable function to transform one volume of activations into another. The convolution layer, pooling layer, and fully connected layer are the three main hidden layers of CNN, and their neurons are arranged in three dimensions (width, height, and depth) Amit and Aoki (2017). Multiple pre-trained CNN models have been created for a variety of image recognition tasks. As they have been trained on substantial amounts of data, they have acquired beneficial characteristics for applying to fresh or unseen data Khan et al. (2023). These models include Dense Net, Inception, VGG, and so on. DenseNet is a newly suggested CNN architecture consisting of an intriguing connectivity pattern where every layer is linked to every other layer inside a dense block. Each of the layers in this scenario has access to the feature maps from the layers that came before them, which promotes extensive feature reuse. Because of this, the model is smaller and less prone to overfitting Zhu and Newsam (2017). Contrarily, since its development by Google in 2015, Inception v3 has been using various applications. The ImageNet dataset is used to train this model. It is a combination of convolutional layers, pooling layers, and inception modules that are intended to extract multi-scale details from the supplied image. It can identify 1000 classes in ImageNet with an error rate of 3.5% for the top-5 and 17.3% for the top-I Xia et al. (2017).
This research analyzes social media data to detect the types of disasters that require fast weather responses. Social media data from six different disasters has been collected from 2017 namely: the California wildfire, Hurricane Harvey Texas, Hurricane Irma Florida, Hurricane Maria Puerto Rico, Sri Lanka flood. Four different types of language models, for instance, BERT-Base-Uncased Devlin et al. (2018a), DistilBERT-Base-Uncased Rajapakse (xxxx), Twitter-RoBERTa-Base Liu et al. (2019), FinBERT Brockman et al. (xxxx) have been utilized to compare them. Additionally, two CNN models for image processing have been used including, Dense Net Huang et al. (1608), and Inception V3 Szegedy et al. (1512) for comparison. The following is a summary of the study’s overall contributions:
Demonstrate a comparison among four Natural Language Processing models to find out which one better suits the purpose of disaster type identification from social media text data.
Comparison between two CNN models for image processing on social media image data.
A comparison of all models in terms of four evaluation matrices and training/testing time is needed to gather a comprehensive summary and assess the usability of the models.
Literature Review
The literature review contains three different subsections. The 1st subsections discuss some research related to disaster detection from social media and the use of NLP in disaster detection and 2nd sub section provides the research related to image processing.
Disaster Detection using Natural Language Processing
Social media data has a lot of potential in terms of finding recent updates. Social media has been used to a great extent over the years, not only in omnipresent interaction but also in terms of emergency situations. A number of studies have been conducted on social media in terms of disaster mitigation. For instance, Sakaki et al. Sakaki et al. (2013) examined events similar to earthquakes that interact in real-time on Twitter and suggested an algorithm to track tweets and target events. Sufi et al. Sufi (2021) developed an AI-based software that utilizes automated regression analysis, decomposition analysis, and anomaly identification to uncover hidden insights on landslide features. Ashktorab et al. Ashktorab et al. (2014b) introduced a Twitter mining tool. During natural disasters, it collects information that disaster aid workers can use.
Additionally, social media text analysis is also popular in terms of disaster detection. Zheye Wang et al. Wang et al. (2016) examined tweets about wildfires to determine the user’s contextual and regional awareness. Beigi et al. Beig et al. (2016) employed NLP in sentiment analysis to find and gather thoughts and responses to a specific issue, for example, identifying geographical responses to calamities to enhance emergency preparedness. Sit et al. Sit et al. (2019) extracted information regarding disasters from tweets as well as set apart impact areas, time frames, and the relative importance of each category. They did this by combining a two-step classification method with spatial analysis. Sufi et al. Sufi and Khalil (2022) created a system to gain thorough information on social media feeds connected to disasters in 110 languages through AI and NLP-based sentiment analysis, named entity recognition (NER), anomaly detection, regression, and Getis Ord Gi* algorithms. Shahbazi et al. Shahbazi and Byun (2022) extracted real-time content linked to emergency occurrences to ease the quick response in a critical situation by utilizing machine learning (ML), deep learning (DL), and natural language processing (NLP). Verma et al. Verma et al. (2011) developed a classifier to automatically discover statements that contribute to situational awareness using four distinct crisis occurrences. Corvey et al. Corvey et al. (2010) demonstrated how NLP methods created for Twitter data will help locate and extract information during major emergencies. Ramachandran et al. Dharini and Parvathi (2021) proposed an approach aimed at adapting to different domains and events to improve the performance of tweet classification systems used in crisis management applications. The authors begin by highlighting the importance of accurate and timely classification of crisis-related tweets to aid in disaster response and relief efforts. They then introduce the limitations of existing tweet classification approaches, which fail to adapt to different domains and events and may suffer from data scarcity. To address these limitations, the authors propose a tweet augmentation approach that utilizes a combination of keyword-based, rule-based, and neural-based methods to generate additional data samples. These data samples are then used to train a classification model that can adapt to different domains and events. The authors evaluate the proposed approach using two publicly available crisis-related tweet datasets and compare its performance with several state-of-the-art tweet classification models. The experimental results show that the proposed approach outperforms the existing models, especially in scenarios with limited training data or when dealing with new and unknown events. In their latest study, Sambandam et al. (Palaniappan and Y. D, P. P and S. Swaminathan 2023) propose a deep learning model for improving the quality of geospatial data ontology by using bidirectional long short-term memory (Bi-LSTM) and attention mechanisms. The authors begin by discussing the importance of geospatial data ontology, which refers to the formal representation of concepts and relationships in geospatial data. They highlight the challenges in developing accurate and complete geospatial data ontologies, such as dealing with incomplete or ambiguous data, and the need for effective machine learning models to address these challenges. The proposed model uses a Bi-LSTM architecture with an attention mechanism to capture the temporal dependencies and spatial correlations in geospatial data. The model also incorporates an optimization algorithm to improve its accuracy and efficiency. The authors evaluate the performance of the proposed model using two real-world geospatial datasets and compare its performance with several state-of-the-art models. The outcome concludes that the proposed model outperforms the existing models, achieving higher accuracy and efficiency while improving the quality of the geospatial data ontology. Neubig et al. Neubig et al. (2011) developed a system that can gather information from earthquake victims in 2011 in East Japan. Chanda Chanda (2021) investigated the effectiveness of BERT embeddings for catastrophe prediction using Twitter data and contrasted them with conventional context-free word embedding techniques (GloVe, Skip-gram, and FastText). Ishino et al. Ishino et al. (2012) provide techniques for automatically obtaining transportation data and traffic issues from tweets released during the catastrophe that have been written in Japanese. Ofli et al. Ferda et al. (2004) suggested two concurrent deep learning frameworks, one of which represents the text modality and the other represents the imaging modality, to learn a combined model. Klein et al. (Klein et al. xxxx) suggested employing a novel event identification approach to aggregate posts and filter the real-time media stream by analyzing the seriousness of messages and extracting facts through natural language processing. Jing et al. Jing et al. (2016) demonstrate a new flooding scenario detection system that uses image content and language analytics from social networks.
Disaster Detection using Image Processing
Furthermore, numerous studies have been conducted on image analysis. For example, Bischke et al. Bischke et al. 2017) demonstrated that during flood detection, deep network-extracted visual and textual elements can be combined efficiently to get social multimedia reports that offer directed proof of flooding. Yu et al. (Yu et al. (2019) developed a neural network-structured model to address multiclass across five different themes and obtain awareness of multiple crisis situations. Aipe et al. Aipe et al. (2018) created a deep-learning CNN to categorize tweets about emergencies in a multiclass scenario. Hjorth et al. Larissa and Jean (2014) studied the types and resonant themes in the 100 photos that received the most retweets during the Queensland flood. In order to localize and quantify damage in social media photographs released following catastrophes, Li et al. Li et al. (2018) suggested an approach based on class activation mapping. Mouzannar et al. (Mouzannar et al. 2018) showed heterogeneous data that included both photos and text from social media, as well as established a deep learning strategy to recognize damaged photographs in their dataset. Amit et al. Amit et al. (2016) used a convolutional neural network (CNN) to analyze satellite photos for an automatic catastrophe detection system. Hartawan et al. Hartawan et al. (2019) developed a CNN-inspired system for identifying natural disaster victims and deployed it on a Raspberry Pi to identify victims using streaming cameras mounted on unmanned aerial vehicles. Yu et al. Yu et al. (2017) suggested an algorithm to detect landslide intelligence based on a depth convolutional neural network (CNN) and an improved region-growing algorithm (RSG_R). Islam et al. Islam et al. (2023) proposed a combined convolutional neural network (CNN) and sorting algorithm to classify images and detect areas affected by flooding with unmanned aerial vehicles. Dhongade et al. Dhongade et al. (2023) utilized twelve convolutional neural network models, namely VGG-16, Resnet-50, Inceptionv3, Densenet, Alexnet, Squeezenet, Shufflenet, Resnext, Wideresnet, Googlenet, Mobilenetv3, and a bespoke convolutional model to detect storm damage. Table 1 shows the summary of the current themes in the disaster detection literature using social media. These general themes serve as a baseline in developing the proposed model.
Table 1. Current themes in the disaster detection literature using social media
Author | Algorithm | Application |
|---|---|---|
Sakaki et al. (Sakaki et al. 2013) | Support Vector Machine (SVM) | Earthquake prediction |
Sufi et al. (Sufi 2021) | automated regression analysis | disaster recovery strategists |
Ashktorab et al. (Ashktorab et al. 2014b) | (sLDA, SVM, logistic regression) | Infrastructure damage and casualties |
Zheye Wang et al. (Wang et al. 2016) | Text mining, social network analysis, KDE, Duel KDE | wildfire management |
Beigi et al. (Beig et al. 2016), | SVM, logistic regression, LSTM, BERT | disaster response and management |
Bischke et al. (Bischke et al. 2017) | deep neural networks, RGB and Infrared (IR) channels | |
Yu et al. (Yu et al. 2019), Aipe et al. (Aipe et al. 2018) | CNN | |
Sit et al. (Sit et al. 2019) | LSTM, LDA, Spatially Adaptive Kernel Smoothing & Density-Based Spatial Clustering | Area of infrastructure damage, casualties and hotspots |
Sufi et al. (Sufi and Khalil 2022) | Sentiment Analysis, NER, CNN, Getis Ord Gi | |
Shahbazi et al. (Shahbazi and Byun 2022) | ML, DL, NLP | improve disaster management and emergency response |
Verma et al. Verma et al. (2011) | combination of hand-annotated and automatically extracted linguistic features | enhance situational awareness during mass emergencies |
Corvey et al. Corvey et al. (2010), Klein et al. (Klein et al. xxxx) | NLP | Extracting actionable information during mass emergencies |
Ramachandran et al. Dharini and Parvathi (2021) | CrisisLex lexicon, Word2Vec embeddings, and WordNe | tweets that offer or request help |
Sambandam et al. (Palaniappan and Y. D, P. P and S. Swaminathan 2023) | deep attention-based bidirectional search, rescue LSTM | ontology based geospatial data integration |
Neubig et al. Neubig et al. (2011) | word segmentation, named entity recognition | individual safety information, support relief efforts |
Chanda (Chanda 2021) | BERT embeddings | predicting disaster-related content, disaster detection and analysis |
Ishino et al. Ishino et al. (2012) | CRF + + | Extract post-disaster transportation information and traffic problems |
Ofli et al. Ferda et al. (2004), Li et al. Li et al. (2018), Mouzannar et al. Mouzannar et al. (2018), Hartawan et al. Hartawan et al. (2019) | CNN | reports of injuries, infrastructure damage, and missing people |
Amit et al. Amit et al. (2016) | Automatic Disaster Detection System | |
Jing et al. Jing et al. (2016) | Squiral Image Processing | flood event scene recognition system |
Hjorth et al. Larissa and Jean (2014) | Qualitative Analysis | Crisis and Emergency Visual Representation |
Yu et al. Yu et al. (2017) | CNN, RSG_R | Landslide Detection System |
Islam et al. Islam et al. (2023) | Inception v3, DenseNet | Autonomous Drones for Flood Management |
Dhongade et al. Dhongade et al. (2023) | VGG-16, ResNet50, Inceptionv3, DenseNet, AlexNet, SqueezeNet, ShuffleNet, ResNeXt, WideResNet, GoogLeNet, MobileNetv3 | Damage Assessment Post-Hurricane |
It is evident from the above discussion that, although a handful of research has been done on detecting the disaster from Twitter data mining, none of them integrated and reviewed the state-of-the-art BERT models such as (BERT-Base-Uncased, DistilBERT-Base-Uncased, Twitter-RoBERTa-Base, FinBERT) and CNN model (DEnseNet and Inception v3) and made a comparison of their performance. To bridge this gap, this study significantly incorporates both Natural Language Processing (NLP) such as BERT and Computer Vision (CNN) models. Throughout the analysis of both text and visual data, these multimodal techniques helped us figure out the best model for disaster detection. It also illustrates the advantages and disadvantages of modern NLP models. The BERT model unlike Word2Vec or GloVe model has azoo groundbreaking impact on how machines learn and interpret languages, making them extremely crucial in the field of machine learning. It bidirectionally processed the Twitter text, while enabling itself to capture a wide variety of language patterns to increase the efficiency of this study. Along with that, the CNN model added a visual element to disaster detection. The DenseNet and Inception v3 stand out from other traditional CNN models because of their unique architecture and efficiency. The DenseNet model’s dense connection between layers improved gradient flows, which ultimately led to higher accuracy of the model. Moreover, the Inception v3 model’s “Inception Modules” were able to recognize different types of patterns in photos for consistent performance. This study also took computing efficiency such as training testing time frames into account which is important for prompt response systems. The use of real-world data also provides the real-world scenario using the CrisisMMD data.
In general, using social media data to identify disasters is not a novel concept. because it provides a great deal more information about the disaster that is occurring on the other side of the globe. For disaster relief organizations and emergency responders, using the NLP and CNN models will surely aid in situation analysis and defense strategy creation. This study’s primary implications are streamlined as follows: (i) Improve the disaster detection system’s accuracy and dependability. (ii) save lives and lessen the effect of catastrophe by analyzing real-time social media data. (iii) Allocate proper resources and suitable reaction measures by detecting disaster types. (iv) gain insights about the disaster’s location and the severity of the damage more quickly.
Methodology
The methodology of this study involves the use of two types of models: language models for processing text data and pre-trained convolutional neural network (CNN) models for processing image data. The CrisisMMD dataset, which contains both text and image data related to disaster events, will be used for training and testing these models. For the language models, this study will be using Hugging Face Transformers, which is a popular library for working with pre-trained language models such as BERT and GPT Alec et al. (2018). These models have been trained on large amounts of text data and are capable of performing a variety of natural language processing tasks, including text classification. GPT (Generative Pre-trained Transformer) models are designed for generating natural language text, making them well-suited for tasks such as language generation, completion, and summarization. However, they are not as effective in tasks such as classification, which require a deeper understanding of the meaning of the text. In contrast, BERT (Bidirectional Encoder Representations from Transformers) models are designed for deep bidirectional representations of text, making them highly effective for tasks such as natural language understanding and classification. Thus, for classification tasks, BERT models are a better choice than GPT models. This is supported by various studies, including the comparison of BERT and GPT models for text classification by researchers at Google Devlin et al. (2018b).
For the image processing models, this study will be using pre-trained CNN models in PyTorch. These models have been trained on large datasets such as ImageNet and are capable of extracting meaningful features from images. This study will fine-tune these models using the CrisisMMD dataset to classify disaster-related images.
Overall, the methodology involves training separate models for text and image data and then combining the predictions from these models to create a final prediction for each instance in the dataset. This approach allows us to leverage the strengths of both language models and CNN models to improve the accuracy of disaster detection using social media data. The methodology for this system can be broken down into two parts: one for text processing and the other for image processing.
Text Processing Methodology
Data preprocessing The text data from the CrisisMMD dataset will be cleaned and preprocessed, which involves removing any irrelevant information, and punctuation marks, and converting the text to lowercase.
Tokenization The preprocessed text will then be tokenized into individual words or subwords using the tokenizer provided by the hugging face transformers library.
Embedding Next, the tokenized text will be transformed into dense numerical vectors called embeddings using a pre-trained language model from the hugging face transformers library, such as BERT, RoBERTa, DistilBERT, and FinBERT.
Classification The embeddings will be fed into a classification model, such as a feedforward neural network, to predict which disaster type the text is related to.
Figure 1 depicts the pre-training and fine-tuning processes of BERT models. During pre-training, BERT uses a masked language model (MLM) and a next-sentence prediction (NSP) task. MLM randomly masks some of the input tokens and trains the model to predict the original word based on the surrounding context. NSP, on the other hand, aims to determine whether two input sentences are sequential or not. Once pre-training is completed, the model is fine-tuned for specific NLP tasks, such as sentiment analysis, question answering, or disaster detection. Fine-tuning involves adapting the pre-trained BERT model to a specific task by adding a task-specific output layer and fine-tuning the weights on a labeled dataset. This methodology outlines the steps involved in using BERT models for text classification tasks. By leveraging the power of pre-trained BERT models, this study can generate embeddings that capture the contextual information of the text and use them to perform accurate text classification. This approach has shown excellent performance in various natural language understanding tasks and can be applied to a wide range of applications, including disaster response and management.
Fig. 1 [Images not available. See PDF.]
convolutional neural network (CNN) architecture Devlin et al. (2018a)
Image Processing Methodology
Data preprocessing: Data preprocessing refers to the manipulation of a dataset to better suit a specific task. It is crucial for optimizing efficiency and speeding up operations. The techniques used for pre-processing can vary depending on the task at hand. In this case, the data underwent format conversion, normalization, and image resizing to ensure its suitability for the task. The photos were in jpg, jpeg, png, and jfif formats among others. To ensure consistency among the dataset’s image types, each image to the.jpg format has been converted. DenseNet uses photos with a 224 × 224 × 3 input layer, while Inception v3 uses images with a 299 × 299 × 3 input layer. However, because they were downloaded from various sources, the photos have varying dimensions. In order to make the photographs suitable for the models, it should be resized to 224 × 224 × 3 and 299 × 299 × 3 pixels. Additionally, Python Imaging Library (PIL) was used for image preprocessing.
Feature extraction The preprocessed images will then be fed into a pre-trained CNN model, such as Inception V3 or DenseNet, to extract relevant features from the images.
Classification The extracted features will be fed into a fully connected layer which maps each image to its’ corresponding class using SoftMax activation function.
The given diagram, Fig. 2, depicts a basic convolutional neural network (CNN) comprising five distinct layers: input, convolution, pooling, fully connected, and output. These layers can be categorized into two main sections: feature extraction and classification. The feature extraction phase is composed of an input layer, a convolution layer, and a pooling layer, while the classification stage comprises a fully connected layer and an output layer. In this study, the transfer learning method was employed to train the models efficiently. This involved using pre-trained weights from the ImageNet dataset, which were then adjusted based on the new dataset during training to make accurate predictions.
Fig. 2 [Images not available. See PDF.]
convolutional neural network (CNN) architecture V. H. a. R. E. J. Phung (2018)
Dataset Description
We have used the CrisisMMD: Multimodal Crisis Dataset Firoj et al. (2018) Ferda et al. (2004) for the evaluation of NLP and CNN models. The Crisis Multimodal Dataset is a resource maintained by the Qatar Computing Research Institute (QCRI) and can be accessed through the website (https://crisisnlp.qcri.org/crisismmd). It is a collection of data related to real-world crisis events, including natural disasters, conflicts, and other emergencies, and is aimed at promoting research and development in the field of crisis informatics. The dataset includes multiple modalities, such as text and images, and they’re annotated to provide a comprehensive representation of the events and their impact. The Crisis Multimodal Dataset is a valuable resource for researchers, practitioners, and organizations working in the areas of natural language processing, computer vision, machine learning, and crisis informatics. The dataset comprises texts and images extracted from Twitter, predominantly during the disaster period. Initially, it contained 16,058 tweets and 18,082 images, but most of them were irrelevant, including images of news reporters, news channel logos, and unrelated tweets. To filter out such irrelevant data, the dataset have been pre-processed by retaining only the tweets and images that were relevant to disasters. Additionally, the tweets were pre-processed to remove noise and unnecessary information. As shown in Table 2, JSON text file has been converted to CSV for ease of use. To enhance computational efficiency and optimize image classification results, the same type of disasters has been grouped into one class, thereby reducing the overall number of classes. Figure 3 depicts the images from the dataset before and after pre-processing. It can be seen that the raw dataset contains a lot of irrelevant images, which needed to be removed.
Table 2. Tweets (a) before and (b) after pre-processing
Fig. 3 [Images not available. See PDF.]
Dataset images a before pre-processing, b after pre-processing
Table 3 shows the number of tweets and images per class after pre-processing. In order to improve the efficiency of this study analysis and focus on identifying the types of disasters, certain classes of images that were deemed to be redundant or highly similar has been omitted. Specifically, two classes of images related to hurricanes (Irma and Maria) has been excluded, as the images in these classes were found to be almost identical to those in another class (Hurricane Harvey). The decision to exclude these classes was based on the premise that identifying the type of disaster is this study’s primary concern, and having multiple highly similar classes would only complicate the analysis and yield limited additional insights.
Table 3. Number of images and tweets in the Dataset
Class Name | Tweets | Images | |||
|---|---|---|---|---|---|
CaliforniaWildfire | Train: 800 | Test: 250 | Train: 875 | Test: 266 | Validation:110 |
HurricaneHarvey | Train: 2200 | Test: 550 | Train: 800 | Test: 136 | Validation: 100 |
HurricaneIrma | Train: 2000 | Test: 500 | Omitted | Omitted | Omitted |
HurricaneMaria | Train: 2500 | Test: 700 | Omitted | Omitted | Omitted |
MexicoEarthquake | Train: 1500 | Test: 400 | Train: 700 | Test: 148 | Validation: 90 |
SriLankaFlood | Train: 200 | Test: 50 | Train: 650 | Test: 106 | Validation: 90 |
Results and Analysis
This section covers the result analysis of the NLP and CNN models. The 1st subsection talks about the computational details followed by results and interpretations.
Computational Details
TPUs with 25 GB of RAM on Google Colab were used to run several NLP and computer vision models, including BERT-Base-Uncased, DistilBERT-Base-Uncased, Twitter-RoBERTa-Base, GPT-3, Inception v3, and DenseNet. TPUs are specialized hardware accelerators designed for running machine learning workloads and can provide a significant speedup compared to traditional CPUs or GPUs. The Hugging Face Transformers library Wolf et al. (1910) was used for running the NLP models. The tweets first went through a pre-processing pipeline provided in the Transformers library by Hugging Face. The library provides a PreTrained-Tokenizer class that takes care of tokenization, as well as a Basic-Tokenizer class for basic text preprocessing such as lowercasing, removing punctuation, and handling special tokens such as URLs and mentions. For the image classification, the images were trained and evaluated with pre-trained CNN models from the PyTorch framework Paszke et al. (1912). The training, test, and validation set images went through the pre-processing pipeline that normalizes and resizes the images according to each model’s criteria. PyTorch uses ‘bilinear interpolation’ to resize the images. Inception v3 takes an input image size of 299 and DenseNet takes an image size of 224. The model hyperparameters like batch size, learning rate, number of epochs etc. were hand-tuned using trial and error method to give best evaluation scores. Table 4 shows the hyperparameters used to train the models.
Table 4. Hyperparameters for training the models
Model Name | Learning Rate | Batch Size | Epochs | Max Length |
|---|---|---|---|---|
BERT-Base-Uncased | 1e-5 | 32 | 3 | 128 |
DistilBERT-Base-Uncased | 1e-5 | 32 | 3 | 128 |
Twitter-RoBERTa-Base | 1e-5 | 32 | 3 | 128 |
FinBERT | 1e-5 | 32 | 3 | 128 |
Inception v3 | 0.001 | 8 | 15 | N/A |
DenseNet | 0.001 | 8 | 15 | N/A |
Results
To assess the performance of the models on the dataset, four evaluation metrics have been selected, which are accuracy, F1 score, precision, and recall. The method used to calculate the F1 score, precision, and recall. This involves computing the average of the entire dataset. The result tables provided showcase the performance of four different NLP models on a dataset consisting of six different disaster categories. The evaluation metrics used for measuring the models’ performance are accuracy, F1 score, precision, and recall.
Table 5 displays the overall accuracy and time consumption for training and testing each model. The results indicate that all models achieved an accuracy rate of 94%, with DistilBERT-Base-Uncased having the shortest total training and testing times, taking 8,210 and 194 s, respectively.
Table 5. Accuracy and Time comparison of the language models
NLP Model Name | Overall Accuracy | Total Training Time (s) | Total Testing Time (s) |
|---|---|---|---|
BERT-Base-Uncased | 94% | 15,867 | 376 |
DistilBERT-Base-Uncased | 94% | 8,210 | 194 |
Twitter-RoBERTa-Base | 94% | 15,967 | 334 |
FinBERT | 94% | 17,974 | 409 |
Tables 6, 7, and 8 show the precision, recall, and F1 score per class of the NLP models, respectively. The classes in the dataset include California Wildfires, Hurricane Harvey, Hurricane Irma, Hurricane Maria, Mexico Earthquakes, and Sri Lanka Floods. Precision measures the proportion of correctly classified instances within a class, while recall measures the proportion of correctly classified instances among all the instances in the class. The F1 score is a measure of the model’s accuracy that considers both precision and recall.
Table 6. Class-wise precision of the NLP models
Class | Model BERT-Base-Uncased | DistilBERT-Base-Uncased | Twitter-RoBERTa-Base | FinBERT |
|---|---|---|---|---|
CaliforniaWildfire | 1.0 | 1.0 | 1.0 | 1.0 |
HurricaneHarvey | 0.89 | 0.89 | 0.90 | 0.88 |
HurricaneIrma | 0.92 | 0.90 | 0.90 | 0.90 |
HurricaneMaria | 0.94 | 0.96 | 0.96 | 0.94 |
MexicoEarthquake | 1.0 | 1.0 | 0.99 | 0.99 |
SriLankaFlood | 1.0 | 1.0 | 1.0 | 1.0 |
Table 7. Class-wise Recall of the NLP models
Class | Model BERT-Base-Uncased | DistilBERT-Base-Uncased | Twitter-RoBERTa-Base | FinBERT |
|---|---|---|---|---|
CaliforniaWildfire | 1.0 | 1.0 | 1.0 | 0.99 |
HurricaneHarvey | 0.95 | 0.93 | 0.92 | 0.92 |
HurricaneIrma | 0.87 | 0.88 | 0.90 | 0.87 |
HurricaneMaria | 0.96 | 0.96 | 0.95 | 0.96 |
MexicoEarthquake | 0.97 | 0.98 | 0.98 | 0.98 |
SriLankaFlood | 1.0 | 1.0 | 1.0 | 1.0 |
Table 8. Class-wise F1 Score of the NLP models
Class | Model BERT-Base-Uncased | DistilBERT-Base-Uncased | Twitter-RoBERTa-Base | FinBERT |
|---|---|---|---|---|
CaliforniaWildfire | 1.0 | 1.0 | 1.0 | 1.0 |
HurricaneHarvey | 0.92 | 0.91 | 0.91 | 0.90 |
HurricaneIrma | 0.89 | 0.89 | 0.90 | 0.88 |
HurricaneMaria | 0.95 | 0.96 | 0.95 | 0.95 |
MexicoEarthquake | 0.99 | 0.99 | 0.99 | 0.99 |
SriLankaFlood | 1.0 | 1.0 | 1.0 | 1.0 |
In Table 6, all models achieve perfect precision scores for the California Wildfire class, indicating that all models correctly classify all instances of this class. For Hurricane Harvey, the models have precision scores ranging from 0.88 to 0.90, with FinBert having the lowest precision score. For Hurricane Irma, all models achieved similar precision scores, ranging from 0.90 to 0.92. For Hurricane Maria, the models achieved high precision scores ranging from 0.94 to 0.96. Finally, for the Mexico Earthquake and Sri Lanka Flood classes, all models achieved perfect precision scores, indicating that all instances were correctly classified, except Twitter-RoBERTa-Base and FinBERT model for the MexicoEarthquake, which scored 0.99 for both models. The graphical representation of Table 6 can be seen in Fig. 4.
Fig. 4 [Images not available. See PDF.]
Class-wise precision of the NLP models
Table 7 displays the recall scores for each class of models. The graphical representation of Table 7 can be seen in Fig. 5. The results indicate that all models achieved perfect recall scores for the California Wildfire, and Sri Lanka Flood. Similar good results can be seen for Mexico Earthquake also, scores ranging from 0.97 to 0.98. For Hurricane Harvey, the models achieved recall scores ranging from 0.92 to 0.95, with BERT-Base-Uncased achieving the highest recall score. For Hurricane Irma, the models achieved recall scores ranging from 0.87 to 0.90, with Twitter-RoBERTa-Base achieving the highest score. For Hurricane Maria, all models achieved similar recall scores, ranging from 0.95 to 0.96.
Fig. 5 [Images not available. See PDF.]
Class-wise recall of the NLP models
Table 8 displays the F1 score for each class of the models. The graphical representation of Table 8 can be seen in Fig. 6. The results show that all models achieved perfect F1 scores for the California Wildfire, the Mexico Earthquake, and the Sri Lanka Flood. For Hurricane Harvey, the models achieved F1 scores ranging from 0.90 to 0.92, with BERT-Base-Uncased achieving the highest F1 score. For Hurricane Irma, all models achieved similar F1 scores, ranging from 0.88 to 0.90. For Hurricane Maria, the models achieved F1 scores ranging from 0.95 to 0.96.
Fig. 6 [Images not available. See PDF.]
Class-wise F1 score of the NLP models
Table 9 shows the evaluation matrices for the two CNN models, comparing them side by side. Based on the evaluation metrics results presented in Table 9, both the Inception v3 and DenseNet models have performed relatively well in terms of accuracy, F1 score, precision, and recall. Both models compared for the image classification. However, there is a slight difference in performance between the two models. The DenseNet model has slightly outperformed the Inception v3 model in terms of accuracy, F1 score, precision, and recall, with an accuracy of 0.78 compared to 0.76 for Inception v3. Similarly, the F1 score, precision, and recall are slightly higher for DenseNet compared to Inception v3. Therefore, based on these results, it can be concluded that the DenseNet model is slightly better than the Inception v3 model for the task at hand.
Table 9. Evaluation metrics results for the two CNN models, a) Inception v3, b) DenseNet
Evaluation Metrics | Inception v3 | DenseNet |
|---|---|---|
Accuracy | 0.76 | 0.78 |
F1 score | 0.73 | 0.74 |
Precision | 0.74 | 0.75 |
Recall | 0.74 | 0.75 |
From the confusion matrix in Fig. 7, the class-wise accuracy for both models can be seen. Both models can predict California_wildfire pretty accurately, with DenseNet outperforming Inception V3. This is due to the distinctively detailed images of that class. Inception V3 does a better job at identifying hurricane_harvey images, though DenseNet recognizes MexicoEarthquake and SriLankaFlood better than Inception V3. The overall performance of DenseNet is slightly better than Inception V3.
Fig. 7 [Images not available. See PDF.]
a Confusion matrix of Inception v3 b Confusion matrix of DenseNet
Discussion
The aim of this study was to compare the performance of four state-of-the-art Natural Language Processing (NLP) models, namely BERT-Base-Uncased, DistilBERT-Base-Uncased, Twitter-RoBERTa-Base, and FinBERT, on a disaster-related tweet classification task. The evaluation metrics used were accuracy, precision, recall, and F1 score, computed both overall and per class. The results showed that all four models had an overall accuracy of 94%, indicating that they are effective in identifying disaster-related tweets. However, when examining the class-wise metrics, the models’ performance varied. The models achieved perfect precision and recall for the California Wildfire and Sri Lanka Flood classes, which suggests that they are proficient in identifying these events. However, for other classes, such as Hurricane Harvey and Hurricane Irma, the DistilBERT-Base-Uncased and Twitter-RoBERTa-Base models outperformed the BERT-Base-Uncased and FinBERT models. In contrast, for the Hurricane Maria and Mexico Earthquake classes, the BERT-Base-Uncased and DistilBERT-Base-Uncased models had higher precision and recall. In terms of training and testing time, the DistilBERT-Base-Uncased model had the lowest training time of 8,210 s, while the BERT-Base-Uncased model had the highest training time of 15,867 s. For testing time, the DistilBERT-Base-Uncased model had the lowest testing time of 194 s, while the FinBERT model had the highest testing time of 409 s. Overall, the results suggest that the choice of the NLP model is dependent on the specific disaster event being classified. While all models performed well overall, some models were better suited for specific classes. It is important to note that the training and testing time varied between the models. The DistilBERT-Base-Uncased model had the lowest training and testing time, making it a more computationally efficient choice. However, the BERT-Base-Uncased model had the highest training time, suggesting that it may be better suited for larger datasets or more complex tasks. The results of this study suggest that the selection of an NLP model should be based on the specific disaster event being classified, as well as computational resources and training time considerations. Further research could explore other NLP models or modifications of existing models to improve their performance on disaster-related tweet classification tasks.
In this study, the evaluated two CNN models, Inception v3 and DenseNet, for the task of image classification. The models were trained using the transfer learning approach and tested on a test set. The results presented in Table 9 indicate that both models performed relatively well in terms of accuracy, F1 score, precision, and recall.
The DenseNet model has shown slightly better performance compared to the Inception v3 model. This can be attributed to the architecture of the DenseNet model, which has shown better results in previous studies for similar tasks. The DenseNet model has a unique architecture that allows for feature reuse and gradient flow, which helps in better feature representation and information flow. Despite the relatively good performance of both models, there is still room for improvement. Further fine-tuning of the model’s hyperparameters, such as the learning rate, batch size, and a number of epochs, may lead to better performance. Moreover, exploring other pre-trained models or designing a new architecture from scratch may also yield better results. Additionally, the size and quality of the dataset can have an impact on the performance of the models. Using a larger and more diverse dataset may help in improving the models’ accuracy and robustness. In conclusion, this study highlights the effectiveness of transfer learning in image classification tasks. The DenseNet model has shown slightly better performance compared to the Inception v3 model, indicating the importance of selecting an appropriate architecture for the task at hand. However, further research is needed to explore other models and hyperparameters to improve the performance of these models.
There is another stream of research that complex models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) like Long Short-Term Memory (LSTM) and Bidirectional Gated Recurrent Units (Bi-GRU), and different word embedding techniques (e.g.. Word2Vec, Glove) to improve the understanding of public opinions articulated through text data (Pimpalkar and J. R. R. R, 2022; Ghorbani et al. 2020; Başarslan and Kayaalp 2023). The goal of those research was to evaluate textual data using sophisticated machine learning and deep learning techniques, especially for sentiment analysis and opinion classification in a variety of fields. They encompassed hybrid approaches and strategies to get better outcomes compared to what might have been obtained from each strategy alone. A hybrid technique that merges text and image data may improve the performance while utilizing the advantages of both modalities. However, the emphasis of those studies is not real-time applicability instead to deal with static information. Another thing is the investigation was restricted to a particular dataset (e.g. IMDB reviews). Additionally, computational efficiency is not heavily stressed in those works as well as they concentrate on a particular model design. Lastly, those three models are limited due to their unique datasets and cases.
To compensate for this gap, in this study the efficacy of several convolutional neural networks (CNN) and natural language processing (NLP) models have been compared for disaster-type recognition employing social networking data rather than combining them into a hybrid strategy. This study’s concentration was emergency and crisis management which works with real time datasets, demonstrating the models’ versatility and adaptability in many contexts. Additionally, it provides a more thorough performance evaluation of the models and the model is flexible enough to be modified for use on other platforms in the event of a disaster.
This study assessed the performance evaluation of NLP models like Twitter-ROBERTa-Base, FinBERT. DisIBERT-Base-Uncased, and BERT-Base-Uncased to analyze textual data. Simultaneously, this study examined the visual data processing capabilities of two CNN models DenseNet and Inception v3. The main goal of this study was to use the CrisisMMD dataset to test each model’s performance individually in terms of accuracy and processing time. The aim in conducting this comparison analysis was to identify the particular model or method that works best for disaster- type recognition.
Implications and Conclusion
Management of emergencies and crises is still a difficult fact on a global scale. It is essential to have access to real-time data sources and to have the capacity to rapidly gather, process, synthesize, and analyze this data in order to enable better decisions to be made under pressure in close to real-time. The appropriate use of insufficient resources by rescue workers, humanitarian groups, governments, and utility organizations is made possible by understanding which data is available via social networking sites. Recent studies have focused on identifying posts on social media that seem to be relevant to disasters to better determine the scope and severity of damage as well as how to clearly assess impacted people, increase situational awareness, and organize disaster recovery. This study presented two CNN models and four neural language processing models on the Twitter network dataset. These models can ingest and analyze Twitter data in real time to assist humanitarian organizations in better understanding the severity of a crisis.
The goal was to evaluate the performance of the NLP and CNN models of the dataset using four evaluation metrics. According to experimental data, the accuracy of three different NLP models was 94%, however, DistilBERT-Base-Uncased had the shortest training and testing times. In addition, DistilBERT-Base-Uncased once again showed the best performance for precision, with the exception of Hurricane Irma, for which BERT-Base-Uncased had the best precision. Furthermore, BERT-Base-Uncased and DistilBERT-Base-Uncased did well with Hurricane Maria in terms of recall. While Twitter-RoBERTa-Base and DistilBERT-Base-Uncased did well for the Mexico earthquake. Consequently, all three performed best for the Sri Lanka Flood and California wildfires. Last but not least, each model did well for the various sorts of calamity in terms of F1-Score. Nevertheless, the difference was not greater than 1%.
The evaluation measures for the CNN model didn’t yield many encouraging findings. Regarding accuracy, recall, precision, and the F1 score, which stayed within the range of 70–78%. But between the two models, DenseNet dominated in terms of time and accuracy, taking 3 h to train compared to Inception v3 which took 5 h with the same computational power. The following is the model’s main implications and contribution:
This study can help us understand how social media data can be used to evaluate the severity of a disaster in real-time.
It also demonstrates that neural language processing models produce results that are more accurate than convolutional neural network models when assessing social media data.
It also states that DistilBERT-Base-Uncased produced the fastest results, and BERT-Base-Uncased produces the most promising results out of all the four language processing models.
The CNN model, DenseNet, shows the most promising results.
This study will assist humanitarian organizations in taking immediate action to address the post-disaster problem and provide aid appropriately.
The limitations of the current research in this study were tested using the Twitter dataset. As of now, the API on other platforms like Facebook, Instagram, and WhatsApp doesn’t include the extraction of the dataset, but it’s likely this study can use these models for text classification on those platforms as well. So, this research leaves some avenue for future direction. Machine learning can be utilized to the NLP model which can help identify each individual tweets using its feature extraction whether if it is about hurricane, floods, or wildfire. Combining machine learning and deep learning can improve the accuracy of disaster identification. As multi-model approaches have the grasp of more holistic overview of the model. Though the deep learning-based model in our study is originally strong. The NLP based or machine learning based model can balance efficiency and time which is really necessary in real time disaster management. Also using the machine learning and deep learning into our research can handle noisy and unstructured texts or complex images, make the model more robust. Additionally, machine learning (ML) can serve as an interpretable classifier while deep learning (DL) can figure out the complicated structure an put those output to the ML model. This leads to a precise decision-making ability to manage complex pattern and improving accuracy. Furthermore, the real time disaster-type recognition has a key aspect of time complexity. The combination of ML and DL can offer a much faster approach on detecting disaster. The ML model provides a faster interference time with lower time complexity, on the other hand, DL model can offer better understanding on complicated data despite having higher time complexity. So, strategically combining the model can optimize and increase the accuracy as well. In conclusion, The ML model can handle the metadata and DL model can process the raw data. The combination can analyze vast amount of social media data in real-time and aid in prompt disaster making.
Acknowledgements
The authors would like to thank Arkansas State University for their support.
Author Contributions
[AI] was responsible for [Conceptualization, Methodology, Dataset Description, Result and Analysis]. [FR] was responsible for [Introduction, Literature Review, Discussion, Result and Analysis, Implication and Conclusion]. [NUIH] was responsible for [Conceptualization, Methodology, Result and Analysis, Supervision, Reviewing, and Editing] of the whole study. All authors read and approved the final manuscript.
Funding
This research received no external funding.
Data availability
Data will be provided upon request.
Declarations
Conflict of interest
The authors declare that they have no competing interests.
Ethical approval
This article does not contain any studies of human participants or animals performed by any of the authors.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Aipe A , Mukuntha N , Ekbal A, and Kurohashi S, (2018) Deep learning approach towards multi label classification of crisis related tweets, Proc. 15th Int. Conf. Inf. Syst. Crisis Response Manage, Rochester, NY
Alam F, Imran M, and Ofli F, ( 2017) “Image4Act: Online Social Media Image Processing,” in International Conference on Advances in Social Networks Analysis and Mining,.
Alam F, Ofli F, and Imran M,(2018) “Processing social media images by combining human and machine computing,” Int. J. Hum. Comput. Interact, vol. 34, no. 4, pp. 311–327, .
Alec R, Karthik N, Tim S, and Ilya S,(2018) “Improving language understanding by generative pre-training,” OpenAI,
S. N. K. B. Amit and Y. Aoki, “Disaster Detection from Aerial Imagery with Convolutional Neural Network,” in 2017 International Electronics Symposium on Knowledge Creation and Intelligent Computing (IES-KCIC), 2017.
S. N. K. B. Amit, S. Shiraishi, T. Inoshita and Y. Aoki, “Analysis of satellite images for disaster detection,” in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2016.
Z. Ashktorab, C. Brown, M. Nandi and A. Culotta., “Tweedr: Mining twitter to inform disaster response,” in ISCRAM., 2014.
Z. Ashktorab, C. Brown, M. Nandi and A. Culotta, (2014) “Tweedr Mining Twitter to Inform Disaster Response,” in Proceedings of the 11th International ISCRAM Conference – University Park
M. S. Başarslan and F. Kayaalp, “MBi-GRUMCONV: A novel Multi Bi-GRU and Multi CNN-Based deep learning model for social media sentiment analysis,” Journal of Cloud Computing: Advances, Systems and Applications, vol. 12, no. 5, 2023.
Beig, G; Hu, X; Maciejewski, R; Liu, H. An overview of sentiment analysis in social media and its applications in disaster relief. Studies in Computational Intelligence; 2016; 639, pp. 313-340.
Bischke B, P. Bhardwaj, A. Gautam, P. Helber, D. Borth and A. Dengel, “Detection of Flooding Events in Social Multimedia and Satellite Imagery using Deep Neural Networks,” in MediaEval’17, 2017.
Brockman G, M. Murati, P. Welinder and OpenAI, “OpenAI API,” OpenAI.com, [Online]. Available: [Accessed June 11, 2020].
Castillo C, Big Crisis Data: Social Media in Disasters and Time-critical Situations, Cambridge University Press., 2016.
Chanda A. K , (2021)“Efficacy of BERT embeddings on predicting disaster from Twitter data,” in Archive, .
Corvey W. J., Vieweg S., Rood T. and Palmer M.,(2010) “Twitter in Mass Emergency: What NLP Techniques Can Contribute,” in Proceedings of the NAACL HLT 2010 Workshop on Computational Linguistics in a World of Social Media,
Devlin J., Chang M.-W., Lee K. and Toutanova K.,(2018) “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Computation and Language, .
Devlin J., Chang M.-W., Lee K.and Toutanova K., (2018)“BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” ArXiv. /abs/1810.04805, .
Devlin J. , Chang M., Lee K.and Toutanova K., (2019) “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Dharini R. and Parvathi R., (2021)“A novel domain and event adaptive tweet augmentation approach for enhancing the classification of crisis related tweets,” Data \& Knowledge Engineering, vol. 135, p. 101913, .
Dhongade A., Thorat A., Alone D., Sawant S. and Joshi A.,(2023.) “Flood Damage Detection Using Satellite Images,” in International Conference on Advancements in Interdisciplinary Research,
Erik C. and Bebo W., (2014)“Jumping NLP Curves: A Review of Natural Language Processing Research,” IEEE Computational Intelligence magazine,.
Ferda O., Firoj A. and Muhammad I.,(2020) “Analysis of social media data using multimodal deep learning for disaster response,” arXiv preprint arXiv:2004.11838, .
Firoj A., Ferda O. and Muhammad I.,(2018) “Crisismmd: Multimodal twitter datasets from natural disasters,” Proceedings of the international AAAI conference on web and social media, vol. 12, .
Ghorbani M., Bahaghighat M., Xin Q.and Özen F.,(2020.) “ConvLSTMConv network: a deep learning approach for sentiment analysis in cloud computing,” Journal of Cloud Computing: Advances, Systems and Applications, vol. 9, no. 16,
Hartawan D. R., Purboyo T. W. and Setianingsih C. , (2019)“Disaster Victims Detection System Using Convolutional Neural Network (CNN) Method,,” in 2019 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), .
Hossain, NUI; Fattah, MA; Morshed, SR; Jaradat, R. Predicting land cover driven ecosystem service value using artificial neural network model. Remote Sensing Applications: Society and Environment; 2024; 34, [DOI: https://dx.doi.org/10.1016/j.rsase.2024.101180] 101180.
Huang G., Liu Z.,Weinberger K. Q. and Maaten L. v. d., “Densely Connected Convolutional Networks,(2016)” arXiv:1608.06993,
Ishino A.,Odawara S. , Nanba H.and Takezawa T., ( 2012)“Extracting Transportation Information and Traffic Problems from Tweets during a Disaster Where do you evacuate to?,” in IMMM 2012, The Second International Conference on Advances in Information Mining and Management ,.
Islam M. A., Rashid S. I.,. Hossain N. U. I, Fleming R. and Sokolov A. ,(2023) “An integrated convolutional neural network and sorting algorithm for image classification and efficient flood disaster management,” Decision Analytics Journal, .
Jaeger, PT; Shneiderman, B; Fleischmann, KR; Preece, J; Qu, Y; Wu, PF. Community response grids: E-government, social networks, and effective emergency management. Telecommunications Policy; 2007; 31,
Jing M., Scotney B. W., Coleman S., McGinnity T. M. , Kelly S.,Zhang X. , Ahmad K., Schlaf A., Grunder-Fahrer S. and Heyer G., (2016)“Flood Event Image Recognition via Social Media Image and Text Analysis,” .
M. M. Khan, I. Bashar, G. M. Minhaj, A. I. Wasi and N. U. I. Hossain, “Resilient and sustainable supplier selection: an integration of SCOR 4.0 and machine learning approach,” Sustain. Resilient Infrast, 2023.
B. Klein, F. Castanedo, I. Elejalde, D. López-de-Ipiña and A. P. Nespral, “Emergency Event Detection in Twitter Streams Based on Natural Language Processing,” in Ubiquitous Computing and Ambient Intelligence. Context-Awareness and Context-Driven Interaction.
H. Larissa and B. Jean, “Intimate banalities: The emotional currency of shared camera phone images during the Queensland flood disaster,” The Routledge companion to mobile media, pp. 499–513, 2014.
X. Li, H. Zhang, D. Caragea and M. Imran, “Localizing and quantifying damage in social media images.,” in Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)., Barcelona, Spain,, 2018.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer and V. Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” Computation and Language, 2019.
G. Lotan, “All shook up: mapping earthquake news on Twitter from Virginia to Maine,” SocialFlow, 2011.
M. Lukauskas, V. Pilinkiene, J. Bruneckiene, A. Stundžiene, A. Grybauskas and T. Ruzgas, “Economic Activity Forecasting Based on the Sentiment Analysis of News,” mathematics, vol. 10, no. 3461, 2022.
H. Mouzannar, Y. Rizk and M. Awad, “Damage identification in socialmedia posts using multimodal deep learning,” in Proceedings of the 15th International Conference on Information Systems for Crisis Response and Management (ISCRAM)., Rochester, NY,, 2018.
G. Neubig, Y. Matsubayashi, M. Hagiwara and K. Murakami, “Safety Information Mining — What can NLP do in a disaster,” in Proceedings of the 5th International Joint Conference on Natural Language Processing, 2011.
S. Palaniappan, Y. D, P. P and S. Swaminathan, “Deep attention based optimized Bi-LSTM for improving geospatial data ontology,” Data \& Knowledge Engineering, vol. 144, p. 102123, 2023.
A. Paszke, F. M. Sam Gross, J. B. Adam Lerer, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani and S. Chilamkurthy, “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” preprint arXiv:1912.01703, 2019.
Pekar, V; Binner, J; Najafi, H; Hale, C; Schmidt, V. Early Detection of Heterogeneous Disaster Events Using Social Media. J. Association Info Sci Techno; 2020; 71,
A. Pimpalkar and J. R. R. R, “MBiLSTMGloVe: Embedding GloVe knowledge into the corpus using,” Expert Systems With Applications, vol. 203, p. 117581, 2022.
Rabbi, F; Banik, D; Hossain, NUI; Sokolov, A. Using Process Mining Algorithms for Process Improvement in Healthcare. Healthcare Analytics; 2024; 5, [DOI: https://dx.doi.org/10.1016/j.health.2024.100305]
F. Rabbi, N. U. I. Hossain and N. Sadik, “Twitter Network Analysis of Industry 5.0 Leveraging Graph Theory Simulation Approach,” in IISE Annual Conference and Expo, New Orleans, LA, 2023.
F. Rabbi, N. U. I. Hossain and O. Ashour, “Predicting Management Skills of Undergraduate Students by Leveraging Different Machine Learning Classifiers,” in Proceedings of the IISE Annual Conference & Expo 2024 at , Montreal, Canada, 2024.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI blog, 2019.
T. Rajapakse, “To Distil or Not To Distil: BERT, RoBERTa, and XLNet,” Towards Data Science, [Online]. Available:. [Accessed February 08 2023].
Sakaki, T; Okazaki, M; Matsuo, Y. Tweet analysis for real-time event detection and earthquake reporting system development. IEEE Trans Knowl Data Eng; 2013; 25,
Z. SHAHBAZI and Y.-C. BYUN, “Blockchain-Based Event Detection and Trust Verification Using Natural Language Processing and Machine Learning,” IEEE Access, vol. 10, 2022.
Shaukat, K; Luo, S; Varadharajan, V. A novel method for improving the robustness of deep learning-based malware detectors against adversarial attacks. Eng Appl Artif Intell; 2022; 116, [DOI: https://dx.doi.org/10.1016/j.engappai.2022.105461]
Shaukat, K; Luo, S; Varadharajan, V. A novel deep learning-based approach for malware detection. Eng Appl Artif Intell; 2023; 122, [DOI: https://dx.doi.org/10.1016/j.engappai.2023.106030]
Shaukat, K; Luo, S; Varadharajan, V. A novel machine learning approach for detecting first-time-appeared malware. Eng Appl Artificial Intelligence; 2024; 131, [DOI: https://dx.doi.org/10.1016/j.engappai.2023.107801]
K. Shaukat, A. Rubab, I. Shehzadi and R. Iqba, “A Socio-Technological analysis of Cyber Crime and Cyber Security in Pakistan,” Transylvanian Review, vol. 1, 2017.
K. Shaukat, F. Iqbal, T. M. Alam and G. K. Aujla (2020) “The Impact of Artificial intelligence and Robotics on the Future Employment Opportunities,” Trends in Computer Science and Information Technology 5 1 050 054
K. Shaukat, T. M. Alam, S. Luo, S. Shabbir, I. A. Hameed, J. Li, S. K. Abbas and U. Javed, “A Review of Time-Series Anomaly Detection Techniques: A Step to Future Perspectives,” in Advances in Information and Communication, 2021.
K. Shaukat, T. M. Alam, I. A. Hameed, W. A. Khan, N. Abbas and S. Luo, “A Review on Security Challenges in Internet of Things (IoT),” in 2021 26th International Conference on Automation and Computing (ICAC), 2021.
Sit, MA; Koylu, C; Demir, I. Identifying disaster-related tweets and their semantic, spatial and temporal context using deep learning, natural language processing and spatial analysis: a case study of Hurricane Irma. Int J. Digital Earth; 2019; 12,
Sufi F. K., (2021)“AI-Landslide: Software for acquiring hidden insights from global landslide data using Artificial Intelligence,” Software Impacts, vol. 10, .
Sufi F. K. and Khalil I.,(2022) “Automated Disaster Monitoring from Social Media Posts using AI based Location Intelligence and Sentiment Analysis,” IEEE Transactions on Computational Social Systems (preprint), .
Szegedy C., Vanhoucke V., Ioffe S., Shlens J. and Wojna Z.,(2015) “Rethinking the Inception Architecture for Computer Vision,” arXiv:1512.00567, .
Tariq U., Ahmed I., Bashir A. K. and Shaukat K. , (2023)“A Critical Cybersecurity Analysis and Future Research Directions for the Internet of Things: A Comprehensive Review,” sensors, vol. 23, no. 8, p. 4117, .
Tom, B; Benjamin, M; Nick, R; Melanie, S; Jared, K; Prafulla, D; Arvind, N; Pranav, S; Girish, S; Amanda, A. Language models are few-shot learners. Adv Neural Inf Process Syst; 2020; 33, pp. 1877-1901.
V. H. a. R. E. J. Phung, (2018)“A Deep Learning Approach for Classification of Cloud Image”. Journal of information and communication convergence engineering 16 173--178
Vaswani A., Shazeer N. , Parmar N, Uszkoreit J. ,Jones L. , Gomez A. , Kaiser Ł. and Polosukhin I.,(2017) “Attention is all you need,” in Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, .
Verma S. , Vieweg S.,Corvey W. J. , Palen L., Martin J. H., Palmer M., Schram A. and Anderson K. M. ,( 2011.) “Natural Language Processing to the Rescue? Extracting “Situational Awareness” Tweets During Mass Emergency,” in Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media,
Vigna F. D. , Cresci S., Marchetti A. , Tesconi M. and Avvenuti M., (2015.)“Pulling Information from Social Media in the Aftermath of Unpredictable Disasters,” in PUblication MAnagement.
Wang, Z; Ye, X; Tsou, M-H. Spatial, temporal, and content analysis of twitter for wildfire hazards. Nat Hazards; 2016; 83,
Wang Y. , Jaradat R., Wang H. and Ibne Hossain N. U., ( 2024.)“Multi-filed data fusion through attention-based networks for readiness prediction in aircraft maintenance: natural language processing (NLP) approach.,” International Journal of Management Science and Engineering Management, pp. 1–11,
T. Wolf, L. Debut, V. Sanh, J. Chaumond, A. M. Clement Delangue, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. v. Platen, C. Ma, Y. Jernite, J. Plu, C. Xu and T. L. Scao, “HuggingFace’s Transformers: State-of-the-art Natural Language Processing,” preprint arXiv:1910.03771, 2019.
X. Xia, C. Xu and B. Nan, (2017) “Inception-v3 for Flower Classification,” in 2017 2nd International Conference on Image, Vision and Computing.
J. Yin, S. Karimi, B. Robinson and M. Cameron., “ESA: Emergency situation awareness viamicrobloggers,” in Proceedings of the 21st ACM International Conference on Information and Knowledge Management, New York, 2012.
Yu, M; Huang, Q; Qin, H; Scheele, C; Yang, C. Deep learning for real-time social media text classification for situation awareness– using Hurricanes Sandy, Harvey, and Irma as case studies. Int J. Digital Earth; 2019; 12,
Yu H., Ma Y.,Wang L. , Zhai Y.and Wang X.,(2017) “A landslide intelligent detection method based on CNN and RSG_R,” in 2017 IEEE International Conference on Mechatronics and Automation (ICMA), .
Zhou B. , Zou L., Mostafavi A., Lin B. , Yang M.,Gharaibeh N. , Cai H., Abedin J. and Mandal D. , (2022)“VictimFinder: Harvesting rescue requests in disaster response from social media with BERT,” Computers, Environment and Urban Systems,.
Zhu Y. and Newsam S., ( 2017)“DENSE NET FOR DENSE FLOW,” in IEEE international conference on image processing (ICIP),.
Copyright Springer Nature B.V. Dec 2024